January 2010 - Darwin-dev - lists.apple.com

Re: running ditto from firefox on snow leopard
by Damien Sorresso 27 Jan '10

27 Jan '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jan 27, 2010, at 12:21 AM, Joel Reymont wrote: > Damien, > > On Jan 27, 2010, at 7:44 AM, Damien Sorresso wrote: > >> By doing this, you're basically guaranteeing that Firefox can never participate in the code signing model. > > Firefox download partial updates right now, including binary code. It kicks off the updater once the download is done and updates itself. This prevents Firefox from participating in the code signing model so compression would not be much different. With a partial update, you can update the code signature. You are talking about generated a new executable at launch-time and exec(3)ing that. That is different. Regardless of where it lives, the code that's executing will not be signed. > What are the huge benefits to participating in code signing? You attach a reliable identity to a piece of code. >> Maybe you don't care right now, but then again, maybe one day you'll want to use the Keychain APIs. > > Is code signing a must to use the Keychain APIs? Can you please clarify? You can update your application without the Keychain nagging the user that it has changed (as long as the signature is intact). Also, the ServiceManagement framework uses code signing to establish a handshake between an unprivileged application and a privileged helper tool that it's trying to install. If Firefox gets a launch-time benefit from compression, it implies that Firefox's executable is structured in such a way that, when the kernel maps it in, it has to do a lot more work upfront than is normal. The overhead of decompressing the executable should exceed that of just launching it directly. After all, the HFS compression stuff was developed for files that are infrequently accessed, not hot files. -- Damien Sorresso BSD Engineering Apple Inc. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: running ditto from firefox on snow leopard
by Damien Sorresso 27 Jan '10

27 Jan '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jan 26, 2010, at 11:47 PM, Joel Reymont wrote: > On Jan 27, 2010, at 7:44 AM, Damien Sorresso wrote: >> By doing this, you're basically guaranteeing that Firefox can never participate in the code signing model. Maybe you don't care right now, but then again, maybe one day you'll want to use the Keychain APIs. > > I don't think it's a concern for now. That said, what is the best option of compressing Firefox on Snow Leopard? Ditto launched from a separate app using Launchd APIs? The launchd APIs you're referring to are undocumented, and we generally discourage their use for anything but checking in to get a socket file descriptor. -- Damien Sorresso BSD Engineering Apple Inc. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: running ditto from firefox on snow leopard
by Shantonu Sen 27 Jan '10

27 Jan '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com It sounds like you have all the tools to resolve this, I'm not sure what input you are requesting from the low-level Darwin OS community. If Firefox already has a mechanism for self-patching using binary deltas, it must have already solved the authentication issue. Self-patching should be enhanced to use ditto --hfsCompression on each file after the patch is applied on 10.6. Future versions of the OS may or may not support similar technology (that's why there's no API for filesystem-level compression). On first launch, Firefox could detect it's on 10.6 and synthesize a phantom patchset that applies to every file with 0 diffs, which should have the side effect of compressing files afterward. However, if your launch time is really affected by this in a substantial way, there are probably things you can do to improve the footprint without resorting to filesystem compression, like using order files, or making sure you put read-only data in the text segment so that it only gets paged in on demand. Also, note that filesystem-level compression is HFS+-specific, and will not work on other filesystems, and indeed since it is a system implementation detail, may not work in the way you expect it to in the future. Shantonu Sen ssen(a)apple.com Sent from my Mac Pro On Jan 27, 2010, at 1:03 AM, Jean-Daniel Dupas wrote: > > Le 27 janv. 2010 à 09:34, Joel Reymont a écrit : > >> >> On Jan 27, 2010, at 8:29 AM, Jean-Daniel Dupas wrote: >> >>> wrong, partial update do not prevents codesign. Especially as the partial update can contains the updated code signature. >> >> Can partial updates preserve compression on Snow Leopard? >> >> How does Apple do this? >> >> In other words, can we deliver a compressed Firefox for Snow Leopard -and- deliver compressed updates as well? I gather that the objection is to compress Firefox post-installation. >> > > Apple uses pkg to distribute their software and their updates. They don't have to support system older than the target one (as they distribute one package by major version if when needed), and the pkg format supports HFS compression (a class-dump of the private PackageKit reveal a useHFSPlusCompression property in PKPackageInfo), but as I avoid pkg as much as I can, I don't know if this is a public features and how to use it. > > > -- Jean-Daniel > > > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-dev/ssen%40apple.com > > This email sent to ssen(a)apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: running ditto from firefox on snow leopard
by Jean-Daniel Dupas 27 Jan '10

27 Jan '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Le 27 janv. 2010 à 09:34, Joel Reymont a écrit : > > On Jan 27, 2010, at 8:29 AM, Jean-Daniel Dupas wrote: > >> wrong, partial update do not prevents codesign. Especially as the partial update can contains the updated code signature. > > Can partial updates preserve compression on Snow Leopard? > > How does Apple do this? > > In other words, can we deliver a compressed Firefox for Snow Leopard -and- deliver compressed updates as well? I gather that the objection is to compress Firefox post-installation. > Apple uses pkg to distribute their software and their updates. They don't have to support system older than the target one (as they distribute one package by major version if when needed), and the pkg format supports HFS compression (a class-dump of the private PackageKit reveal a useHFSPlusCompression property in PKPackageInfo), but as I avoid pkg as much as I can, I don't know if this is a public features and how to use it. -- Jean-Daniel _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: running ditto from firefox on snow leopard
by Joel Reymont 27 Jan '10

27 Jan '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=IWFLperz6J17X5GUCeLcP2GOAbBvTSjhZdiDTP/5mSk=; b=MTQBC5B7mzNJfP+07ytbpB5wx5bvP6kKVCoWBP4FKxpX4ceDCuaIeJ/yHbmqiuDtDj c2khOCPy3KxXN30gou27TsWTcttR/dJ4dec/dhYlnNezFl2N2/aIFSbwQ6z7rXcuFElN wwxx8xhuMhfD78j2kW1zvfVxhD+PriPhnP77Y= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=bTAIjafKtYWDNCrPtctqAW7M1mH5dQcWVinshzSMitp9PfuCqSIOSB5mF/wnReN2XU AkGunaeeJM3i+kccMWtcUVVwoDucuZm8iJ3Nkav2YtE4mJs3lNdJRorYWFmLuB0MB/8C xeN4RxduCQBKmGXWCM42G1a2kefTnkv/0Wl0Y= On Jan 27, 2010, at 8:29 AM, Jean-Daniel Dupas wrote: > wrong, partial update do not prevents codesign. Especially as the partial update can contains the updated code signature. Can partial updates preserve compression on Snow Leopard? How does Apple do this? In other words, can we deliver a compressed Firefox for Snow Leopard -and- deliver compressed updates as well? I gather that the objection is to compress Firefox post-installation. --- http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: running ditto from firefox on snow leopard
by Jean-Daniel Dupas 27 Jan '10

27 Jan '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Le 27 janv. 2010 à 09:21, Joel Reymont a écrit : > Damien, > > On Jan 27, 2010, at 7:44 AM, Damien Sorresso wrote: > >> By doing this, you're basically guaranteeing that Firefox can never participate in the code signing model. > > Firefox download partial updates right now, including binary code. It kicks off the updater once the download is done and updates itself. This prevents Firefox from participating in the code signing model so compression would not be much different. > wrong, partial update do not prevents codesign. Especially as the partial update can contains the updated code signature. http://developer.apple.com/mac/library/documentation/Security/Conceptual/Co… > What are the huge benefits to participating in code signing? > >> Maybe you don't care right now, but then again, maybe one day you'll want to use the Keychain APIs. > > Is code signing a must to use the Keychain APIs? Can you please clarify? > Please, read the documentation: http://developer.apple.com/mac/library/documentation/Security/Conceptual/Co… -- Jean-Daniel _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: running ditto from firefox on snow leopard
by Joel Reymont 27 Jan '10

27 Jan '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=pamr3rOoVAkdkNQycrrBF++9TBS60fJTWnLrVcvjaCo=; b=fh8fzTJwe+2MGe6JzjM+O7aLln0vlHt4+wnHo8aaTm0nGPDteGANYRPDwRUV/5pobZ qZwZEhbA9Lb4QG+n8HUd2YxP6p1HJvPCdMowbnGAW4dXvkebrF2fqQHUexR82TE964E4 hcajMcaslmuJ8xXzaz3pw++xAJWeP6bVKO6bY= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=tduoe8mYSgpd1DfLSJfCf0JWTtI7YB2Jg6H/sR/TQcOEx76Z9+dR2MTL+K4w/vOOPy 429bDnquu1u9/hJ+qVNWtlWDB96xyYkOl6xiMGe4Ucar+1UMXgaUexZMn6Kh2BCvYzJ3 Ur5Jl4VhAKfw3ggjqzEWuHO/+XRw4c1LHhD+Y= Damien, On Jan 27, 2010, at 7:44 AM, Damien Sorresso wrote: > By doing this, you're basically guaranteeing that Firefox can never participate in the code signing model. Firefox download partial updates right now, including binary code. It kicks off the updater once the download is done and updates itself. This prevents Firefox from participating in the code signing model so compression would not be much different. What are the huge benefits to participating in code signing? > Maybe you don't care right now, but then again, maybe one day you'll want to use the Keychain APIs. Is code signing a must to use the Keychain APIs? Can you please clarify? Thanks, Joel --- http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: running ditto from firefox on snow leopard
by Joel Reymont 27 Jan '10

27 Jan '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=6n17pbc4aL+yHZl3/zPGGZvXssEIDuuqgYABDghKguQ=; b=qYydilpOXy8Yq1eRIoEqGgXf57hua4GT77b2aML5mrEi6KekqdnOU/0leDxPhADFPz PsXiF1iNe3mAsMoRwvhWC65k/FBvLk5CNXJ473w/jmnlr2MUTbCEkIZgsKFvJMqHsCT1 9epv0BXZkU2s0aiIsd4cnenZJQuRJY568qOlo= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=yHiSkee58Q122beRVAn4QTMJa9NSxuIeEeu2xQmuwdoduH7lXLnMoCRtB3804fxVYO c976miL3nEl0fyUMBOHszcaod4WVODvabn+0ERumQhCj2t+MrwMw6N2zYa5LEAzY2Z7o LVfmeZa90B1SFK2IKjyiZNjWpOW9alHTek+Bc= On Jan 27, 2010, at 7:56 AM, Jean-Daniel Dupas wrote: > And now, if someone ask you, you can tell him you know at least one person. I have all my downloaded applications in a separate partition shared by both Leopard and Snow Leopard. Thanks for pointing it out. I'll make sure compression is optional then. > As a side note, the user that first launch Firefox may not be the one that install it and so, do not have the right to change it. I'll add a pointer to this thread to the bug. I think we still want compression wherever we can. Thanks, Joel --- http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: running ditto from firefox on snow leopard
by Jean-Daniel Dupas 27 Jan '10

27 Jan '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Le 27 janv. 2010 à 08:21, Joel Reymont a écrit : > > On Jan 26, 2010, at 10:40 PM, Shantonu Sen wrote: > >> Let me clarify. Modifying yourself is probably pretty dangerous, > > So I'll have to 'schedule' the move, exit and do it, or something like that. > >> and may cause problems if you need to share the app with an older system. > > The assumption is that you are running on Snow Leopard. I don't know anyone who shares Firefox with an older system, usually people just download it. > And now, if someone ask you, you can tell him you know at least one person. I have all my downloaded applications in a separate partition shared by both Leopard and Snow Leopard. As a side note, the user that first launch Firefox may not be the one that install it and so, do not have the right to change it. And as Damien noted, updating an application content breaks signature. -- Jean-Daniel _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: running ditto from firefox on snow leopard
by Joel Reymont 27 Jan '10

27 Jan '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=TqoE2+3aIrppqHyPvgEu5LTOQagQZ8xMIgNyB21PYnM=; b=Hf7Z0+UYZoPol5Vw9dpO2qPIMlrcoW7jWfm1YeqtV1lOSH0rcTbfWBnVJZFlYdkcU5 hcfgg/Yb3qIPwPM8HgEOFWCSehmy8ueSMISk4LVn2fHCVAVz4UFxsG4/HYTKrFG7Kl7D 0YfMQ57AGOZ+vqrYEwqxaAtvH2CVY6uS076Rg= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=o8acx1AEgVPpcb3vKl1tu0z2gMSj0ywd6EW/aq7eJZh0sV0b4fx76HqXghKMZIZokQ G78odC5XE2hqHeqigIDnuWg/EB15V46pNwKBid2y51sNg01/K87ilo58OUo3WLNi9Uzj KapjozFr7TQthW77Ggzku4W60e8N0jwbj3GHA= On Jan 27, 2010, at 7:44 AM, Damien Sorresso wrote: > By doing this, you're basically guaranteeing that Firefox can never participate in the code signing model. Maybe you don't care right now, but then again, maybe one day you'll want to use the Keychain APIs. I don't think it's a concern for now. That said, what is the best option of compressing Firefox on Snow Leopard? Ditto launched from a separate app using Launchd APIs? Thanks, Joel --- http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0