September 2008 - Darwin-dev

trouble copying and unmounting a new drive
by Andy Taylor 28 Sep '08

28 Sep '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type; bh=Po8FAIT/d0XBcUZMJn0cBStKd+Dx1Sy43glwWgPE3g8=; b=UL7CNs9DzdJSEbfBNc1ztw5jUBqb5fK210D3l9jMbIdFG9ie066b2GdcUAef47EAyH zlIy93zpZpOhN8vu+xuzaPMOhRWG8g/ePtcECvWEqiZCgedPJjz1KmfKTQ7I7WsiO//N JvIYjdNueExvkIj88dXWi+N5cbULKLicJTYJ0= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=FAxXgNE+0LlHURw9XLFELa3byEFRcV0qdrn0/xLUs1j+vIS4fTrKv2WWbbRBKjIArq +3nY0JbqxrMUgN/TC0gjdnc/3hS3Xl11O/u5PwWfuvQAfz6l9sfN5k2PijGzQk9up/fl 5FWfyY/Y+8oBcsT1v/W7h3LrBhhZbZ/0WZFuc= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: launchd daemon, spawning children?
by Terry Lambert 26 Sep '08

26 Sep '08

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com The primary paper on UNIX setuid() is: Setuid Demystified Hao Chen, David Wagner, Drew Dean Proceedings of the 11th USENIX Security Symposium <www.cs.berkeley.edu/~daw/papers/setuid-usenix02.pdf> - -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Sep 26, 2008, at 2:37 AM, Mario Emmenlauer wrote: Damien Sorresso wrote: The rules of setuid(2) have evolved into a vastly complex morass with subtle variations on different platforms. Entire papers have been written about this behavior. You name it. I will look for a good tutorial, though Terry's suggestion contained a nice start. The kernel interfaces are not really vastly complext; most of the complexity is in user space session setup and teardown, which you mostly do not have to care about if you write "UNIXy code". The main difference between Mac OS X and POSIX standard behaviour is that we support external group resolution for group membership above and beyond the default list of the effective group plus 15 supplementary groups (the 0th supplementary group is a synonym for the effective group, for a total of 16 in the array for getgroups()/ setgroups(); this is an historical BSD thing, permitted, per POSIX). You can initialize this using initgroups(), or you can use syscall(SYS_initgroups,...), which takes the same arguments as setgroups(), plus a uid argument as the key for looking up group membership for the groups served through DirectoryServices. A secondary difference is in per-thread credentials, which inverts the group list and uid set ordering. In per thread credentials, you initialize the group list last; this is permissible since you cannot set them except from a privileged process, and they are being used for proxy operations, usually by a file services dameon, and are not security sensitive. Since the initial privileged process that started the thread within itself (1) has a still-accessible heap and stack after the change and (2) has the rights to set grouplist - and resolver uid membership - anyway, on a process-wide basis, permitting setting this in a thread running under an assumed credential does not decrease security. In general, we do not want you to establish your own sessions, and would prefer that you use /bin/login to do session management and support the utmp/utmpx interfaces for process accounting and knowing who is logged into your system, etc.; otherwise, things get more complicated fast, in order to support those interfaces, since you have to (1) maintain privilege in order to access the files to write out the closure records on sessions started by you (login does this - it hangs around until its children go away), and (2) deal with PAM and all the other things login deals with for authenticating you to the system. This email sent to site_archiver(a)lists.apple.com

1 0

Re: launchd daemon, spawning children?
by Mario Emmenlauer 26 Sep '08

26 Sep '08

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com User-agent: Thunderbird 2.0.0.16 (Macintosh/20080707) Hi, You're right. Back at that time I wasn't aware of the difference between launching application, agent or daemon. You name it. I will look for a good tutorial, though Terry's suggestion contained a nice start. Ok. Then the dynamic launchd option is no good. Now thats a word. The user data is plain binary files. They needn't be in the users home, but they might lack group read properties (unless I set up an on-demand launchd daemon that sets them correctly). Seems I will (for now) stick to plain Unixy stuff. But I feel this was valuable information, my understanding of Agents and the issue of bootstrapping user environments got a lot better, thanks. Cheers, Mario _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Damien Sorresso wrote: On Sep 26, 2008, at 12:52 AM, Mario Emmenlauer wrote: Now this was a misunderstanding (and its probably my fault because I didn't note it more explicit): I don't want to show GUI. Quoting You mentioned LSOpenApplication() as an alternative you investigated. So I concluded that you were interested in launching applications. Also, The rules of setuid(2) have evolved into a vastly complex morass with subtle variations on different platforms. Entire papers have been written about this behavior. - upon event, master dynamically sets up launchd plist for worker daemon for user (can a non-root process do that?) No. See above. Or I'm going back to Terry's suggestion of posix_spawning with the correct uid, which seems safe and sound for my case too. It all depends on what user data you need access to. If the needs are simple (i.e. you just need the path to the home directory so you can read and write state), then sure, posix_spawn(2) should do just fine for you. Just make sure that your master daemon properly reaps its children and all that jazz. In short, if you're doing just plain Unixy stuff, you should be okay. If you want to use any higher-level Mac OS X APIs, you might run into problems. This email sent to site_archiver(a)lists.apple.com

1 0

Re: hdiutil progress
by Brian Bergstrand 26 Sep '08

26 Sep '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com man hdiutil has this to say: This is 10.4+ only. On Sep 26, 2008, at 10:26 AM, Lazuardi wrote: Hello, Regards, Ardian Brian Bergstrand <http://www.bergstrand.org/brian/> PGP Key ID: 0xB6C7B6A2 _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… "-puppetstrings provide progress output that is easy for another program to parse. Any program trying to interpret hdiutil's progress should use -puppetstrings." I have an application that using hdiutil to create disk images. But I don't know how to get the disk image creation progress. The user needs to know the progress, especially when they create a big size disk images, e.g. more than 1GB, that would spends minutes until finish. I have found that by using "-debug" argument, it would printout the percentage progress to the stdout. but "-debug" argument also made diskimage-helper printout the passphrase. So that, I can't give the user using "-debug" argument. Is there any way to get the percentage progress of the disk image creation? I don't know if this is the correct mailing list to ask my problem, but I really need the answer. I would be appreciated for any help. This email sent to site_archiver(a)lists.apple.com

1 0

Re: launchd daemon, spawning children?
by Damien Sorresso 26 Sep '08

26 Sep '08

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com No. See above. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com On Sep 26, 2008, at 12:52 AM, Mario Emmenlauer wrote: Now this was a misunderstanding (and its probably my fault because I didn't note it more explicit): I don't want to show GUI. Quoting myself, my application is a workload distribution process, that is basically similar to an ftp-daemon: user connects over network, daemon gets user request, daemon executes K workers that process user data. Executing the K workers is the problem I'm having now. You mentioned LSOpenApplication() as an alternative you investigated. So I concluded that you were interested in launching applications. Also, for future reference, on Mac OS X, the term "application" is generally used to refer to a process which presents a GUI. Background processes are "daemons". I don't need GUI (even though I wouldn't mind it, if it comes at no additional cost). But I need access to the users data. That doesn't enforce that the daemon is running as the specific user, a dedicated incoming directory would do. But for security and simplicity it seemed a good idea to setuid to the user, both for read-privileges and also for security (not running as root if not needed). You should not call setuid(2) and friends from a launchd daemon. The idea of launchd is to take care of as much of the standard bookkeeping work of writing a daemon as possible through data -- in this case, the property list. You just have to be willing to step outside the traditional Unix model. :) So do I need to bootstrap user sessions for that? It depends on what you mean by "user data". Do you mean plain text files (or binary files in your own format) that you're just reading from the home directory? If so, probably not. But many APIs on Mac OS X have a launch daemon backing them. So to you, you might just be reading an Address Book database. But behind the scenes, a daemon might be accessing that data for you, and it might make the decision of whose data to look up based not on the credentials of the requestor, but what bootstrap he is in. Or possibly both. (That's a hypothetical example. I don't know off-hand if that's really how Address Book works on Mac OS X.) Your idea of per-user daemons/agents is still good, because that would eliminate the need for the master daemon to run as root (only root processes can setuid?). The rules of setuid(2) have evolved into a vastly complex morass with subtle variations on different platforms. Entire papers have been written about this behavior. But yes, in a launchd context, only processes running under the system launchd can set the UserName key and expect it to be enforced. Thus, you can only drop privileges. It might work well in combination with launchd: - master daemon runs as dedicated user - upon event, master dynamically sets up launchd plist for worker daemon for user (can a non-root process do that?) - launchd starts worker daemon - worker daemon posix_spawn's K instances, that connect to master (Could I request launchd to start K instances?) launchd works very hard to ensure that there is, at most, one process running for each job in a given bootstrap. Or I'm going back to Terry's suggestion of posix_spawning with the correct uid, which seems safe and sound for my case too. It all depends on what user data you need access to. If the needs are simple (i.e. you just need the path to the home directory so you can read and write state), then sure, posix_spawn(2) should do just fine for you. Just make sure that your master daemon properly reaps its children and all that jazz. In short, if you're doing just plain Unixy stuff, you should be okay. If you want to use any higher-level Mac OS X APIs, you might run into problems. -- Damien Sorresso BSD Engineering Apple Inc. smime.p7s

1 0

Re: launchd daemon, spawning children?
by Mario Emmenlauer 26 Sep '08

26 Sep '08

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com User-agent: Mozilla-Thunderbird 2.0.0.16 (X11/20080724) Hi Damien, Now this was a misunderstanding (and its probably my fault because I didn't note it more explicit): I don't want to show GUI. Quoting myself, my application is a workload distribution process, that is basically similar to an ftp-daemon: user connects over network, daemon gets user request, daemon executes K workers that process user data. Executing the K workers is the problem I'm having now. I don't need GUI (even though I wouldn't mind it, if it comes at no additional cost). But I need access to the users data. That doesn't enforce that the daemon is running as the specific user, a dedicated incoming directory would do. But for security and simplicity it seemed a good idea to setuid to the user, both for read-privileges and also for security (not running as root if not needed). So do I need to bootstrap user sessions for that? Your idea of per-user daemons/agents is still good, because that would eliminate the need for the master daemon to run as root (only root processes can setuid?). It might work well in combination with launchd: - master daemon runs as dedicated user - upon event, master dynamically sets up launchd plist for worker daemon for user (can a non-root process do that?) - launchd starts worker daemon - worker daemon posix_spawn's K instances, that connect to master (Could I request launchd to start K instances?) Or I'm going back to Terry's suggestion of posix_spawning with the correct uid, which seems safe and sound for my case too. Cheers, Mario _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Damien Sorresso wrote: On Sep 25, 2008, at 3:20 PM, Mario Emmenlauer wrote: launchd.plist(5) actually makes no statements on the launch.h API :-( The API seems quite undocumented. From your answer I deduce that one can not set up or change launchd.plist's dynamically through launch.h API calls. Yes, the API is undocumented. But you can use it to set up a dynamic job. (The launchd.plist(5) is merely a persistent representation of the job data structure, not the job data structure itself.) This will not get you what you want with respect to launching GUI apps, however. [...] The short of it is that there's no good way to launch a GUI app from outside a user session. You could pull off what you want to do for simple, command-line utilities -- though you could easily run into problems with some of our APIs. If it's any consolation, I'm not very happy with that state of affairs either. :) This email sent to site_archiver(a)lists.apple.com

1 0

hdiutil progress
by Lazuardi 26 Sep '08

26 Sep '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to :content-type:content-transfer-encoding:mime-version:subject:date :x-mailer; bh=vgMAEQQQWcmBSfaQRdT8eT+Rn3wvvnw00bRsXfMUHv4=; b=LuVscM8Mcv4S4nMuYeH6vlvq9ZjZROd2RMbwipjsvErdlsZAY//YPDc1dZS7gVTra7 DTt2bH2IIgiIvwz0RsDDBxdqPqz+ghtISpsggU4ngjsmeS456/j9WVXno4FPIQdJ7i+Y iZVHlXv3zUsNHrfO0/IU2jG/inZOI//AMletI= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:content-type:content-transfer-encoding :mime-version:subject:date:x-mailer; b=mlSfvMzDbY+N+LIjnMjyzjGdNtk1h3gd6V4RAeYpSwgtyBgFt6gbsUO9wPv535jZbv G8xbDRlWcW1iQrrt1EaZz0SbM64bJPgXfYF66Ltm2sCAVKnMZ1ZxREmgew0ukhnxxtOG 2hgyBxfK7VdxQHDULoJJ6pxDVVNkoJbJasL1o= Hello, Regards, Ardian _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I have an application that using hdiutil to create disk images. But I don't know how to get the disk image creation progress. The user needs to know the progress, especially when they create a big size disk images, e.g. more than 1GB, that would spends minutes until finish. I have found that by using "-debug" argument, it would printout the percentage progress to the stdout. but "-debug" argument also made diskimage-helper printout the passphrase. So that, I can't give the user using "-debug" argument. Is there any way to get the percentage progress of the disk image creation? I don't know if this is the correct mailing list to ask my problem, but I really need the answer. I would be appreciated for any help. This email sent to site_archiver(a)lists.apple.com

1 0

Re: launchd wrong behavior for nonexistent paths and crashed services
by Damien Sorresso 25 Sep '08

25 Sep '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com The bug report has been reported as a duplicate. Radar number please? -- Damien Sorresso BSD Engineering Apple Inc. dsorresso(a)apple.com 408.974.7488 _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com On Sep 25, 2008, at 3:04 AM, Stéphane Sudre wrote: Please file a bug against interface disabling not automatically disabling Internet sharing via the interface being disabled. Being pessimistic, I guess it means it will never be fixed in 10.5. smime.p7s

1 0

Re: launchd daemon, spawning children?
by Damien Sorresso 25 Sep '08

25 Sep '08

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com On Sep 25, 2008, at 3:20 PM, Mario Emmenlauer wrote: launchd.plist(5) actually makes no statements on the launch.h API :- ( The API seems quite undocumented. From your answer I deduce that one can not set up or change launchd.plist's dynamically through launch.h API calls. Yes, the API is undocumented. But you can use it to set up a dynamic job. (The launchd.plist(5) is merely a persistent representation of the job data structure, not the job data structure itself.) This will not get you what you want with respect to launching GUI apps, however. I would strongly suggest creating an agent that runs for every user, is limited to the Aqua session, listens for events and launches the appropriate application using the appropriate high- level APIs found in the LaunchServices or AppKit frameworks. It sounds like good advice, but you're leaving me half way. My situation is that of a server environment with many users and almost no GUI login. You are right, a per-user pre-login launchd agent would be a solution, but it would also come with several drawbacks: 1) a 'manager-daemon' is needed anyways, for centralized network I/O Sure, but you can have each agent, upon startup, register with the manager. 2) pre-login Agents are only available since Mac OS 10.5, while many of our users have not yet upgraded Why would you need a pre-login agent? Again, that wouldn't do what you want. It'll display UI, but that UI would not be hooked up to the user's window server session. 3) One agent per user can be a lot, in large server environments. Leaving them running is too much, and starting them up via launchd is a lot slower than the fork/exec or posix_spawn. Starting these jobs via posix_spawn(2) from a daemon running under the system launchd will not do what you want, especially in the case of GUI apps. In fact, it'll simply fail to launchd GUI apps. You *could* submit a job to the system launchd with the UserName key set, but that will spawn the job under the system bootstrap, not the user's and not hook you up to any of the user services you might be expecting. The short of it is that there's no good way to launch a GUI app from outside a user session. You could pull off what you want to do for simple, command-line utilities -- though you could easily run into problems with some of our APIs. If it's any consolation, I'm not very happy with that state of affairs either. :) -- Damien Sorresso BSD Engineering Apple Inc. dsorresso(a)apple.com 408.974.7488 smime.p7s

1 0

Re: launchd daemon, spawning children?
by Mario Emmenlauer 25 Sep '08

25 Sep '08

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com User-agent: Mozilla-Thunderbird 2.0.0.16 (X11/20080724) Hi Damien, launchd.plist(5) actually makes no statements on the launch.h API :-( The API seems quite undocumented. From your answer I deduce that one can not set up or change launchd.plist's dynamically through launch.h API calls. Ok, that are some hard facts! It sounds like good advice, but you're leaving me half way. My situation is that of a server environment with many users and almost no GUI login. You are right, a per-user pre-login launchd agent would be a solution, but it would also come with several drawbacks: 1) a 'manager-daemon' is needed anyways, for centralized network I/O 2) pre-login Agents are only available since Mac OS 10.5, while many of our users have not yet upgraded 3) One agent per user can be a lot, in large server environments. Leaving them running is too much, and starting them up via launchd is a lot slower than the fork/exec or posix_spawn. I will start by reading (thanks, Garth :) again. Maybe setting up a dedicated user and a single dedicated daemon is best for me. Thanks for pointing me to the issue of bootstrapping user sessions in a daemon! Cheers, Mario _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Damien Sorresso wrote: On Sep 25, 2008, at 9:40 AM, Mario Emmenlauer wrote: 4) Using launchd() works great: launch_data_t msg = launch_data_alloc(LAUNCH_DATA_DICTIONARY); launch_data_dict_insert(msg,launch_data_new_string("com.c.App"), LAUNCH_KEY_STARTJOB); launch_data_t resp = launch_msg(msg); but I am missing an interface to set the user. Of course a fixed user can be chosen in the plist, but then its not dynamic. Also, I can start an app as root with launchd and setuid from within this app, but if someone exchanges the app, he will gain root access. See launchd.plist(5). You can only set a UserName key in a daemon running under the system launchd. Though if you're looking to launch applications in user sessions from a root-owned daemon, you're almost certainly going about it wrong. To launch an application in a user session, you first have to get that user's session up and running if it isn't already. We don't want third-party developers trying to bootstrap user sessions. I would strongly suggest creating an agent that runs for every user, is limited to the Aqua session, listens for events and launches the appropriate application using the appropriate high-level APIs found in the LaunchServices or AppKit frameworks. http://developer.apple.com/technotes/tn2005/tn2083.html#SECDAEMONOMICON This email sent to site_archiver(a)lists.apple.com

1 0