February 2010 - Darwin-dev - lists.apple.com

Re: What's eating gilbert pid? (kevent NOTE_FORK and pids question)
by Terry Lambert 19 Feb '10

19 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com That's also not really "better". Alas, what I really want to do is track processes in some way such that marking a process causes that mark to be inherited by children, and allow the kernel to read the mark, and a process can cause itself to get a new mark which will be inherited to *its* future children, and it seems that's simply not possible: -The MAC subsystem isn't supported (http://developer.apple.com/mac/library/qa/qa2007/qa1574.html) -login contexts and audit sessions are one-per-process and owned by system software -I'm insufficiently special to use a mach special port (there are 7) -The kauth external cred resolver interface allows but a single resolver and I'm not memberd. What actor needs access to the information? (1) This is what the keychain mechanism gives you -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Feb 19, 2010, at 11:59 AM, Derrick Brashear wrote: (2) Consider adding a directory services plugin, which give you access as part of the authority of memberd (3) Consider simply putting the initail process in an additional supplementary group This email sent to site_archiver(a)lists.apple.com

1 0

Re: What's eating gilbert pid? (kevent NOTE_FORK and pids question)
by Derrick Brashear 19 Feb '10

19 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=Cyd8Uf1QHcvlwBE/0Nr5/laDjmvrF0MyGAeY+MtWSXU=; b=UfFo0jM54I6SsaMVhezmAYy3I35ctKAE2UpZnBoCSr3PvT3nSU/QJ7Z7qgFtFfn3+6 1FuGk+skCHJFC6mbVSj9v8E6uIhvrRm+DlwXQ6EFrlJFhS4lZ3Q7yJUGi2LK9Xs2rvqx HYfwS45BRV0TGiCWIvqvyBCNKjzN4Yb6Z9kzs= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=mPrQFBZo3Y7fy/tv8A2b5/Qy9jeOqMna1fq092cPdaoRysyLwoe7AAc2vvI13ccZU7 P7Nd7XjQGbb9yEPDI6I5E4SkJX/hgf7PSM2ksTuWobKf/T7wk6nPe77jEp13Go3A/OXo vyN4RdPe4UJrkQ0btMSZERU1qdLji8Oujyc00= On Fri, Feb 19, 2010 at 2:52 PM, Terry Lambert <tlambert(a)apple.com> wrote: > On Feb 19, 2010, at 9:45 AM, Derrick Brashear wrote: >> >> I want to track forked children of a given pid. For the purpose of >> this discussion, I am userspace code (I could be kernelspace code if >> it helped) > > [ ... ] > >> If I can really not have this, Launchd seems to use a sysctl to >> enumerate children on a fork: >> int mib[] = { CTL_KERN, KERN_PROC, KERN_PROC_ALL, j->p }; >> >> (KERN_PROC_PPID seems to not be implemented in xnu-1456.1.26) >> >> Am I really reduced to this? It seems very likely to be racy. > > It is. It is also SPI (System Private Interface), subject to change on > every software update (SPI can be thought of as a contract between system > components which all agree to be updated in lock-step with each other when > updates happen). Please do not use this sysctl. Ok. > A better way is to use ptrace (for some definition of "better"; it's still > not a good idea to need to know this, and ptrace has its own issues). This > is what gdb and other debuggers do. If you are denied the rights to do this > via ptrace, it's probably because you aren't supposed to be doing it. If > so, then you should expect that the sysctl-based enumeration will probably > also deny you that right at some point in the future, so even if you can > hack it so that it works now, it will probably break at some later date. That's also not really "better". Alas, what I really want to do is track processes in some way such that marking a process causes that mark to be inherited by children, and allow the kernel to read the mark, and a process can cause itself to get a new mark which will be inherited to *its* future children, and it seems that's simply not possible: -The MAC subsystem isn't supported (http://developer.apple.com/mac/library/qa/qa2007/qa1574.html) -login contexts and audit sessions are one-per-process and owned by system software -I'm insufficiently special to use a mach special port (there are 7) -The kauth external cred resolver interface allows but a single resolver and I'm not memberd. -- Derrick _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: What's eating gilbert pid? (kevent NOTE_FORK and pids question)
by Terry Lambert 19 Feb '10

19 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com I want to track forked children of a given pid. For the purpose of this discussion, I am userspace code (I could be kernelspace code if it helped) [ ... ] If I can really not have this, Launchd seems to use a sysctl to enumerate children on a fork: int mib[] = { CTL_KERN, KERN_PROC, KERN_PROC_ALL, j->p }; (KERN_PROC_PPID seems to not be implemented in xnu-1456.1.26) -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Feb 19, 2010, at 9:45 AM, Derrick Brashear wrote: Am I really reduced to this? It seems very likely to be racy. It is. It is also SPI (System Private Interface), subject to change on every software update (SPI can be thought of as a contract between system components which all agree to be updated in lock-step with each other when updates happen). Please do not use this sysctl. A better way is to use ptrace (for some definition of "better"; it's still not a good idea to need to know this, and ptrace has its own issues). This is what gdb and other debuggers do. If you are denied the rights to do this via ptrace, it's probably because you aren't supposed to be doing it. If so, then you should expect that the sysctl-based enumeration will probably also deny you that right at some point in the future, so even if you can hack it so that it works now, it will probably break at some later date. This email sent to site_archiver(a)lists.apple.com

1 0

What's eating gilbert pid? (kevent NOTE_FORK and pids question)
by Derrick Brashear 19 Feb '10

19 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=evXLtzWqNByt6d7rXxzM+KfUH7EyBRI68OcQUNYLdnE=; b=cU5uwc4Hu1U2qNglJJIFEaEGBLJOoRbYqjLzbOqbBZKo5zU3HdSpJUtV2AKxQ49dzi KfH4Y2NaGdNOMqHlOpwPiypeEvXWropZlKy2JpeqoZkySaL+e/ysilitc8ADfi5NWe30 ZNEXKDDHxO1UoD5L8v9uSasGYGgElFNJWbgAM= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=fPKbSrs8JqoGeTQR3tSDxnGa9uVCkLohx1HkO+XfLnOW4TzSrPSSbsd8zScIFopXhD uRcbC863jIciZO4VDiSioMSbpcbpmR68t4WxpzarMqiReREuc35hrXxMlVrII2e8ld0A bvjUD8uqmS9fNAavdXMQHDWvtkfkYekSjipK0= I want to track forked children of a given pid. For the purpose of this discussion, I am userspace code (I could be kernelspace code if it helped) Best bet would seem to be NOTE_FORK kevents. I can't have NOTE_CHILD or use NOTE_TRACK, as I get ENOTSUP. I have no visibility into it, but this list from 2007 suggests that's rdar://problem/3439519 Ok, fine. NOTE_FORK gets OR'd with the child pid, right? kern_exec.c: proc_knote(p->p_pptr, NOTE_FORK | p->p_pid); kern_fork.c: proc_knote(parent_proc, NOTE_FORK | child_proc->p_pid); kern_fork.c: proc_knote(parent_proc, NOTE_FORK | child_proc->p_pid); So I'd expect: event[0].filter = EVFILT_PROC; event[0].flags = EV_ADD | EV_ENABLE; event[0].fflags = NOTE_EXIT | NOTE_FORK | NOTE_EXEC; kevent(kq, NULL, 0, event, 1, NULL); to give me a child in a NOTE_FORK Instead, event[0].data is unset, and event[0].fflags is *solely* NOTE_FORK. I can see nowhere in the kernel where NOTE_PDATAMASK is masked off. If I can really not have this, Launchd seems to use a sysctl to enumerate children on a fork: int mib[] = { CTL_KERN, KERN_PROC, KERN_PROC_ALL, j->p }; (KERN_PROC_PPID seems to not be implemented in xnu-1456.1.26) Am I really reduced to this? It seems very likely to be racy. -- Derrick _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: close on USB serial port hangs (read pending) on Snow Leopard
by Fritz Anderson 19 Feb '10

19 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 18 Feb 2010, at 10:30 PM, Terry Lambert wrote: > Another non-synchronizing approach would be to use a select with a timeout, or set a vtime as well as a vmin on the descriptor. You would still be polling, but you could do it at much longer intervals, and escape the performance issues. Waking up once every 10 seconds (or whatever is a tolerable wait for a close operation) is going better than buzz-looping, no matter how you look at it. This is just what I needed, and I thank you. I know my solution is third choice, but I'm working with a Cocoa NSThread, which denies access to pthreads. I'd be very grateful if someone can vet my solution. I think I know enough now to proceed, but I'm away from the device till this evening. I know you all have better things to do than code review for a novice, but if I'm doing anything obviously wrong, your help could save me a lot of time. — F /* In the object for which these are methods: inputHandle: the fd for the port; -1 if no further I/O is to be attempted. savedTTYState: the result of a tcgetattr() from initialization time. doClose: a flag for whether polling should stop. */ // Called on read thread - (int) internalClose { int result = tcsetattr(inputHandle, TCSANOW, &savedTTYState); int lagHandle = inputHandle; inputHandle = -1; result = close(lagHandle); return result; } // Called on read thread - (ssize_t) blockingRead: (void *) buffer size: (ssize_t) bufferSize { ssize_t retval = 0; fd_set permFDs; FD_ZERO(&permFDs); FD_SET(inputHandle, &permFDs); while (! doClose) { fd_set readSet; FD_COPY(&permFDs, &readSet); fd_set errSet; FD_COPY(&permFDs, &errSet); struct timeval timeout = { 0, 100000 // 100_000 microseconds = 1/10 second }; int pending = select(inputHandle+1, &readSet, NULL, &errSet, &timeout); if (pending != 0) { if (FD_ISSET(inputHandle, &errSet)) { // An error. Panic out. [self internalClose]; return -1; } else if (FD_ISSET(inputHandle, &readSet)) { // Data! Pull it in. retval = read(inputHandle, buffer, bufferSize); // 9600 baud in 0.1 seconds = ~100 bytes. // I can afford to provide a big buffer and // take my chances that the read won't exceed // the buffer size. if (retval < 0) { [self internalClose]; } return retval; } // The caller uses performSelectorOnMainThread: to // notify a delegate of data or error. } } return retval; } // Called on main thread - (void) close { doClose = YES; } _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Java sockets don't pass through NKE
by Michael Ritter 19 Feb '10

19 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Importance: normal Sensitivity: Normal _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Issue with DiskArbitration unmount approval callback - it is called multiple times
by Vera Tkachenko 19 Feb '10

19 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hello darwin-devs! I'm developing an application which should do some actions with removable disks before it's unmount. I've found out that this could be done with DiskArbitration unmount approval callback. I use the following test code: 1. Registering approval callback <...> //create approval session DAApprovalSessionRef session = DAApprovalSessionCreate(kCFAllocatorDefault); //register unmount callback DARegisterDiskUnmountApprovalCallback(session, kDADiskDescriptionMatchVolumeMountable, unmountApprovalCallback, NULL); // context DAApprovalSessionScheduleWithRunLoop(session, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode); CFRunLoopRun(); 2. Callback function itself - just log it was called and return success. DADissenterRef unmountApprovalCallback(DADiskRef disk, void *context) { //Just log and return success - this should allow unmounting <> NSLog(@"unmountApprovalCallback called"); DADissenterRef dissenter = DADissenterCreate(kCFAllocatorDefault, kDAReturnSuccess, CFSTR("unmount allowed")); return dissenter; } Unmount callback is called, the application returns success and I thought that it will never be called again for this drive. But my tests show different: callback is called multiple times and it seems that system ignores returned value (e.g. Finder shows "Force Eject"). Additionally, in case of Network Volume Finder logs the following: e.g. "Feb 17 16:14:04 macaca /System/Library/CoreServices/Finder.app/Contents/MacOS/Finder[215]: NetworkNode::ejectCallBack returned -47" Summarizing the problem: Why is callback called several times? Why system ignores my successful return code? Should I do something additionally to this? Thanks in advance for your replies! -- With regards, Vera Tkachenko _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: close on USB serial port hangs (read pending) on Snow Leopard
by Terry Lambert 19 Feb '10

19 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com == The original problem: == The portable user-space synchronization solution: -- -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Feb 18, 2010, at 7:27 PM, Fritz Anderson wrote: On 18 Feb 2010, at 8:39 PM, Keun-young Park wrote: Maybe closing from different thread is causing problem. You can use select and use other mechanism like socket to send message from main thread to the read thread to send close request. When close request comes, the read thread can close the serial port and exit. I'm willing to believe this. But a question: If the read thread is blocked on the read(), how does it receive the close request? (Or is this another possible use for select()?) It's late where I am, so I'm a bit slow. — F Closing a descriptor out from under a read or write does not cancel the outstanding I/O. The close thread is blocking waiting for the I/O operations in progress to drain out prior to destroying the resources which are potentially in use by those I/Os. For example, if you are closing an fd on a serial port on one thread, not caring that your read posted by another thread is waking up at the same time as a result of incoming data simultaneous with the close. This occasionally resulted in a thread with an outstanding I/O having the I/O complete and write back to a deallocated memory region. This was bad. Note that we are aware that some operating systems will attempt to cancel the read, and succeed with the drivers they themselves supply. However, not all possible synchronization primitives which might be used in third party drivers record their wait channels in the device driver, allowing the driver to perform cancellation of the outstanding blocking operations. This is an architectural difference, and, frankly, not a standards mandated behaviour in either direction. All platforms suffer from the possibility of a third party driver using a non-OS supplied synchronization primitive. Code that assumes cancellation-by-close is not portable. Upper-level kernel code that assumes this is inherently dangerous, if you load third party drivers. (A) This is a problem which needs to be addressed in the user space code. Ideally, from a kernel's perspective, all I/O operations to a given descriptor would be marshalled to a single thread. This is sometimes inconvenient if you are porting code rather than designing it initially. Without that, you would instead record the state of the thread blocking on the descriptor in a connection state object, and rather than closing the descriptor, you could close the connection object. The connection close method would first change the state of the connection object to prevent new I/Os by other thread being started, then use pthread_kill() on the list of threads to send explicit signals to abort the operations in progress in the kernel, and once they had drained out, perform the actual close on the underlying fd. (B) Keun-young Park's suggestion could be interpreted to say that you should only issue reads when you know the read will immediately return rather than blocking in the kernel. This is also a viable design pattern, but somewhat more complicated to achieve. Here's how it can done: The canonical UNIX way to do this is to read with a large buffer, but have previous used a tcsetattr() to ensure that the device is in raw mode to avoid waiting for a newline, and that vmin is set to 1 byte, which requires at least one byte, but can read up to your buffers size worth of bytes. The issue is that select() only reports data available, not how /much/ data is available, but you want to issue large reads to reduce overhead, and still return with the data which *is* available, if there is less than that. By using a socket, pipe, or other IPC mechanism using a file descriptor as the "control channel" for the read thread, you could wake it up by writing data to the other end (or if you didn't, it would wake up when there was data to be read from the device). (C) Another non-synchronizing approach would be to use a select with a timeout, or set a vtime as well as a vmin on the descriptor. You would still be polling, but you could do it at much longer intervals, and escape the performance issues. Waking up once every 10 seconds (or whatever is a tolerable wait for a close operation) is going better than buzz-looping, no matter how you look at it. My personal suggestion would be to provide a connection state object (struct, or whatever, depending on what language you are programming in). Note that this is the same approach used by stdio implementations to deal with buffer serialization for the stdio buffers: each FILE * object has a lock which it uses in the threads version of the library routine in order to synchronize access to both the buffer and to the underlying fd, when refilling or flushing the buffer. This email sent to site_archiver(a)lists.apple.com

1 0

Re: close on USB serial port hangs (read pending) on Snow Leopard
by Fritz Anderson 19 Feb '10

19 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 18 Feb 2010, at 8:39 PM, Keun-young Park wrote: > Maybe closing from different thread is causing problem. You can use select and use other mechanism like socket to send message from main thread to the read thread to send close request. When close request comes, the read thread can close the serial port and exit. I'm willing to believe this. But a question: If the read thread is blocked on the read(), how does it receive the close request? (Or is this another possible use for select()?) It's late where I am, so I'm a bit slow. — F _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: close on USB serial port hangs (read pending) on Snow Leopard
by Keun-young Park 19 Feb '10

19 Feb '10

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=by0AC5rfu8EGflnrwz+rGKh0c2N3bLVsu+GNegAODJA=; b=BCQUNm6Amx2B6O5Y5nt1fn9TQPGFRMos0Vvc42loP3f2MUkKf3tfNDbQFckCNVx+Y6 1bbga82gK9vfxzuFVDElOfuClbK3uy97JjOJUUaB90NNbl44mTCa28boL+WxI2OCFotf wtxib8CUlsAhe7sgk7M3j41NAM4VMBprbSoaU= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=TXVseFiezaE0OWeBNSKlY4HQ3uNXmqLKtbG1iHczoxtiPSDLvf+yqVMjKd/t8dc5Gd PuxXuHlNLow5RUsBawRhfH9P4TipR27DR8WSUvDSz9zoY6HiP2iVgKOfhEGndqrFS1sw TbsL4EJHwtI6u8RbbFvMTkQw01JMt+qQO23Eo= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0