April 2009 - Darwin-kernel - lists.apple.com

Re: What if process crashes while holding a locked semaphore?
by Terry Lambert 02 Apr '09

02 Apr '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Apr 1, 2009, at 9:31 PM, Steve Checkoway <s(a)pahtak.org> wrote: On Apr 1, 2009, at 4:59 PM, Terry Lambert wrote: We don't support unnamed POSIX semaphores, we don't support sem_getvalue(), and if you try you get a SIGSYS, which if you don't catch it, you're program exits. We also don't guarantee correct error semantics for the POSIX named semaphores we do have, since if we can't run the conformance tests because we don't support unnamed semaphores, we can't tell if the errors we return are correct or not. Is there a technical reason that Darwin doesn't support POSIX unnamed semaphores or it's just not a priority since they're not necessary? I can give you technical information about why I haven't personally done it, but realize I am not speaking officially in any capacity by doing so. With that in mind, I'll try to answer your questions from an engineering perspective, rather than putting it as the more pragmatic "if Apple implemented POSIX unnamed semaphores, how many X Serves would you buy?"... They aren't required for UNIX conformance because they are not mandatory interfaces per Appendix 9 of the Single UNIX Specification, and there are lots of alternatives, most of them with better overall semantics, which this tread has amply demonstrated. Most portable and historical software uses System V semaphores, since POSIX semaphores are relatively new, so there's little software portability incentive. The software which does like to use them is typically based on GNU autoconf/automake which typically uses linkage tests to find interfaces and blows it by doing that and ignoring <unistd.h> contents, since historical conformance with older versions of the specification permitted stub functions which errored out, and the linkage tests only check linkability rather than functionality. So basically the software that wants them usually fails to conform to the standards which would have allowed them to be used safely and reliably in the first place. Then there is the little problem of binary backward compatibility for POSIX named semaphores, if the error or success returns don't happen to match the standard once testing of a full implementation became possible. As a Mac OS X implementation detail, sem_t is the required data type for both POSIX named semaphores and POSIX unnamed semaphores according to the standard. The sem_t definition has historically been a 32 bit value containing an fd cast to a pointer, which is problematic for maintaining binary and source backward compatibility (hint: think symbol decoration) for named semaphores while at the same time permitting typical usage of unnamed semaphores. Specifically, typical usage of unnamed semaphores is to use them as an IPC synchronization mechanism between otherwise unrelated programs by allocating a shared memory region shared between them of sizeof(sem_t) *number_of_semaphores_desired, and the casting the base address of that memory region to a (sem_t *) and indexing off that to get a semaphore index in the range 0..(number_of_semaphores_desired - 1). The implementation problems should now be obvious. They are not insurmountable; I sometimes pose how to resolve the conflicting goals involved as an interview question. 8-). But there isn't really a very obvious fix I'd call elegant, either. Maybe someone will surprise me someday, or maybe an outside developer will attach elegant code capable of balancing the conflicting goals to a bug report the next time they request the feature, or maybe someone will contract to buy a bunch of X Serves. This email sent to site_archiver(a)lists.apple.com

1 0

Re: debuggging a recurring system freeze (syslog redir, ktrace?)
by Thomas Tempelmann 02 Apr '09

02 Apr '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=RxNztaaPufY3nSToS/X1pq+/7N/v3KZxmPJmYmOC8M0=; b=COnZaJYbUAqM6rOND1RnrJ4tarC9gtYhUGRTQEQRX36FrYTWo7eghiPDNnbaetqeSi P1PwenjIRsUCwm8QcwDfsAa8RfToCbVniwfNFERBOO+8J8J6hiPCJ58b87NgziQBzeqN I9AXcZ/0PHeUainyz/+LXCUbz1y+LnFF3Lzx0= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=p8avrxGROahlaZpEYsSwOFj+Gu5mhnIh2IRqcej+ESNKaf85Rl2O24fwmqic3L62hX jFkXWVu/oL5zcdu82X1j4ZehRIIHYFMNwMATaIij/Okm6tCtaSGo5UiDHXVXgsP4LUGE ldLDnmbxhaDSmG8s4Y1nEMZpwaEHoAVZyexH4= On Mon, Mar 30, 2009 at 21:24, Terry Lambert <tlambert(a)apple.com> wrote: > Most likely your process table is full because you have a parent process > that's failing to reap its children. > > If you have Terminal set to not close on exit, and you have a terminal open > you can type: > > exec ps gaxlwww I had another freeze last night. Unfortunately, the exec command does freeze as well. I can type the command via ssh, but it never shows any own output. Internal shell commands still work: a blank "exec" as well as "echo <arg>", "exit", "pwd", "cd <arg>", "times" work. But invocation of any external component leads to the stall. "exec unknowncommand" leads to freeze. I had a few shells open (via separate ssh terminals), so I could run a few more tests that led to freezes. The last one was the most interesting: servo2:~ root# echo "test" >f bash: f: Invalid argument This error makes no sense, does it? I am root, I am in a home dir which is generally writable, and yet it doesn't want to perform the redirection. Could this be an indication of what Andreas Fink suggested earlier: > This sounds like "not enough free file descriptors / too many open files" issues. Is there really a system-wide (global) limit on available file descs? How could I test how many are left, or which apps use most of them (hmm, that's probabl "lsof")? -- Thomas Tempelmann, http://www.tempel.org/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: URGENT: Panic Problems
by Rick Sulack 02 Apr '09

02 Apr '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Apr 1, 2009, at 7:11 PM, George Plymale wrote: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Hi, I am having a nightmare at the moment. I woke up this morning and my system had panicked. It is a MacPro with a software RAID 01 configuration. I rebooted and the system tried to start, got some way into the init it appeared even though I did not having it booting verbosely and then panicked again. The next time it would simply not recognize any bootable drive. It seems that I can get into single user mode, but cannot load the root fs. I drove a couple hours away to an Apple store and we spoke for a while, but they refused to do anything as they were not allowed to do so. One fellow there seemed fairly confident that it would rebuild properly as we had, at that time assumed that a drive was bad. They listened and one of the drives appeared to sound clunky as though it had the click of death, so I figured that to be true. I brought the machine home with a couple new drives to replace the bad one with, took each drive out and listened to them initialize outside of the machine as the store was quite noisy and unfortunately that was the standard initialization of each machine. It appeared as though there was nothing really physically wrong with the drives, plus when printing out the raid information from diskutil none showed to be degraded nor did they have any SMART failure. I had already, though, tried to replace the drive and it had restarted to build, but about 10 minutes in the OS panicked again. So then that was when I thought that perhaps the drive was not bad and then replaced the new drive with the old one to see what would happen. Same thing, it tried to rebuild the array and then about 10 minutes in, it would panic. This panic happened on a Tech Tool Pro boot eDrive and an install DVD. It also appears to happen when a rebuild is not running and the OS is simply idling. I did not have enough time to write the entire panic stack down here, but could reproduce it and go back and type it up for you in this email. The first line in the backtrace was had to do with: com.apple.comkext.aty_lamna The last line I believe had something to do with the Nvidia graphics card if I recall correctly. Given the above and the panic log from your other email doesn't sound like this is RAID related. I am in desperate need to recover this array. I will loose 2 weeks of work plus all of my family photos and some videos if I cannot rebuild this. I fear that there is more of a root cause here vs. RAID failure. Any thoughts or ideas as to how I should track this down and recover from this issue? You only need one drive to recover your data. The drive you removed and then replaced again is no longer complete until you finish rebuilding it again, but even so, you should be able to get some data off it if you have to. The other drive should be fine. I would recommend removing your good drive and keeping it somewhere safe (and labeled). Panics are great way to corrupt data. Then try to figure out what is wrong with your machine by reinstalling the OS on a different drive if you can. The panic looks video card related. Maybe it's a bad video card or bad memory or ... I would try swapping things until your machine seems to be working again. Once you get the machine up and running again, then try putting the good half of the raid mirror back in the machine and backing it up. Mirror are not replacement for backups. Once you have a back up then try to rebuild the mirror. The most important thing is to try and make a backup before you start messing around. Ideally, you should have had already had made a backup. Good luck, rick __ rick sulack darwin core os group apple, inc -------------------------------------------------------------- "More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason - including blind stupidity" -Wulf 1972 This email sent to site_archiver(a)lists.apple.com

1 0

Re: What if process crashes while holding a locked semaphore?
by Steve Checkoway 02 Apr '09

02 Apr '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On Apr 1, 2009, at 4:59 PM, Terry Lambert wrote: -- Steve Checkoway "Anyone who says that the solution is to educate the users hasn't ever met an actual user." -- Bruce Schneier _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com We don't support unnamed POSIX semaphores, we don't support sem_getvalue(), and if you try you get a SIGSYS, which if you don't catch it, you're program exits. We also don't guarantee correct error semantics for the POSIX named semaphores we do have, since if we can't run the conformance tests because we don't support unnamed semaphores, we can't tell if the errors we return are correct or not. Is there a technical reason that Darwin doesn't support POSIX unnamed semaphores or it's just not a priority since they're not necessary? smime.p7s

1 0

Re: URGENT: Panic Problems
by Dave Keck 02 Apr '09

02 Apr '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Based on the back trace it looks like a graphics card issue. This is a naive suggestion, but I would try this: 1. Boot the machine on the potentially bad drive, and, assuming you have enough time to do so, enable SSH. 2. Shut the machine down, remove the graphics card, and start the machine back up. 3. SSH into the machine from another machine and backup everything you can. If the machine doesn't panic, then I'd say it's a pretty good bet it's the graphics card. Anyone want to correct me if this is a bad idea? David _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: What if process crashes while holding a locked semaphore?
by Quinn 02 Apr '09

02 Apr '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com I'm not |-: TN2083 has all the gory details. <http://developer.apple.com/technotes/tn2005/tn2083.html> S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Relations, Developer Technical Support, Core OS/Hardware _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… At 07:43 -0700 31/3/09, Jerry Krinock wrote: Very good, Chris. I'm an NSPortNameServer fan now. When using Mach ports like this, you have to be very careful about bootstrap namespaces. Each process inherits a reference to a bootstrap namespace, meaning that which namespace it registers in depends on how it was launched. A port registration will only fail if the conflicting process is in the same namespace. This may or may not be what you want. This email sent to site_archiver(a)lists.apple.com

1 0

Re: URGENT: Panic Problems
by George Plymale 02 Apr '09

02 Apr '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

URGENT: Panic Problems
by George Plymale 02 Apr '09

02 Apr '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Desperately yours, -George _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… Hi, I am having a nightmare at the moment. I woke up this morning and my system had panicked. It is a MacPro with a software RAID 01 configuration. I rebooted and the system tried to start, got some way into the init it appeared even though I did not having it booting verbosely and then panicked again. The next time it would simply not recognize any bootable drive. It seems that I can get into single user mode, but cannot load the root fs. I drove a couple hours away to an Apple store and we spoke for a while, but they refused to do anything as they were not allowed to do so. One fellow there seemed fairly confident that it would rebuild properly as we had, at that time assumed that a drive was bad. They listened and one of the drives appeared to sound clunky as though it had the click of death, so I figured that to be true. I brought the machine home with a couple new drives to replace the bad one with, took each drive out and listened to them initialize outside of the machine as the store was quite noisy and unfortunately that was the standard initialization of each machine. It appeared as though there was nothing really physically wrong with the drives, plus when printing out the raid information from diskutil none showed to be degraded nor did they have any SMART failure. I had already, though, tried to replace the drive and it had restarted to build, but about 10 minutes in the OS panicked again. So then that was when I thought that perhaps the drive was not bad and then replaced the new drive with the old one to see what would happen. Same thing, it tried to rebuild the array and then about 10 minutes in, it would panic. This panic happened on a Tech Tool Pro boot eDrive and an install DVD. It also appears to happen when a rebuild is not running and the OS is simply idling. I did not have enough time to write the entire panic stack down here, but could reproduce it and go back and type it up for you in this email. The first line in the backtrace was had to do with: com.apple.comkext.aty_lamna The last line I believe had something to do with the Nvidia graphics card if I recall correctly. I am in desperate need to recover this array. I will loose 2 weeks of work plus all of my family photos and some videos if I cannot rebuild this. I fear that there is more of a root cause here vs. RAID failure. Any thoughts or ideas as to how I should track this down and recover from this issue? This email sent to site_archiver(a)lists.apple.com

1 0

Re: What if process crashes while holding a locked semaphore?
by Terry Lambert 01 Apr '09

01 Apr '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com After studying Terry's latest remarks I realize where the real "bug" is. It seems that Posix semaphores on Mac OS X have limitations, are not supported and not recommended. Probably their only use is to allow porting of linux programs. So the bug is that man pages for the sem_xxx() methods lack a statement saying that these functions are depracated and not recommended for new designs. They aren't deprecated, we just don't promise that they are fully POSIX compliant at this time. Other than that, yeah, there's a man page. 8-). -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Apr 1, 2009, at 12:31 PM, David Gatwood wrote: On Mar 31, 2009, Terry Lambert wrote: On Mar 31, 2009, at 8:21 PM, Jerry Krinock wrote: Would anyone disagree with this? Hmm. I just checked and I see nothing in the POSIX spec for sem_wait/sem_post that requires such behavior. Maybe there's something elsewhere and I just missed it---not sure---but I think Mac OS X may be compliant and correct as-is. It looks like the only guaranteed cross-platform and safe way to handle this situation would be to add a signal handler that calls sem_post(...) and then _exit(255) if you get a SIGSEGV or SIGBUS signal. We don't support unnamed POSIX semaphores, we don't support sem_getvalue(), and if you try you get a SIGSYS, which if you don't catch it, you're program exits. We also don't guarantee correct error semantics for the POSIX named semaphores we do have, since if we can't run the conformance tests because we don't support unnamed semaphores, we can't tell if the errors we return are correct or not. For future reference, "supported" in POSIX terms is whether or not there's a 200112L value for the manifest constant for the subssytem in <unistd.h>. This email sent to site_archiver(a)lists.apple.com

1 0

alternative to selwakeup/selrecord?
by Brendan Creane 01 Apr '09

01 Apr '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=B/RbnlWz/vlTxMQisnaqrb3i8BPyCyks/klZNnBsCaU=; b=Z/XmvCyibdV6pA/sGWFQ/qKx6Akh673DpbFI15Sv7XRlj8mALy1vsYyrWvkQxQIDxi 3NmalZNXQSsRGW96EeMjHqjCAcZpDhO1jUcxeKpHm9RWFLEEGG+77CSEIQF6kOkOtA7P krIEVlSSNUGX9uyMDPJ49fri7J4HjpeiOY7l8= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=BFh3Qj3C/yy4dE663/JaTykoNs241wz6FY/f9PiulC4V0rYNkXuRKZX8kje55a2074 dLnxGY7Gaeu85Y1jYSzOnaNtdmY771dCJgKjMp3cYwLYTkocEE1EOAjZkg9uXjqar7xv WLWscgl22/lFtT6scPqd0b9s0Vv2mA1O0pKz4= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0