May 2009 - Darwin-dev - lists.apple.com

hdiutil behavior in automated scripts
by Michael Plagge 29 May '09

29 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi, hdiutil: create: returning 6 hdiutil: create failed - device not configured [translated] I actually expected something like this: Thanks Michael _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I would like to use hdiutil to automatically create disk images that are to be provided by a web server running on OS 10.4.11. When testing the functionality, everything works as expected, given this example command (for a yet empty image): echo -n "pass" | /usr/bin/hdiutil create -ov -encryption -stdinpass - size 1m output.dmg -fs HFS+ The problem occurs when trying to execute this as the web server. I put this command sequence into an executable shell file which is called by PHP's exec() function. The disk image is simply not created as I get error messages like this: Supplying the -verbose and -debug parameters it comes across the following error hint: 2009-05-29 11:59:07.626 hdiutil[1970] helperClass is DIHelperNBI 2009-05-29 11:59:07.638 hdiutil[1970] error: unable to set up framework server 2009-05-29 10:56:14.809 hdiutil[1199] helperClass is DIHelperNBI 2009-05-29 10:56:14.819 hdiutil[1199] using helper tool at "/System/ Library/PrivateFrameworks/DiskImages.framework/Resources/diskimages- helper". 2009-05-29 10:56:15.287 hdiutil[1199] connectToFramework ... I would like to know if there is something environmentally that I missed out by just asking for hdiutil executed by the web server / PHP. I also came across a post from Kevin Harris (http://lists.apple.com/archives/darwin-dev/2004/Oct/msg00044.html ) that seems to have a similar background, but this has not been answered at all. The task to be fulfilled is "just" creating an encrypted disk image, filled with files from a specific location, and without doing this interactively but executed by a web server. I did not find any alternatives to hdiutil, so it would be great to know if this is possible at all. This email sent to site_archiver(a)lists.apple.com

1 0

Re: -minimize
by Esteban Barahona 29 May '09

29 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=tU0rBJuQCSjjTHC+avkdLpnI4+P3N9upYKbFEg/yhDc=; b=Lf1gFTAUFVIRjEp5G0R5o66DDVjQy2jMR5vNH1TN+WCwvN+cUFq9fZnIUFRmKTx9RL xvSBONan0cooH/+69MJpazTgZETA7BJPNw2VqXM1C2AuzgzWHrKB9AVWWRHyIkdF0Ohs yC/zCW2i0km2miMfopmsCTBzAhYGc3/YfLGxY= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Ld/gskdfsxbbgKSoDge/jZd0ChILgYV6BkfIexCQrguPrvpb0biajv2ngz02MsZKpR EzbVLt+vsWL0rnsz98fmx3KdpunD1VJinF54bg87GlNZC+U7ajaQQtmheRtB/qqVsGea iTQsGwV9CoIIQgXSExqPCRjbbvJ5cXQt7eocc= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: -minimize
by Eli Bach 29 May '09

29 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:subject :mime-version:date:references:x-mailer; bh=VR9Ro5zcQppcrhoq8LWhQvo1+H5yjzqaTpkklxZjGBI=; b=xgtv4C5dzNnOOeo//nhOH+w49tWJuyjCGWJjwZFJej0hWLgXf43lDw5sl6jEjx8d1V nVOAvKj6tPhIdboJd8mn5IpQ9K5mGHWjr7YyapWdn+YlI3hSQZROgqbNziVhbAC6jonR 11rHTJKwrPGmDko2k0J6DczGANU1r6ssyP+C8= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:in-reply-to:content-type :content-transfer-encoding:subject:mime-version:date:references :x-mailer; b=SIpQaJAejsa2aahCPg7eoBfol9V6usUBgvd3hHMWbyhF0cgwjtB2zkEylhI/DUeaUt o0adei/nSUdmufK2ta+iPVLNGSHEchvEgeyrnYytHcDIWfnCYDtCJEZUlUshuA6AwKrw Og02jb4XkRX+R0H7GURBWOhMQw6WXdUQSENao= On May 28, 2009, at 11:55 PM, Esteban Barahona wrote: 68x68 Ok Eli _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… i will get rid of the minimize button (interface enhancement request; making the dock static) ^_^ This email sent to site_archiver(a)lists.apple.com

1 0

-minimize
by Esteban Barahona 29 May '09

29 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=Bpl3tDVlBYNY5U2JF2ACTNgNssvAdaQGIK+C650c4HI=; b=lK9uOAaazSDrnbt8JSTPMHfTY6ZmMHtlumIiNGrpyjavYMMVFYpKSgTb4GvtGMuZ8C EYtKZyGIWTLYP9Aqa7mMQ5UgQSGete6PT+D5biL2gizB0EntfjD3OxqBcQD3ZritDzEC QNIlFG0UZOyehiy3kEwE6DkwcpT+FjHfryGd4= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=HYr8jAoBzX9SsDhlZQH4KZsGdfxojAFvzDpSF1s7uoZz9EHj+MQw6eU7wpmfOzOJPf YSElepc/Upr5VRp8foAm2HzMCyFunNR1uzv5b+Vohz1uUB08tF3L4Rzsw8aRCDaQAFna Nxje/cVb0plQO3zrQ5azdsGa8KG2ay+K04cws= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Why isn't pthread_rwlock_rdlock() deadlocking???
by Terry Lambert 28 May '09

28 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com pthread deadlock detection is optional in the standard. Even something as simple as two consecutive pthread_mutex_lock() calls will hang instead of returning EDEADLK. (And given the amount of code in the world that fails to check the return value, that's probably a good thing.) % nm /usr/lib/libSystem.B.dylib | grep UNIX2003 | wc -l 174 -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On May 28, 2009, at 12:16 PM, Wade Tregaskis wrote: Unfortunate but true! Are there any options that can be turned on to enable all the optional debug parts? I'm thinking something like the malloc environment variables, like MallocScribble. Otherwise, I'm going to have to roll my own test code into the wrapper that I'm writing, but that will be significantly less efficient than using a prebuilt debug library. Not that I'm aware. For mutexes you can specify an explicit attribute at initialisation time, with the type set to PTHREAD_MUTEX_ERRORCHECK (see man pthread_mutexattr_settype), which should enable deadlock detection. Alas, rwlocks don't have an equivalent attribute setting. Looking at the Darwin source, iff pthreads is compiled with __DARWIN_UNIX03 defined, then some deadlock detection is enabled - such as when trying to take a read or write on the lock if you already hold it for writing - but I don't know if it's compiled that way on Mac OS X. There are multiple compilation environments available; see the comments in <sys/cdefs.h> for details. The internal tag that you're looking at (__DARWIN_UNIX03 ) is a derived tag and should not be used directly; internally, it just means "UNIX 2003 conformat APIs plus Apple APIs not covered by the UNIX standard". The answer is that if you are targeting Leopard or later using supported development environments for Leopard or later, you're going to get the UNIX conformant behaviour by default, which will get you the checks you're wanting to get, and get you the error return. For any UNIX conformant system where the behaviour is "implementation defined", you should probably avoid using the interface the way you're trying to use it, since it's going to render your code non-portable to any implementation that defines the behaviour differently. However, for any UNIX implementation which has passed certification (like Mac OS X Leopard), you can find out what the implementation does with the implementation questionnaire responses, which is on file with The OpenGroup, <http://www.opengroup.org>. The lack of checking is required for binary backward compatibility, to not change the behaviour of programs which have, for reasons of their own, written code which deadlocks, and, rather than eliminating the deadlocks, they've come up with some mechanism of their own for dealing with them (like alarm()'ing out of the blocking call), and effectively now depend on the deadlock behaviour. Given that you are using POSIX interfaces, you should probably be using that compilation environment, but I can't tell you to target only Leopard or later with your code, that'd have to be your decision. If you don't, however, you'll either have to avoid causing deadlocks by writing standards compliant code, or you'll have to come up with your own coping mechanism for dealing with them, like other have with their pre-Leopard code. When we did the standards conformance effort, there were a lot of places where we had to maintain the old behaviour for binary backward compatibility, and provide the new behaviour in a different compilation environment, to avoid introducing binary compatibility issues. In LibSystem alone, there are 174 cases of this: Obviously, there are also cases where frameworks reflect untranslated (raw UNIX) types or values through their own APIs, and those also have similar issues that had to be addressed by their maintainers. Yeah, it's kind of annoying, if you are only coming to the UNIX table now, instead of years ago when the changes first went in, and are wanting to support OS releases from years ago without multiple binaries, but if you don't take the standards-compliant compilation environment, you don't get the protections and behavioural guarantees the environment offers you. I like to think of it as additional incentives for making the switch to standards compliant code. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Why isn't pthread_rwlock_rdlock() deadlocking???
by Karan, Cem (Civ, ARL/CISD) 28 May '09

28 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: Acnfx0N0CaCUl+dXSmW6wux4MR6ZQgACoA1g Thread-topic: Why isn't pthread_rwlock_rdlock() deadlocking??? On Thu, 28 May 2009 11:59:57 -0700 Jason Coco wrote: <<SNIP>> > > That's definitely clearer, but the man page does say that the > results of obtaining a > read or write lock when the current thread already holds the > opposite type of lock > are undefined. Since returning EDEADLOCK is optional, Apple probably > wants to choose what to do in the future, so the best choice is to > ensure that your code never > does this through review or some other mechanism (as annoying > as that is). Agreed, but I find that paranoid self-checking code is useful when someone else borrows your code and breaks your assumptions. It's happened to me before, which is why I prefer some way of building in debugging checks whenever possible. Thanks, Cem Karan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Why isn't pthread_rwlock_rdlock() deadlocking???
by Wade Tregaskis 28 May '09

28 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com pthread deadlock detection is optional in the standard. Even something as simple as two consecutive pthread_mutex_lock() calls will hang instead of returning EDEADLK. (And given the amount of code in the world that fails to check the return value, that's probably a good thing.) Wade _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Unfortunate but true! Are there any options that can be turned on to enable all the optional debug parts? I'm thinking something like the malloc environment variables, like MallocScribble. Otherwise, I'm going to have to roll my own test code into the wrapper that I'm writing, but that will be significantly less efficient than using a prebuilt debug library. Not that I'm aware. For mutexes you can specify an explicit attribute at initialisation time, with the type set to PTHREAD_MUTEX_ERRORCHECK (see man pthread_mutexattr_settype), which should enable deadlock detection. Alas, rwlocks don't have an equivalent attribute setting. Looking at the Darwin source, iff pthreads is compiled with __DARWIN_UNIX03 defined, then some deadlock detection is enabled - such as when trying to take a read or write on the lock if you already hold it for writing - but I don't know if it's compiled that way on Mac OS X. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Why isn't pthread_rwlock_rdlock() deadlocking???
by Jason Coco 28 May '09

28 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=gXplYZp/yQlYfvjumX9+MbRqMR9GqW12F6Djg8028os=; b=MBbPWkr5xiYRcaakTf/ybV+3OgZypTuU/aSjwLB7DQpoLanx/1cG8i8pM2skqswT8c UzI2et6mBHL2a2CFDRTJfNAkxYRK8D3kcotFgDsCndqJAPr2DKC+Qtpb/npYkv2CI3Qj 3+E3HJV7SJpeO3SJEzoP8E4tnJmpWlD6P1qKA= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=DXkQpzqw2R8sRIR1KTJvhU/fugi6SKosZY2cKm7jqteqhS17p2gdKBcYkyfEYQzF22 IzCRASN0RMput4QL1cedKlwXmN73UIkC4PAo1rnqtYsWB9P5vkP7gTj78NtYUOQdl6iZ ANMDvFqmBkq0fa1FenusjOeHH470BXe2mnjyg= On May 28, 2009, at 04:08 , Karan, Cem (Civ, ARL/CISD) wrote: "For each lock obtained" is supposed to mean "for each successful rdlock call". The POSIX description is more explicit: "A thread may hold multiple concurrent read locks on rwlock (that is, successfully call the pthread_rwlock_rdlock() function n times). If so, the thread must perform matching unlocks (that is, it must call the pthread_rwlock_unlock() function n times)." http://www.opengroup.org/onlinepubs/009695399/functions/pthrea d_rwlock_rdlock.html That is MUCH clearer! Unless someone has already modified the man pages, I'll file a bug report requesting that they change the page to match the above. First off, I apologize for messing up the subject line in my last post; I hit the send button a little too quickly. Greg Parker wrote on Wednesday, May 27, 2009 4:40 PM That's definitely clearer, but the man page does say that the results of obtaining a read or write lock when the current thread already holds the opposite type of lock are undefined. Since returning EDEADLOCK is optional, Apple probably wants to choose what to do in the future, so the best choice is to ensure that your code never does this through review or some other mechanism (as annoying as that is). _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Why isn't pthread_rwlock_rdlock() deadlocking???
by Karan, Cem (Civ, ARL/CISD) 28 May '09

28 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thread-index: AcnfC2o23t0IPxyBRYuq/bZi+7MYDwAdmZYg Thread-topic: Why isn't pthread_rwlock_rdlock() deadlocking??? First off, I apologize for messing up the subject line in my last post; I hit the send button a little too quickly. Greg Parker wrote on Wednesday, May 27, 2009 4:40 PM > On May 27, 2009, at 1:19 PM, Karan, Cem (Civ, ARL/CISD) wrote: > > Interesting. My interpretation was that one thread could > hold as many > > DIFFERENT r/w locks at one time as it wanted (which really confused > > me, locks would be quite useless if you COULDN'T do so, but that is > > what I got out of "...for each lock obtained") > > "For each lock obtained" is supposed to mean "for each > successful rdlock call". The POSIX description is more explicit: > > "A thread may hold multiple concurrent read locks on rwlock > (that is, successfully call the pthread_rwlock_rdlock() > function n times). If so, the thread must perform matching > unlocks (that is, it must call the pthread_rwlock_unlock() > function n times)." > > http://www.opengroup.org/onlinepubs/009695399/functions/pthrea > d_rwlock_rdlock.html That is MUCH clearer! Unless someone has already modified the man pages, I'll file a bug report requesting that they change the page to match the above. > > > Reading the code makes it much clearer though; thank you! > I did one > > other stupid test; I locked for reading and then tried to lock for > > writing. I seem to have deadlocked without an error on that one. > > Unless that is considered normal behavior, once I confirm that it > > isn't a problem with my test code, I'll file a bug about it. > > pthread deadlock detection is optional in the standard. Even > something as simple as two consecutive pthread_mutex_lock() > calls will hang instead of returning EDEADLK. (And given the > amount of code in the world that fails to check the return > value, that's probably a good thing.) Unfortunate but true! Are there any options that can be turned on to enable all the optional debug parts? I'm thinking something like the malloc environment variables, like MallocScribble. Otherwise, I'm going to have to roll my own test code into the wrapper that I'm writing, but that will be significantly less efficient than using a prebuilt debug library. Thanks, Cem Karan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kernel source code
by Esteban Barahona 28 May '09

28 May '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=QA8sIfcKWKhbmGX5wrrBVLwDunMnTMcv7jhFtF0ncgQ=; b=EEutZRk+KZWY/WsPKaiv1FM9l9EWv93RW6Te1WTC3sdCffs4W/G+drtjAuKUJ/pMbM 9BTkoSPqtJVY2knzu7JD9QtCRG+hTg+M3x4MyLmZOrld8KHmjaLqnXI+M/tmwbTRwVMt 2yqfZN535KCszfqjlAN4p+gGKwppVy0YVi3CQ= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=Jy7qQ96SCfW85m3ZjsxT9w0JneHQmZoBbm44e5lmMR90X0XtY/huts33IegkcYyv1S UfPFJo8t+RuUNLleDqS2lfFSIxeBm9HV7O40ech6XEQJVDRWG+romfj1+JT86hVgMOgM 6lDVYOO2eQ4xo8m3q9qKOXjjqvKlylj7G3QSc= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0