December 2007 - Darwin-dev - lists.apple.com

Re: link(2) allows directory hard links, man page says otherwise
by Rosyna 13 Dec '07

13 Dec '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com It's not my intention to ever intentionally create directory hard links. On Dec 12, 2007, at 4:37 PM, Jordan K. Hubbard wrote: - Jordan On Dec 12, 2007, at 3:12 PM, Rosyna wrote: --- Sincerely, Rosyna Keller Technical Support/Carbon troll/Always needs a hug Unsanity: Unsane Tools for Insanely Great People It's either this, or imagining Phil Schiller in a thong. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This is true. And directory hard links are a concept I do not like for various reasons. So, I was really pleased when people at WWDC and other places repeatedly said (or strongly implied) Time Machine used a private function to make directory hard links, that this SPI would not be exposed, and developers should not be making directory hard links. But alas, if some developer has code that depends on link() failing with EPERM for a directory (they don't check the path first), their code will magically start working on HFS+ on 10.5. My hope was that time machine would call some super private function (like hfs_super_private_link) that exposes hfs_vnop_link() to do its dirty deeds that no one else could, should, or can call. You're not supposed to notice that directory hard links are now allowed (for Time Machine's exclusive use) so it's neither an error in the documentation or an error in the function. The error is that you're being too nosy. :-) The actual implementation of link() on 10.5 allows for directory hard links on HFS+ as long as they follow these rules: This email sent to site_archiver(a)lists.apple.com

1 0

Re: link(2) allows directory hard links, man page says otherwise
by Rosyna 13 Dec '07

13 Dec '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com I quoted it from Apple's man page x-man-page://2/link/ or http://developer.apple.com/documentation/Darwin/Reference/ManPages/man2/lin… Which seems to imply Apple's man page is out of date, On Dec 12, 2007, at 4:35 PM, Mike Frysinger wrote: the POSIX standard you reference is outdated. please refer to: http://www.opengroup.org/onlinepubs/009695399/functions/link.html where it quotes: If path1 names a directory, link() shall fail unless the process has appropriate privileges and the implementation supports using link() on directories. and under errors it states: [EPERM] The file named by path1 is a directory and either the calling process does not have appropriate privileges or the implementation prohibits using link() on directories. --- Sincerely, Rosyna Keller Technical Support/Carbon troll/Always needs a hug Unsanity: Unsane Tools for Insanely Great People It's either this, or imagining Phil Schiller in a thong. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: link(2) allows directory hard links, man page says otherwise
by Jordan K. Hubbard 12 Dec '07

12 Dec '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com - Jordan On Dec 12, 2007, at 3:12 PM, Rosyna wrote: The man page for link(2) says (in various places): As mandated by POSIX.1, path1 may not be a directory. [EPERM] The file named by path1 is a directory. --- Sincerely, Rosyna Keller Technical Support/Carbon troll/Always needs a hug Unsanity: Unsane Tools for Insanely Great People It's either this, or imagining Phil Schiller in a thong. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/jkh%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… You're not supposed to notice that directory hard links are now allowed (for Time Machine's exclusive use) so it's neither an error in the documentation or an error in the function. The error is that you're being too nosy. :-) The actual implementation of link() on 10.5 allows for directory hard links on HFS+ as long as they follow these rules: - Source parent and destination parent cannot match - A link is not permitted in the root directory - Parent of 'pointed at' directory is not the root directory - The 'pointed at' directory (source) is not an ancestor of the new directory hard link (destination). - No ancestor of the new directory hard link (destination) is a directory hard link. STANDARDS The link() function is expected to conform to IEEE Std 1003.1-1988 (``POSIX.1''). But actually calling link() allows for directory hard links. Is this an error in the documentation, an error in the function, or am I connecting two things that shouldn't be? This email sent to jkh(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: link(2) allows directory hard links, man page says otherwise
by Mike Frysinger 12 Dec '07

12 Dec '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=yob4VeNKMJ5urJUmL2diXrw+D0da6VTWXOLCA6c5KkU=; b=r4Pk/vLROktlAPPvIQtDV1NPkY5AQstDE9DRaBoTRu/PPIjZWNkJAZ7Hu6SJWFI7YGfvTu/gFiKN5EnpT7h5veHPahPzw29vXHWIdfMxo+/MiweSMFZ96QNLRuCI2/lI6yP3TB2hKFF5YiFr88cLeQFKsBTYr4HHQ5YmfUhcnGM= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=gFafCdcrtfpu4vAATQgEd61NJR+y4IgflWWHx45EgNmlIVQYtMEeFvZI44EBov9EQ10Ky3HIokKy9hHBYVSPwsrTiiv92pX7Rn16JhICzCaCPnlwKbowBh3BDRc5AXB9ojcC0J1OFHdN1lgup9cko7f20tAbx6keToavO7w5p2k= On Dec 12, 2007 6:12 PM, Rosyna <rosyna(a)unsanity.com> wrote: > The actual implementation of link() on 10.5 allows for directory hard > links on HFS+ as long as they follow these rules: > > - Source parent and destination parent cannot match > - A link is not permitted in the root directory > - Parent of 'pointed at' directory is not the root directory > - The 'pointed at' directory (source) is not an ancestor of the new > directory hard link (destination). > - No ancestor of the new directory hard link (destination) is a > directory hard link. > > The man page for link(2) says (in various places): > > As mandated by POSIX.1, path1 may not be a directory. > > > [EPERM] The file named by path1 is a directory. > > > STANDARDS > The link() function is expected to conform to IEEE Std > 1003.1-1988 (``POSIX.1''). > > > > But actually calling link() allows for directory hard links. Is this > an error in the documentation, an error in the function, or am I > connecting two things that shouldn't be? the POSIX standard you reference is outdated. please refer to: http://www.opengroup.org/onlinepubs/009695399/functions/link.html where it quotes: If path1 names a directory, link() shall fail unless the process has appropriate privileges and the implementation supports using link() on directories. and under errors it states: [EPERM] The file named by path1 is a directory and either the calling process does not have appropriate privileges or the implementation prohibits using link() on directories. -mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

link(2) allows directory hard links, man page says otherwise
by Rosyna 12 Dec '07

12 Dec '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com The man page for link(2) says (in various places): As mandated by POSIX.1, path1 may not be a directory. [EPERM] The file named by path1 is a directory. --- Sincerely, Rosyna Keller Technical Support/Carbon troll/Always needs a hug Unsanity: Unsane Tools for Insanely Great People It's either this, or imagining Phil Schiller in a thong. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… The actual implementation of link() on 10.5 allows for directory hard links on HFS+ as long as they follow these rules: - Source parent and destination parent cannot match - A link is not permitted in the root directory - Parent of 'pointed at' directory is not the root directory - The 'pointed at' directory (source) is not an ancestor of the new directory hard link (destination). - No ancestor of the new directory hard link (destination) is a directory hard link. STANDARDS The link() function is expected to conform to IEEE Std 1003.1-1988 (``POSIX.1''). But actually calling link() allows for directory hard links. Is this an error in the documentation, an error in the function, or am I connecting two things that shouldn't be? This email sent to site_archiver(a)lists.apple.com

1 0

Re: getrusage only partially working?
by Mike Fischer 12 Dec '07

12 Dec '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi Terry, Am 12.12.2007 um 18:03 schrieb Terry Lambert: Hi Terry, Am 12.12.2007 um 03:27 schrieb Terry Lambert: Hi! Ah ok. That makes much more sense. Ok. Done: rdar://5643393 Understood. Not a very good workaround but certainly doable IMHO. See above. Thanks for taking the time to explain! Mike -- Mike Fischer Softwareentwicklung, EDV-Beratung Schulung, Vertrieb Note: I read this list in digest mode! Send me a private copy for faster responses. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Dec 12, 2007, at 12:21 AM, Mike Fischer <mike_fischer(a)mac.com> wrote: On Dec 11, 2007, at 7:26 AM, Mike Fischer wrote: For some reason I can't seem to get getrusage(2) to work the way I expect it to under 10.4.11 on a PPC G5 machine (haven't tested on other machines). I expected the memory related fields to show values other than 0 which more or less correlate with the actual memory usage of the application. But what I am seeing is values of 0 only in each of these fields (ru_maxrss, ru_ixrss, ru_idrss, ru_isrss). Probably I'm doing something wrong, but what? Most of the information you are requesting in Mac information, and you are requesting it using a BSD API. Generally I don't see anything wrong with that. Why wouldn't I call BSD APIs in a command line tool? (Also your sentence structure is not entirely clear to me so I may be missing some of the meaning here?) Sorry, iPhone correction-itis. "is Mach information". The BSD portion of the kernel does not maintain separate, parallel state information for this Mach data, as doing so would be prohibitively expensive. As a result, the only way to get the information is to post-process the corresponding Mach data, and to avoid repeated expense, this is only done after process termination, to provide totals, rather than a running tally. What I fail to see is why tools like top(1) or ps(1) can get this information while getrusage(2) can't. Obviously top and ps don't rely on any parallel state information to be kept up to date. They can, it seems, access the Mach data somehow. Given that all three are implemented by Apple, why can't getrusage use the same underlying data as well? As for the performance, as a developer I would find a strongly worded warning in the documentation to be much more helpful since I can then decide if the function call is worth the performance penalty. (Assuming of course that this penalty is incurred only when calling getrusage, not otherwise.) The BSD API idea of the information will not be updated until the process exits (see xnu/bsd/kern/kern_exit.c:proc_exit()). Err, in that case how would the call ever work with the given option RUSAGE_SELF? Generally, it would not. Our command line (and GUI) performance tools tend to use Mach, rather than BSD, APIs for reporting most information like this, if that information needs to be collected at runtim, rather than on a child process. Seems like a chicken and egg problem to me: the info is updated when the process exits but since the process has exited it can't call getrusage any more. At the very least the man page could be a bit more clear about this. File a bug; most likely we will change the man page to deprecate use of this call, and point you at our performance tools instead. If you want this information while the process is running, you're going to need link against the whole kernel and use unsupported APIs. No thanks, I don't want the info that badly ;-) I was just experimenting to see if I could access similar values to what top (1) or Activity Monitor report inside my application. And since they can do this while my app is running I thought the process itself should be able to as well. These tools use a combination of APIs, SPIs, and unstable interfaces which can change from release to release, even in software updates. The reason it's ok for us to do this is that when we modify the means of getting the information, we are able to modify system supplied tools in lock-step with kernel modifications, so the tools keep working. OK, but getrusage is implemented by Apple as well. So it could be kept in sync with the "combination of APIs, SPIs, and unstable interfaces" just like top and ps. This is the primary reason I suggested a popen of a "ps -o<list of fields> -<scoping options>" would probably be a more or less portable way of getting state information on processes. At least you know that to keep certification, we have to keep supporting the options POSIX requires, and if we change the way the ps program obtains information, we'll change the ps program itself in lockstep with the kernel/line changes. Needless to say, this will end up being rather expensive, which is why we don't export them via BSD in the first place. Yeah, that makes sense. But given the documentation of getrusage my expectations where different. Should I file a bug? If so, against getrusage You can; the most likely response, as noted above, will be to modify the man page to tell you not to use it. I understand people wanting to write their own performance tools, but people who do this also need to understand that doing so will likely tie them to the same constraints Apple has with regard to using SPIs and unstable interfaces. This translates to them potentially having to recompile their tools on each software update, just like we frequently have to do, to keep them working, unless they wrap out tools. This email sent to site_archiver(a)lists.apple.com

1 0

Re: getrusage only partially working?
by Steve Thompson 12 Dec '07

12 Dec '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Wed, 12 Dec 2007, Terry Lambert wrote: Steve _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I understand people wanting to write their own performance tools, but people who do this also need to understand that doing so will likely tie them to the same constraints Apple has with regard to using SPIs and unstable interfaces. What I need to do is the capability, while a process is running, is to determine the total CPU time used by that process and any of its children, _while the children are still running_. Obviously I can easily determine the CPU time used by each child while it is running by summing up the thread times and the terminated thread times, but there does not appear to be a way to determine, for each still-running child, what the CPU time used by that child's own terminated children is. In short, I need to calculate the total CPU used by a process while it is running, including all children and children of children (and so on), both terminated and still running. This is critical for my application. Is there any way to do this at all? I don't care if the method is version specific or not. On Tiger, I do a sysctl() to get the kinfo_proc structure for all processes of interest, and use a kvm_read(kinfo_proc.kp_eproc.e_paddr) to get the p_stats structure of interest. This doesn't appear to be possible anymore on Leopard, and from what I read, may not point to valid data even if I could get at it. This email sent to site_archiver(a)lists.apple.com

1 0

Re: getrusage only partially working?
by Terry Lambert 12 Dec '07

12 Dec '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi Terry, Am 12.12.2007 um 03:27 schrieb Terry Lambert: Hi! -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Dec 12, 2007, at 12:21 AM, Mike Fischer <mike_fischer(a)mac.com> wrote: On Dec 11, 2007, at 7:26 AM, Mike Fischer wrote: For some reason I can't seem to get getrusage(2) to work the way I expect it to under 10.4.11 on a PPC G5 machine (haven't tested on other machines). I expected the memory related fields to show values other than 0 which more or less correlate with the actual memory usage of the application. But what I am seeing is values of 0 only in each of these fields (ru_maxrss, ru_ixrss, ru_idrss, ru_isrss). Probably I'm doing something wrong, but what? Most of the information you are requesting in Mac information, and you are requesting it using a BSD API. Generally I don't see anything wrong with that. Why wouldn't I call BSD APIs in a command line tool? (Also your sentence structure is not entirely clear to me so I may be missing some of the meaning here?) Sorry, iPhone correction-itis. "is Mach information". The BSD portion of the kernel does not maintain separate, parallel state information for this Mach data, as doing so would be prohibitively expensive. As a result, the only way to get the information is to post-process the corresponding Mach data, and to avoid repeated expense, this is only done after process termination, to provide totals, rather than a running tally. The BSD API idea of the information will not be updated until the process exits (see xnu/bsd/kern/kern_exit.c:proc_exit()). Err, in that case how would the call ever work with the given option RUSAGE_SELF? Generally, it would not. Our command line (and GUI) performance tools tend to use Mach, rather than BSD, APIs for reporting most information like this, if that information needs to be collected at runtim, rather than on a child process. Seems like a chicken and egg problem to me: the info is updated when the process exits but since the process has exited it can't call getrusage any more. At the very least the man page could be a bit more clear about this. File a bug; most likely we will change the man page to deprecate use of this call, and point you at our performance tools instead. If you want this information while the process is running, you're going to need link against the whole kernel and use unsupported APIs. No thanks, I don't want the info that badly ;-) I was just experimenting to see if I could access similar values to what top(1) or Activity Monitor report inside my application. And since they can do this while my app is running I thought the process itself should be able to as well. These tools use a combination of APIs, SPIs, and unstable interfaces which can change from release to release, even in software updates. The reason it's ok for us to do this is that when we modify the means of getting the information, we are able to modify system supplied tools in lock-step with kernel modifications, so the tools keep working. This is the primary reason I suggested a popen of a "ps -o<list of fields> -<scoping options>" would probably be a more or less portable way of getting state information on processes. At least you know that to keep certification, we have to keep supporting the options POSIX requires, and if we change the way the ps program obtains information, we'll change the ps program itself in lockstep with the kernel/line changes. Needless to say, this will end up being rather expensive, which is why we don't export them via BSD in the first place. Yeah, that makes sense. But given the documentation of getrusage my expectations where different. Should I file a bug? If so, against getrusage You can; the most likely response, as noted above, will be to modify the man page to tell you not to use it. I understand people wanting to write their own performance tools, but people who do this also need to understand that doing so will likely tie them to the same constraints Apple has with regard to using SPIs and unstable interfaces. This translates to them potentially having to recompile their tools on each software update, just like we frequently have to do, to keep them working, unless they wrap out tools. This email sent to site_archiver(a)lists.apple.com

1 0

Re: getrusage only partially working?
by Mike Fischer 12 Dec '07

12 Dec '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi Terry, Am 12.12.2007 um 03:27 schrieb Terry Lambert: Hi! Should I file a bug? If so, against getrusage or its documentation? Thanks! Mike -- Mike Fischer Softwareentwicklung, EDV-Beratung Schulung, Vertrieb Note: I read this list in digest mode! Send me a private copy for faster responses. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Dec 11, 2007, at 7:26 AM, Mike Fischer wrote: For some reason I can't seem to get getrusage(2) to work the way I expect it to under 10.4.11 on a PPC G5 machine (haven't tested on other machines). I expected the memory related fields to show values other than 0 which more or less correlate with the actual memory usage of the application. But what I am seeing is values of 0 only in each of these fields (ru_maxrss, ru_ixrss, ru_idrss, ru_isrss). Probably I'm doing something wrong, but what? Most of the information you are requesting in Mac information, and you are requesting it using a BSD API. Generally I don't see anything wrong with that. Why wouldn't I call BSD APIs in a command line tool? (Also your sentence structure is not entirely clear to me so I may be missing some of the meaning here?) The BSD API idea of the information will not be updated until the process exits (see xnu/bsd/kern/kern_exit.c:proc_exit()). Err, in that case how would the call ever work with the given option RUSAGE_SELF? Seems like a chicken and egg problem to me: the info is updated when the process exits but since the process has exited it can't call getrusage any more. At the very least the man page could be a bit more clear about this. If you want this information while the process is running, you're going to need link against the whole kernel and use unsupported APIs. No thanks, I don't want the info that badly ;-) I was just experimenting to see if I could access similar values to what top(1) or Activity Monitor report inside my application. And since they can do this while my app is running I thought the process itself should be able to as well. Needless to say, this will end up being rather expensive, which is why we don't export them via BSD in the first place. Yeah, that makes sense. But given the documentation of getrusage my expectations where different. This email sent to site_archiver(a)lists.apple.com

1 0

Re: getrusage only partially working?
by Terry Lambert 12 Dec '07

12 Dec '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi! -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Dec 11, 2007, at 7:26 AM, Mike Fischer wrote: For some reason I can't seem to get getrusage(2) to work the way I expect it to under 10.4.11 on a PPC G5 machine (haven't tested on other machines). I expected the memory related fields to show values other than 0 which more or less correlate with the actual memory usage of the application. But what I am seeing is values of 0 only in each of these fields (ru_maxrss, ru_ixrss, ru_idrss, ru_isrss). Probably I'm doing something wrong, but what? Most of the information you are requesting in Mac information, and you are requesting it using a BSD API. The BSD API idea of the information will not be updated until the process exits (see xnu/bsd/kern/kern_exit.c:proc_exit()). If you want this information while the process is running, you're going to need link against the whole kernel and use unsupported APIs. Needless to say, this will end up being rather expensive, which is why we don't export them via BSD in the first place. This email sent to site_archiver(a)lists.apple.com

1 0