April 2007 - Darwin-dev - lists.apple.com

launchd question
by jmzorko＠mac.com 12 Apr '07

12 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hello, all ... Firstly, thanks for the help re: Shark ... I finally get it :-) Regards, John Falling You - exploring the beauty of voice and sound http://www.fallingyou.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Now, for my next trick, i'm going to make this app into the OSX equivalent of a Win32 service by making it run under launchd. This was really easy to do, but I noticed from the logs that this app writes, that launchd doesn't seem to be nicely shutting it down e.g. I think it just a kill -9. The app catches SIGINT and shuts down nicely when it gets it ... can I make launchd send it? This email sent to site_archiver(a)lists.apple.com

1 0

Re: Detecting case-sensitivity of a filesystem
by Nathaniel Gray 12 Apr '07

12 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Apr 9, 2007, at 4:57 PM, Nathaniel Gray wrote: I'm attaching sample code for anybody else who's trying to do this. Thanks again, -n8 -- >>>-- Nathaniel Gray -- Caltech Computer Science ------> >>>-- Mojave Project -- http://mojave.cs.caltech.edu --> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com On Apr 10, 2007, at 9:17 AM, Brian Bergstrand wrote: On Apr 9, 2007, at 11:51 PM, Nathaniel Gray wrote: On Apr 9, 2007, at 6:04 PM, Don Brady wrote: Is there any non-hackish way to detect the case-sensitivity of a filesystem, like a system call? statvfs doesn't appear to do it and statfs doesn't help either (case-sensitive and case- insensitive hfs+ have the same f_type and f_fstypename). Or is there no avoiding the "create foo, open Foo" hack? You can use pathconf(2) with the _PC_CASE_SENSITIVE selector. Note an EINVAL result for this selector generally implies case- sensitive. This works nicely for hfs+ in the case-sensitive and insensitive variants. Unfortunately it looks like EINVAL is returned by smbfs and fat32, which are both case-insensitive, and ufs, which is case- sensitive. So I guess this test is only reliable when EINVAL is not returned (probably just for hfs-family filesystems). Any other suggestions? You can get this info via the getattrlist() syscall (std disclaimer concerning code written in Mail applies): Thanks! This works really well AFAICT. I haven't tested on nfs or smbfs yet but it worked on ufs, fat32, hfs+, hfs+ case-sensitive, and afp. fscaps.c

1 0

Re: Redirecting Output for a fork()'d Process
by Duane Murphy 12 Apr '07

12 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com You need to take a look at on of the Steven's books or there must be some reference to this on the web. This is a very common thing to do and what every shell program does to redirect output. The basic idea is that after forking, you connect the stdin, stdout, stderr of the forked process to a known descriptor in the parent process. Then exec the child process. This has the affect of having all writes to the stdout in your tool go directly to a descriptor in the parent. This is really text book stuff. There should be examples somewhere. :-) Enjoy, ...Duane --- At Thu, 12 Apr 2007 10:51:37 -0500, David Alger wrote: >Hi Guys, > >I'm writing a Carbon app that needs to use fork and execv to spawn a >command line tool. I also need to be able to read the output of the tool >as it's running to provide progress bar updates. I came up with the >following, but am not sure if it's the best way (or proper) to do what I >want to accomplish. I'm also not sure if any of the duplication of file >descriptors and opening of files here is creating a memory leak. > >Basically what I'm trying to do is to have the output of the tool >redirected to a file that I will read from while it is running, but I >still want the output of my app to go to the normal stdout stream. > >In the below code error checking has been removed. Also, some variables >that are used here are declared in the function that contains this block. >_START_CODE_ > >//... > >{ > int d; > int stdoutS, stderrS; > FILE* stream; > EventLoopTimerRef timerRef; > ProgressUpdateTimerUserData ud; > > unlink(redirectPath); //remove the old output file to start with a fresh file > stream = fopen(redirectPath, "a+"); > d = fileno(stream); > > stdoutS = dup(STDOUT_FILENO); > stderrS = dup(STDERR_FILENO); > dup2(d, STDOUT_FILENO); > dup2(d, STDERR_FILENO); > > if((pid = fork()) == 0) > { > execv(commandPath, argv); > } > > dup2(stdoutS, STDOUT_FILENO); > dup2(stderrS, STDERR_FILENO); > stream = fopen(redirectPath, "r"); > > ud.pid = pid; > ud.progWindow = progWindow; > ud.stream = stream; > ud.parentWindow = window; > > //... > //Timers and event loop code here. Event loop returns on completion of >the spawned process. > //... > > fclose(stream); >} > > //... > >_END_CODE_ > >Any help is greatly appreciated. ...Duane _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Redirecting Output for a fork()'d Process
by Avi Drissman 12 Apr '07

12 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Thunderbird 1.5.0.10 (Macintosh/20070221) for (int i = STDERR_FILENO + 1; i < 1024; ++i) close(i) Not a constant, but close. getdtablesize(). Avi -- Avi Drissman avi(a)drissman.com +--------------------------------------------------------------------+ http://avi.drissman.com/ Argh! My mail server is trunca _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com On 4/12/07 12:23 PM, Marc Epard wrote: Except that there's a constant for 1024, but I'm not finding it at the moment. Also, my Linux guy told me to use fcntl(i, F_SETFD, FD_CLOEXEC) rather than close(). I don't recall the reason why. smime.p7s

1 0

Re: Redirecting Output for a fork()'d Process
by Marc Epard 12 Apr '07

12 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com <http://developer.apple.com/samplecode/QISA/listing38.html> for (int i = STDERR_FILENO + 1; i < 1024; ++i) close(i) -Marc _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… You can set up file descriptors between the call to fork and the call to execv. Since it's already in the child process, it doesn't do anything to to the descriptors in the parent. MoreIsBetter has an example: I've also read that you should close all unused file descriptors in the child process so they don't leak information into the other executable. After setting up std in/out/err, something like: Except that there's a constant for 1024, but I'm not finding it at the moment. This email sent to site_archiver(a)lists.apple.com

1 0

Redirecting Output for a fork()'d Process
by David Alger 12 Apr '07

12 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Darwin 8.8 versions
by Kevin Van Vechten 12 Apr '07

12 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com - Kevin On Mar 31, 2007, at 17:25 PM, Mo McRoberts wrote: This is possibly slightly off-topic, I apologise if there's a more appropriate list for this—I couldn't find one. I'm having a little trouble getting confirmation for this, but I seem to remember the situation being: * Darwin 8.8.0 was released (corresponding to Mac OS X 10.4.8/ppc), but didn't support x86 platforms * A short while later (2006-11-08, I think?) Darwin 8.8.1 was released, corresponding to Mac OS X 10.4.8 for Intel. Is anybody able to confirm whether this is actually the case, or if I've missed something somewhere? The darwinsource page just shows 8.8/ x86 and 8.8/ppc, though all of the differences between the two releases are increased version numbers for x86. Also, is it possible to build and run what I think is 8.8.1 (10.4.8.x86) for PPC, or does it _only_ support x86? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… 10.4.8.ppc are the sources that correspond to the 10.4.8 update for PowerPC-based Macintoshes, and 10.4.8.x86 are the sources that correspond to the 10.4.8 update for Intel-based Macintoshes. For some projects, these are the same set of sources, for others they are not. The 10.4.8.x86 sources are intended to be compiled universal, and those sources generally work on PowerPC, but keep in mind that configuration differs from what is distributed with Mac OS X for PowerPC. This email sent to site_archiver(a)lists.apple.com

1 0

Re: ::select() and threads question, hopefully posed better :-)
by Ernesto Corvi 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Ernesto. All, ml_set_interrupts_enabled thread_block_reason thread_block uiomove selprocess select Regards, _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… A blocked thread by the OS, by means of select(), read(), recv(), etc does not add up to the CPU cycles count (it's effectively sleeping, waiting for an event to occur). So you can probably safely ignore that result from Shark and move on to the other routines that are consuming the other 40% of the CPU time, and see what those routines are. Then try to figure out why are them consuming that many cycles. If the routines are OS routines, then trace back until you hit one of your program's routines, and see what calls it, and what could be wrong in that call. On Apr 11, 2007, at 3:41 PM, jmzorko(a)mac.com wrote: While I appreciate the info on how to use select(), that's not the nature of my question. I'll try to explain it better :-) I've a bunch of C++ code that runs on Windows and OSX. 99% of the code is portable, and the 1% that isn't mostly has to do with file operations (seeking the POSIX way is slow on Windows, so we had to use the Win32 APIs for that stuff). Anyway, with such a large majority of our code the same across both operating systems, I was expecting to see more-or-less-comparable CPU usage. However, when I run the code, I see the OSX version taking up 4-5x (or more, sometimes 10x) as much CPU as it does under Windows -- this is on my dual 2ghz G5 as well as my 2ghz Core Duo MacBook. So, I said to myself, "self, you should check out that nifty Shark app that ships with OSX, and see if you can find out what is taking all the CPU" So, I did. I launched Shark, pointed it at my process, selected Time Profile, and pressed Start. A minute or so later, it came back and showed me lots of cool stuff, but the most interesting thing was that, according to Shark, something like 60% of the time my program was running, it was in ml_set_interrupts_enabled. I said, "self, I don't know what that is, but let's see if it calls something that I do know about." So I followed the arrows, and this is what Shark told me: ... so the nature of my question is this: if 60% of the time my program is running is spent in the above, how can I optimize it? Or am I interpreting the output of Shark correctly (for instance, does ml_set_interrupts_enabled call thread_block_reason which calls thread_block, or vice versa)? John This email sent to site_archiver(a)lists.apple.com

1 0

Re: ::select() and threads question, hopefully posed better :-)
by Mark F. Murphy 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com It might give a glue if the two ports are vastly different. mark -- --------------------------------------------------------------------------- Mark F. Murphy, Director Software Development <mailto:markm@tyrell.com> Tyrell Software Corp <http://www.tyrell.com> --------------------------------------------------------------------------- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… At 12:41 PM -0700 4/11/07, jmzorko(a)mac.com wrote: I've a bunch of C++ code that runs on Windows and OSX. 99% of the code is portable, and the 1% that isn't mostly has to do with file operations (seeking the POSIX way is slow on Windows, so we had to use the Win32 APIs for that stuff). Anyway, with such a large majority of our code the same across both operating systems, I was expecting to see more-or-less-comparable CPU usage. However, when I run the code, I see the OSX version taking up 4-5x (or more, sometimes 10x) as much CPU as it does under Windows -- this is on my dual 2ghz G5 as well as my 2ghz Core Duo MacBook. Usually when porting apps of this nature and run into issues such as this, you have to look to see if calls are executing in the same way on the new target environment. My guess without seeing your code (and it seems others are guessing this as well)... is that your call to select() is returning a lot more on Mac OS X than it does on your Windows version. Usually when you see CPU being churned in this way, it means there's a tight loop occurring (which may or may not be valid depending on your app). The loop that might be occurring is a return from select() in Mac OS X when such a return would not occur in Windows. In other words, Windows is blocking where as Mac OS X shows different behavior, is not blocking, and returns. Have you compared how many times you're calling select() in Mac OS X versus how many times you call it in Windows? This email sent to site_archiver(a)lists.apple.com

1 0

Re: ::select() and threads question, hopefully posed better :-)
by Josh Graessley 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Apr 11, 2007, at 12:41 PM, jmzorko(a)mac.com wrote: <snip> ml_set_interrupts_enabled thread_block_reason thread_block uiomove selprocess select -josh _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com I said, "self, I don't know what that is, but let's see if it calls something that I do know about." So I followed the arrows, and this is what Shark told me: ... so the nature of my question is this: if 60% of the time my program is running is spent in the above, how can I optimize it? Or am I interpreting the output of Shark correctly (for instance, does ml_set_interrupts_enabled call thread_block_reason which calls thread_block, or vice versa)? That means that 60% of the time shark stepped in to sample your thread, it was blocked doing nothing. If all you're concerned about is reducing CPU, try to increase the % of samples that catch your thread in thread_block. The longer a thread is stuck in thread_block, the less CPU it will use. In general, that's where the thread goes to twiddle its thumbs because it doesn't need to do anything. Look at the top hits that aren't somewhere in thread_block and see if you can speed up those functions or call them less often. smime.p7s

1 0