April 2007 - Darwin-dev - lists.apple.com

Re: ::select() and threads question, hopefully posed better :-)
by Marc Epard 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Apr 11, 2007, at 2:41 PM, jmzorko(a)mac.com wrote: ml_set_interrupts_enabled thread_block_reason thread_block uiomove selprocess select -Marc _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… However, when I run the code, I see the OSX version taking up 4-5x (or more, sometimes 10x) as much CPU as it does under Windows .... So, I said to myself, "self, you should check out that nifty Shark app that ships with OSX, and see if you can find out what is taking all the CPU" So, I did. I launched Shark, pointed it at my process, selected Time Profile, and pressed Start. A minute or so later, it came back and showed me lots of cool stuff, but the most interesting thing was that, according to Shark, something like 60% of the time my program was running, it was in ml_set_interrupts_enabled. I said, "self, I don't know what that is, but let's see if it calls something that I do know about." So I followed the arrows, and this is what Shark told me: ... so the nature of my question is this: if 60% of the time my program is running is spent in the above, how can I optimize it? As someone else posted, a thread executing along that path is blocked waiting for something else to happen. It is not consuming CPU cycles. What's happening on the other threads? Or in the other 40% of the time on that thread? This email sent to site_archiver(a)lists.apple.com

1 0

::select() and threads question, hopefully posed better :-)
by jmzorko＠mac.com 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com All, ml_set_interrupts_enabled thread_block_reason thread_block uiomove selprocess select Regards, John Falling You - exploring the beauty of voice and sound http://www.fallingyou.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… While I appreciate the info on how to use select(), that's not the nature of my question. I'll try to explain it better :-) I've a bunch of C++ code that runs on Windows and OSX. 99% of the code is portable, and the 1% that isn't mostly has to do with file operations (seeking the POSIX way is slow on Windows, so we had to use the Win32 APIs for that stuff). Anyway, with such a large majority of our code the same across both operating systems, I was expecting to see more-or-less-comparable CPU usage. However, when I run the code, I see the OSX version taking up 4-5x (or more, sometimes 10x) as much CPU as it does under Windows -- this is on my dual 2ghz G5 as well as my 2ghz Core Duo MacBook. So, I said to myself, "self, you should check out that nifty Shark app that ships with OSX, and see if you can find out what is taking all the CPU" So, I did. I launched Shark, pointed it at my process, selected Time Profile, and pressed Start. A minute or so later, it came back and showed me lots of cool stuff, but the most interesting thing was that, according to Shark, something like 60% of the time my program was running, it was in ml_set_interrupts_enabled. I said, "self, I don't know what that is, but let's see if it calls something that I do know about." So I followed the arrows, and this is what Shark told me: ... so the nature of my question is this: if 60% of the time my program is running is spent in the above, how can I optimize it? Or am I interpreting the output of Shark correctly (for instance, does ml_set_interrupts_enabled call thread_block_reason which calls thread_block, or vice versa)? This email sent to site_archiver(a)lists.apple.com

1 0

Re: ::select() and threads
by Shawn Erickson 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Apr 10, 2007, at 8:00 PM, jmzorko(a)mac.com wrote: Josh, -Shawn _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Can I infer from this that the CPU time that the scheduler spends in my process, trying to determine which thread to run, is counted as time spent by my process e.g. is part of the time that 'top - ocpu' reports for my process actually the system scheduler checking thread states in my process? Or do I have it completely wrong? I'm just trying to understand -- I sincerely appreciate the info :-) I don't know but the time spent scheduling in likely so short you wont notice it (unless somehow you are thrashing the scheduler). Back to you question about what you saw in Shark... you can generally ignore threads that are found to spend most of their "time" blocked (thread_block, etc.). You can ask Shark to filter things like this so you can find the thread(s) that are actually doing something if you find you have something consuming a lot of CPU time. Basically a thread burning CPU will spend most of its time someplace other then in a function related to blocking a thread (Shark should sample that thread some place other then blocked). Also consider Shark's system trace feature which give a different view of how your threads are running. Often when first looking at a performance issue I will ask Shark to charge library calls to the calling process so I can find the location in my code that appear to be sampled most often by Shark. I then look these over and selectively dive down deeper if needed. This email sent to site_archiver(a)lists.apple.com

1 0

Re: ::select() and threads
by jmzorko＠mac.com 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Josh, I'm just trying to understand -- I sincerely appreciate the info :-) Regards, John Falling You - exploring the beauty of voice and sound http://www.fallingyou.com -josh On Apr 10, 2007, at 2:05 PM, jmzorko(a)mac.com wrote: Hello all ... ml_set_interrupts_enabled thread_block_reason thread_block uiomove selprocess select Regards, John Falling You - exploring the beauty of voice and sound http://www.fallingyou.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Can I infer from this that the CPU time that the scheduler spends in my process, trying to determine which thread to run, is counted as time spent by my process e.g. is part of the time that 'top -ocpu' reports for my process actually the system scheduler checking thread states in my process? Or do I have it completely wrong? This just means that when shark sampled your process, that thread was blocked (not running) waiting for an event. I've a question -- under Darwin, is select() used to determine which threads are ready to run? Shark is reporting that our app spends most of its' time in select(), but the callstack that represents the majority of select() calls isn't our select() loop, it's something else: ... which makes me think that the kernel is using select() to determine when threads are ready. This makes some sense, as our app is very multithreaded. If this is the case, how can I get our (possibly too) multithreaded app to be kinder to the CPU under OSX? This email sent to site_archiver(a)lists.apple.com

1 0

Re: Starting a Volume Filter KEXT
by Duane Murphy 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com --- At Tue, 10 Apr 2007 17:52:36 -0700, Kevin Elliott wrote: >At 17:45 -0700 on 4/10/07, Jason Townsend wrote: >>On 4/10/07, Duane Murphy <duanemurphy(a)mac.com> wrote: >>> We have a volume filter kext that we would like to start operating/ >>> matching without having to restart. >>> >>> How can we make that happen? >>> >>> I've tried a few flavors of installing the kext file, and touching / >>> System/Library/Extensions/ but when an appropriate drive is inserted, >>> the kext is not loaded. Of course restarting causes everything to work >>> just dandy. >>> >>> How do we get the kext cache to notice our new file? >>> >>> Thanks for the help, >>> ...Duane Murphy >> >>Try running "/sbin/kextload <path to your kext>". There's a man page >>for this tool as well. > >That won't work in the general case, because your kext may not match >anything at load time, and will eventually get unloaded automatically >by kextd. > >The right solution is here: ><http://developer.apple.com/qa/qa2001/qa1319.html> > >Short answer is on 10.3 or later send SIGHUP to kextd if it's >running. If it's not, require a restart. Thanks folks. This looks like the ticket. I wish there was a better way of getting the pid for kextd. Seems like using the kill API might be easier. In any case, Thanks! ...Duane _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Starting a Volume Filter KEXT
by Kevin Elliott 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com We have a volume filter kext that we would like to start operating/ matching without having to restart. How can we make that happen? I've tried a few flavors of installing the kext file, and touching / System/Library/Extensions/ but when an appropriate drive is inserted, the kext is not loaded. Of course restarting causes everything to work just dandy. How do we get the kext cache to notice our new file? Thanks for the help, ...Duane Murphy Try running "/sbin/kextload <path to your kext>". There's a man page for this tool as well. The right solution is here: <http://developer.apple.com/qa/qa2001/qa1319.html> At 17:45 -0700 on 4/10/07, Jason Townsend wrote: On 4/10/07, Duane Murphy <duanemurphy(a)mac.com> wrote: That won't work in the general case, because your kext may not match anything at load time, and will eventually get unloaded automatically by kextd. Short answer is on 10.3 or later send SIGHUP to kextd if it's running. If it's not, require a restart. -- ______________________________________________________ Arguing with an engineer is like wrestling with a pig in mud. After a while, you realize the pig is enjoying it. ______________________________________________________ Kevin Elliott <mailto:kelliott@mac.com> AIM/iChatAV: kelliott(a)mac.com (video chat available) ______________________________________________________ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Starting a Volume Filter KEXT
by Dean Reece 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com We have a volume filter kext that we would like to start operating/ matching without having to restart. How can we make that happen? I've tried a few flavors of installing the kext file, and touching / System/Library/Extensions/ but when an appropriate drive is inserted, the kext is not loaded. Of course restarting causes everything to work just dandy. How do we get the kext cache to notice our new file? Thanks for the help, ...Duane Murphy Cheers, - Dean _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On 4/10/07, Duane Murphy <duanemurphy(a)mac.com> wrote: I think you may want to have a look at: Technical Q&A QA1319 "Installing an I/O Kit KEXT Without Rebooting" <http://developer.apple.com/qa/qa2001/qa1319.html> This email sent to site_archiver(a)lists.apple.com

1 0

Re: Starting a Volume Filter KEXT
by Jason Townsend 11 Apr '07

11 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=rDd4K70ZA4OVmBwVgrXkaH6g+EKNULkXecsr0+dD42KNj8ZEvSwebBuW76++UhscJwu9+SUNyFMeQh8gRzX3np0775NiJO9r94U+z9HYB65IuJwybHqWdHu0LlnkTd3YRQ0A38M1XmMHeBwkStbSkG/y8MiKgYrWrFOCcl54Y7c= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=RhrZ0o1nkJh88vhrmEXTGm8zYY1CwIlJtCz1+D3j/b8nkkJApfSivlEhcdtlRnBkfMHS2lvQrFxGCCn+bTdDzUY13N/F0FiP6XrDS3/PQcOryt32+VprGlrcjq+z/XSt5vd0Njb0ZJ+FTMZLmEuj2+Eh3JwymwdVEOLpv2gPZAA= We have a volume filter kext that we would like to start operating/ matching without having to restart. How can we make that happen? I've tried a few flavors of installing the kext file, and touching / System/Library/Extensions/ but when an appropriate drive is inserted, the kext is not loaded. Of course restarting causes everything to work just dandy. How do we get the kext cache to notice our new file? Thanks for the help, ...Duane Murphy Try running "/sbin/kextload <path to your kext>". There's a man page for this tool as well. -Jason _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On 4/10/07, Duane Murphy <duanemurphy(a)mac.com> wrote: This email sent to site_archiver(a)lists.apple.com

1 0

Disk Utility and Mounting
by Duane Murphy 10 Apr '07

10 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com We have a volume filter kext. I noticed that in Disk Utility, the filter shows up in the list of "disks" for the volume we are attached to. There are a few things that I am trying to make nicer in this view but I am most interested in mounting. Is it possible for the filter to be notified of a "mount" event? Is that the right term here? Basically, Disk Utility can select the filter and the "Mount" button can be pressed. We'd like to act on that button press, but I'm not seeing that anything in the kext is being called. What does "Mounting" a volume filter do and how can I intercept it? Thanks, ...Duane _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Starting a Volume Filter KEXT
by Duane Murphy 10 Apr '07

10 Apr '07

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com We have a volume filter kext that we would like to start operating/ matching without having to restart. How can we make that happen? I've tried a few flavors of installing the kext file, and touching / System/Library/Extensions/ but when an appropriate drive is inserted, the kext is not loaded. Of course restarting causes everything to work just dandy. How do we get the kext cache to notice our new file? Thanks for the help, ...Duane Murphy _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0