August 2009 - Darwin-dev - lists.apple.com

Re: safari dynamic linking
by Jean-Daniel Dupas 26 Aug '09

26 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Le 26 août 2009 à 23:47, Joel Reymont a écrit : How can this be explained? Thanks, Joel Look like it's expected: man dyld: [1] [2] --- fastest mac firefox! http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/devlists%40shadowlab.org This email sent to devlists(a)shadowlab.org _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I'm noticing a large discrepancy between the list of dynamic libraries that Safari is linked against [2] vs the list of libraries loaded according to dyld [1]. This is even more puzzling since I saw OpenGL, etc. static initializers being run when I set DYLD_PRINT_INITIALIZERS. The OpenGL framework does not appear in the dyld output below. DYLD_PRINT_LIBRARIES_POST_LAUNCH This does the same as DYLD_PRINT_LIBRARIES but the printing starts after the program gets to its entry point. The list you get is the libraries that are loaded after the entry point is reached. DYLD_PRINT_LIBRARIES_POST_LAUNCH=1 /Applications/Safari.app/Contents/ MacOS/Safari dyld: loaded: /System/Library/CoreServices/RawCamera.bundle/Contents/ MacOS/RawCamera dyld: loaded: /System/Library/PrivateFrameworks/Shortcut.framework/ Versions/A/Shortcut dyld: loaded: /System/Library/PrivateFrameworks/HelpData.framework/ Versions/A/HelpData dyld: loaded: /System/Library/Frameworks/ ApplicationServices.framework/Versions/A/Frameworks/ CoreGraphics.framework/Versions/A/Resources/libCSync.A.dylib dyld: loaded: /System/Library/Frameworks/ ApplicationServices.framework/Versions/A/Frameworks/ CoreGraphics.framework/Versions/A/Resources/libCGATS.A.dylib dyld: loaded: /System/Library/Frameworks/ ApplicationServices.framework/Versions/A/Frameworks/ CoreGraphics.framework/Versions/A/Resources/libRIP.A.dylib dyld: loaded: /System/Library/Frameworks/ ApplicationServices.framework/Versions/A/Frameworks/ CoreGraphics.framework/Versions/A/Resources/libCGXCoreImage.A.dylib otool -L /Applications/Safari.app/Contents/MacOS/Safari / Applications/Safari.app/Contents/MacOS/Safari: /System/Library/PrivateFrameworks/CrashReporterSupport.framework/ Versions/A/CrashReporterSupport /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon /System/Library/Frameworks/WebKit.framework/Versions/A/WebKit /System/Library/Frameworks/AddressBook.framework/Versions/A/ AddressBook /System/Library/Frameworks/Security.framework/Versions/A/Security /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/ SystemConfiguration /System/Library/Frameworks/SecurityFoundation.framework/Versions/A/ SecurityFoundation /System/Library/Frameworks/SecurityInterface.framework/Versions/A/ SecurityInterface /System/Library/PrivateFrameworks/SyndicationUI.framework/Versions/ A/SyndicationUI /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/ JavaScriptCore /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit /System/Library/Frameworks/Quartz.framework/Versions/A/Quartz /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL /usr/lib/libicucore.A.dylib /usr/lib/libstdc++.6.dylib /usr/lib/libgcc_s.1.dylib /usr/lib/libSystem.B.dylib /System/Library/Frameworks/CoreServices.framework/Versions/A/ CoreServices /usr/lib/libobjc.A.dylib /System/Library/Frameworks/CoreFoundation.framework/Versions/A/ CoreFoundation /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ ApplicationServices /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation /System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore This email sent to site_archiver(a)lists.apple.com

1 0

safari dynamic linking
by Joel Reymont 26 Aug '09

26 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com How can this be explained? Thanks, Joel [1] [2] --- fastest mac firefox! http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I'm noticing a large discrepancy between the list of dynamic libraries that Safari is linked against [2] vs the list of libraries loaded according to dyld [1]. This is even more puzzling since I saw OpenGL, etc. static initializers being run when I set DYLD_PRINT_INITIALIZERS. The OpenGL framework does not appear in the dyld output below. DYLD_PRINT_LIBRARIES_POST_LAUNCH=1 /Applications/Safari.app/Contents/ MacOS/Safari dyld: loaded: /System/Library/CoreServices/RawCamera.bundle/Contents/ MacOS/RawCamera dyld: loaded: /System/Library/PrivateFrameworks/Shortcut.framework/ Versions/A/Shortcut dyld: loaded: /System/Library/PrivateFrameworks/HelpData.framework/ Versions/A/HelpData dyld: loaded: /System/Library/Frameworks/ApplicationServices.framework/ Versions/A/Frameworks/CoreGraphics.framework/Versions/A/Resources/ libCSync.A.dylib dyld: loaded: /System/Library/Frameworks/ApplicationServices.framework/ Versions/A/Frameworks/CoreGraphics.framework/Versions/A/Resources/ libCGATS.A.dylib dyld: loaded: /System/Library/Frameworks/ApplicationServices.framework/ Versions/A/Frameworks/CoreGraphics.framework/Versions/A/Resources/ libRIP.A.dylib dyld: loaded: /System/Library/Frameworks/ApplicationServices.framework/ Versions/A/Frameworks/CoreGraphics.framework/Versions/A/Resources/ libCGXCoreImage.A.dylib otool -L /Applications/Safari.app/Contents/MacOS/Safari /Applications/ Safari.app/Contents/MacOS/Safari: /System/Library/PrivateFrameworks/CrashReporterSupport.framework/ Versions/A/CrashReporterSupport /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon /System/Library/Frameworks/WebKit.framework/Versions/A/WebKit /System/Library/Frameworks/AddressBook.framework/Versions/A/AddressBook /System/Library/Frameworks/Security.framework/Versions/A/Security /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/ SystemConfiguration /System/Library/Frameworks/SecurityFoundation.framework/Versions/A/ SecurityFoundation /System/Library/Frameworks/SecurityInterface.framework/Versions/A/ SecurityInterface /System/Library/PrivateFrameworks/SyndicationUI.framework/Versions/A/ SyndicationUI /System/Library/Frameworks/JavaScriptCore.framework/Versions/A/ JavaScriptCore /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit /System/Library/Frameworks/Quartz.framework/Versions/A/Quartz /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL /usr/lib/libicucore.A.dylib /usr/lib/libstdc++.6.dylib /usr/lib/libgcc_s.1.dylib /usr/lib/libSystem.B.dylib /System/Library/Frameworks/CoreServices.framework/Versions/A/ CoreServices /usr/lib/libobjc.A.dylib /System/Library/Frameworks/CoreFoundation.framework/Versions/A/ CoreFoundation /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ ApplicationServices /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation /System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore This email sent to site_archiver(a)lists.apple.com

1 0

dyld shared cache: firefox vs safari
by Joel Reymont 26 Aug '09

26 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com --- #!/bin/bash How can I make Firefox use the dyld shared cache like Safari does? Thanks, Joel [1] http://github.com/wagerlabs/firefox-startup/tree/master [2] http://gist.github.com/175842 [3] http://gist.github.com/175843 --- fastest mac firefox! http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I wrote io.d, a DTrace script and a derivative of iosnoop, to roll up disk io time during the first 20s of the Firefox startup [1]. I already have a 'cold startup' shell script that I use to trace Firefox, e.g. cmd="./Minefield.app/Contents/MacOS/firefox-bin -no-remote -foreground -P 2" sync && purge && dtrace -x dynvarsize=64m -x evaltime=exec -c "$cmd" - wZs $1 --- I cold-started and traced both Firefox [2] and Safari [3]. I now have a hunch that the slow Firefox startup may have a lot to do with the fact that most of the Safari bits are pulled from the dyld shared cache. This email sent to site_archiver(a)lists.apple.com

1 0

Re: dtrace: catching thread yield
by Terry Lambert 26 Aug '09

26 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Aug 26, 2009, at 10:27 AM, Jamison Hope <jrh(a)theptrgroup.com> wrote: They just need to have a taller slide so that the ratio of sliding time to waiting time is improved. And they can make it a pipelined slide so that multiple children can be in transit from top to bottom simultaneously. The only problem is that if there's a blocking instruction somewhere along the line (e.g. a fat kid gets stuck), it may be difficult to flush the pipe safely. There would definitely be advantages to having parallel slides, but you know that it's inevitable that there would be a large degree of slide affinity which would keep throughput well below the theoretical maximum ("But I don't like this slide! I wanna go on the same slide that Tommy's on!"). ;-) We never solved the scalability issues for execution units, or execution window size, or tag broadcast efficiency, to make dataflow architectures competitive with Von Neumann architectures. Also, I am pretty sure no one is selling 2G sticks of content addressable memory. 8-). The perceived need for procedural affinity, on the other hand, is mostly just bad problem decomposition by people who think finite state automata are hard.. This email sent to site_archiver(a)lists.apple.com

1 0

Re: dtrace: catching thread yield
by Jamison Hope 26 Aug '09

26 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Aug 26, 2009, at 12:57 PM, Terry Lambert wrote: I would like to get a kernel stack trace whenever a thread yields. This may explain why the tread is yielding. This is done with the sched provider on Solaris but is there a way to do it on the Mac? A non-platform-dependant question: does an "involuntary preemted" thread means that it was blocked by "someone else"? ;-) -- Jamison Hope The PTR Group www.theptrgroup.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Aug 26, 2009, at 4:31 AM, Esteban Bodigami <esteban.bodigami(a)gmail.com > wrote: Terry Lambert <tlambert(a)apple.com>: On Aug 25, 2009, at 3:26 PM, Joel Reymont wrote: I thought of thread_yield_internal in the fbt provider but it didn't fire when tried. Threads generally don't yield; they either block, or they are involuntarily preempted. The following routines fbt probes will correspond roughly to the following subset of sched provider routines: Threads can only be said to have been involuntarily prempted if they are ready to run and aren't currently running. Usually it's more "circumstances" than "someone else" that preempts them. Like children in line for a slide in a playground, the ride only last so long before they have to go wait in line again for their next ride. A child can blame the other children for being between them and their next ride, but it's not "someone" who caused the ride to end at the end of the slide. I suppose you could always blame gravity or blame the playground designers for not figuring out how to fit an infinite number of slides into a small playground so each child can have their own slide and never have to wait in line. They just need to have a taller slide so that the ratio of sliding time to waiting time is improved. And they can make it a pipelined slide so that multiple children can be in transit from top to bottom simultaneously. The only problem is that if there's a blocking instruction somewhere along the line (e.g. a fat kid gets stuck), it may be difficult to flush the pipe safely. There would definitely be advantages to having parallel slides, but you know that it's inevitable that there would be a large degree of slide affinity which would keep throughput well below the theoretical maximum ("But I don't like this slide! I wanna go on the same slide that Tommy's on!"). This email sent to site_archiver(a)lists.apple.com

1 0

Re: dtrace: catching thread yield
by Terry Lambert 26 Aug '09

26 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com I would like to get a kernel stack trace whenever a thread yields. This may explain why the tread is yielding. This is done with the sched provider on Solaris but is there a way to do it on the Mac? I thought of thread_yield_internal in the fbt provider but it didn't fire when tried. A non-platform-dependant question: does an "involuntary preemted" thread means that it was blocked by "someone else"? -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Aug 26, 2009, at 4:31 AM, Esteban Bodigami <esteban.bodigami(a)gmail.com > wrote: Terry Lambert <tlambert(a)apple.com>: On Aug 25, 2009, at 3:26 PM, Joel Reymont wrote: Threads generally don't yield; they either block, or they are involuntarily preempted. The following routines fbt probes will correspond roughly to the following subset of sched provider routines: Threads can only be said to have been involuntarily prempted if they are ready to run and aren't currently running. Usually it's more "circumstances" than "someone else" that preempts them. Like children in line for a slide in a playground, the ride only last so long before they have to go wait in line again for their next ride. A child can blame the other children for being between them and their next ride, but it's not "someone" who caused the ride to end at the end of the slide. I suppose you could always blame gravity or blame the playground designers for not figuring out how to fit an infinite number of slides into a small playground so each child can have their own slide and never have to wait in line. This email sent to site_archiver(a)lists.apple.com

1 0

Re: dtrace: catching thread yield
by Wade Tregaskis 26 Aug '09

26 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Wade _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I would like to get a kernel stack trace whenever a thread yields. This may explain why the tread is yielding. This is done with the sched provider on Solaris but is there a way to do it on the Mac? System Trace, in Shark. I'm not sure exactly what you mean by yield, but it'll show you both context switches and system calls. You won't get the callstack at the time of the context switch, but if you're concerned with "explicit" yields (e.g. system calls that block), then it's exactly what you want. This email sent to site_archiver(a)lists.apple.com

1 0

Re: dtrace: catching thread yield
by Terry Lambert 26 Aug '09

26 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com ---------------- ------- fbt probe sched ---------------- ------- compute_averages sleep thread_unblock wakeup thread_continue on-cpu thread_dispatch on-cpu thread_dispatch off-cpu ---------------- ------- -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Aug 25, 2009, at 3:26 PM, Joel Reymont wrote: I would like to get a kernel stack trace whenever a thread yields. This may explain why the tread is yielding. This is done with the sched provider on Solaris but is there a way to do it on the Mac? I thought of thread_yield_internal in the fbt provider but it didn't fire when tried. Threads generally don't yield; they either block, or they are involuntarily preempted. The following routines fbt probes will correspond roughly to the following subset of sched provider routines: Note that for some of them, Mach will have discarded the kernel stack on exit. Also, the "on-cpu" is for the new thread (if there is one) and the "off-cpu" is for the old thread, so you will have to introspect the arguments, assuming they are available, in order to decode what's happening (the actualy switch takes place in thread_invoke, which is static to keep you from putting fbt probes on it; it gets inlined from thread_run and thread_block_reason). Generally, the overhead of the kernel still using type 1 probes and therefore needing to marshal and convert arguments to the probe- expected types that look like the Solaris types meant that the cost was too high to implement the sched provider in any performance path. Digging the onformation out of the functions themselves is going to mean you reading the xnu source code. This email sent to site_archiver(a)lists.apple.com

1 0

dtrace: catching thread yield
by Joel Reymont 25 Aug '09

25 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thanks, Joel --- fastest mac firefox! http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I would like to get a kernel stack trace whenever a thread yields. This may explain why the tread is yielding. This is done with the sched provider on Solaris but is there a way to do it on the Mac? I thought of thread_yield_internal in the fbt provider but it didn't fire when tried. This email sent to site_archiver(a)lists.apple.com

1 0

Re: dtrace, profile and ustack
by Joel Reymont 25 Aug '09

25 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Aug 25, 2009, at 3:36 AM, Wade Tregaskis wrote: Thanks, Joel --- fastest mac firefox! http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Though I don't know any details about the error dtrace is emitting, I would suspect it's not that the frames are paged out but that the addresses are actually invalid - such as you'd see if you have certain kinds of aggressive optimisations turned on I all have optimizations disabled (-O0) and ufunc(arg0), i.e. top of the stack does work, e.g. @[ufunc(arg0)] = count(). This email sent to site_archiver(a)lists.apple.com

1 0