June 2004 - Darwin-kernel - lists.apple.com

Re: boundary crossing latencies
by Andrew Gallatin 24 Jun '04

24 Jun '04

Jim Magee writes: > On Jun 22, 2004, at 5:30 PM, Andrew Gallatin wrote: > > I found that MacOSX 10.3.3's ioctl takes nearly 4x as long as the same > > code under linux ppc64 on the same exact G5. And it takes 8x as long > > as > > the same ioctl on an AMD64. > > Mac OS X/Darwin-ppc provides a full 32-bit address space to > applications (and a separate full 32-bit address space to the kernel). > Each transition from user to kernel (and back) requires an address > space change. Linux (and others) take 1GB (or more) away from the > application address space so they can map the kernel as part of each > address space to avoid this overhead (they just have a privilege-level > transition to make instead). Mac OS X can't afford to take that much > address space away from some applications. So, we pay a higher > transition cost. ppc64 linux provides a fully 64-bit kernel with much more than the 4GB of address space provided by MacOSX. So its not loosing anything, rather its gaining performance. Not having to pay the price to swap address spaces would be just one benefit moving to a real 64-bit kernel ;) > We also use a single kernel for both SMP and UP systems. The SMP > locking in the kernel imposes some additional overhead that isn't > always needed (or seen in those other OSes). This was an SMP linux kernel. I beleive there is at lease one lock in the ioctl path. > These issues aside, there are still plenty of things we can (and are) > doing to improve those latencies. But there always seem to have been > bigger performance fish to fry. One quick way to speed up ioctls would be to hang a driver ioctl right off of fops, and not force every ioctl to traverse the vnode layer. > --Jim > > PS: When using an "averaging" benchmark like you describe, don't forget > to take those measurements in single-user mode, network disconnected. > Background activity plays an important part of the final result in such > cases. Mac OS X, in full multi-user mode, tends to have a little bit > more of this than the typical OS (because of all the niceties like > Rendezvous, etc...) - throwing the numbers off slightly (or sometimes > a lot). I don't intend to use the system in single user mode with the network disconnected, so I don't think benchmarking it like that is realistic. I ran a huge number of iterations to factor background activity in. Both systems were sitting at the login prompt (no gui) and this was done via an ssh session. BTW, you should see the jitter under MacOSX in our (OS-bypass, non-IP, non ethernet) ping-pong latency benchmarks. Huge jumps. Linux is stable run-to-run, MacOSX is all over the map. I don't want to sound totally negative. I'm not a linux biggot at all (more of a BSD one) There are pleny of good things in MacOSX. For example, the MacOSX kernel is excellent at precise sleep times. Its totally amazing. You tell it to wake up in 1ms, and it it bloody well *does*. You're lucky if you can get 20ms granularity in any other OS. Kudos to whoever did the MacOSX sleep/wakeup code. Drew _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: boundary crossing latencies
by Jim Magee 24 Jun '04

24 Jun '04

On Jun 22, 2004, at 5:30 PM, Andrew Gallatin wrote: > I found that MacOSX 10.3.3's ioctl takes nearly 4x as long as the same > code under linux ppc64 on the same exact G5. And it takes 8x as long > as > the same ioctl on an AMD64. Mac OS X/Darwin-ppc provides a full 32-bit address space to applications (and a separate full 32-bit address space to the kernel). Each transition from user to kernel (and back) requires an address space change. Linux (and others) take 1GB (or more) away from the application address space so they can map the kernel as part of each address space to avoid this overhead (they just have a privilege-level transition to make instead). Mac OS X can't afford to take that much address space away from some applications. So, we pay a higher transition cost. We also use a single kernel for both SMP and UP systems. The SMP locking in the kernel imposes some additional overhead that isn't always needed (or seen in those other OSes). These issues aside, there are still plenty of things we can (and are) doing to improve those latencies. But there always seem to have been bigger performance fish to fry. --Jim PS: When using an "averaging" benchmark like you describe, don't forget to take those measurements in single-user mode, network disconnected. Background activity plays an important part of the final result in such cases. Mac OS X, in full multi-user mode, tends to have a little bit more of this than the typical OS (because of all the niceties like Rendezvous, etc...) - throwing the numbers off slightly (or sometimes a lot). [demime 0.98b removed an attachment of type application/pkcs7-signature which had a name of smime.p7s] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: struct proc definition
by Justin Walker 24 Jun '04

24 Jun '04

On Jun 24, 2004, at 9:11, Matthew Jaffa wrote: search for proc.h on your machine in User space it will be something like usr/include/sys/proc.h and in kernel space Frameworks/Kernel.Framework/......./sys/proc.h <---- pick among the search results Just to clarify Matthew's response: there is no user/kernel space difference here. The file 'proc.h' defines the proc structure, of which there is only one. The file can be found in "/usr/include/sys", for access by the normal C compiler operation. It is also in the Kernel framework, for access by Xcode's KEXT build scheme. This also lays the groundwork for building KEXTs for different versions of the kernel, but that is a story for another time. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | If you're not confused, | You're not paying attention *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re-routing System Calls
by Krishna Monian 24 Jun '04

24 Jun '04

Hi All I am in the process of rerouting system calls to my own versions. Most of the calls are related to the file system. Could someone tell me where I could get the signatures for all the functions for which there is a define in the syscall.h header. Thanks Krishna __________________________________ Do you Yahoo!? Yahoo! Mail - 50x more storage than other providers! http://promotions.yahoo.com/new_mail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Little more about mbufs
by Peter Lovell 24 Jun '04

24 Jun '04

On Jun 24, 2004, at 11:04 AM, Matthew Jaffa wrote: Hi, I have read chapter 2 about mbufs in TCP/IP books by Stevens. I am trying to get a little more understanding. My NKE is intercepting POST and GET Requests, normally these are within the range of 200 - 1500 in size so that it is easy to get all of the information from the mbuf by just doing char * data = mtod(mbuf,char*); But I now am dealing with POSTS that are huge, when I submit a huge POST form data is big. The first POST Request I intercept however is of the size > 2051, but when I look at outgoing traffic using ethereal each one is made sure to be split into chunks less than 2051, so how come the mbuf's m_pkthdr.len is greater then 2051 when it is only storing 2051, does this mean that there is another mbuf chained to it that I must then get the remaining data for that POST that has m_pkthdr.len > 2051? Thanks, Matt There's a big difference between m_pkthdr.len and m_len. Have a close look at the diagram on page 35 and it should help explain where the various bits of the packet are located. Regards.....Peter _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Loading kerenl extensions
by Justin Walker 24 Jun '04

24 Jun '04

On Jun 24, 2004, at 7:40, Krishna Monian wrote: [snip] kextload: extension xxx/HelloKernel.kext is not authentic. Could someone tell me the reason for this. Is it due to the version difference or due to some other reason? I think the error message tells you the problem: there is an authentication problem. This means, in all likelihood, that you haven't obeyed the rules when installing the extension. The extension has to be owned by root (and writable only by root). Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | When LuteFisk is outlawed | Only outlaws will have | LuteFisk *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

RE: struct proc definition
by Matthew Jaffa 24 Jun '04

24 Jun '04

search for proc.h on your machine in User space it will be something like usr/include/sys/proc.h and in kernel space Frameworks/Kernel.Framework/......./sys/proc.h <---- pick among the search results Matt From: Krishna Monian <k_tutorials(a)yahoo.com> To: darwin-kernel(a)lists.apple.com Subject: struct proc definition Date: Thu, 24 Jun 2004 08:29:17 -0700 (PDT) Hi All, Can someone tell me where I can find the definition for struct proc. I see it being used in so many places, but I can't seem to find the definition. Thanks __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - 100MB free storage! http://promotions.yahoo.com/new_mail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Loading kerenl extensions
by Peter Lovell 24 Jun '04

24 Jun '04

On Jun 24, 2004, at 10:40 AM, Krishna Monian wrote: Hi All, I have currently written a Kernel extension on my development machine which has Mac OS X 10.3.4 (Build 7H63) running on it. It seems to run fine on this machine. However when I try to load it on another machine which has Mac OS X 10.3.3 (Build 7F44) I get the following message. kextload: extension xxx/HelloKernel.kext is not authentic. Could someone tell me the reason for this. Is it due to the version difference or due to some other reason? Thanks Krishna It's almost certainly an error with ownership and permissions of the kext. They probably changed when you copied it to the other machine. Correct those and then it should load. Regards.....Peter _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

struct proc definition
by Krishna Monian 24 Jun '04

24 Jun '04

Hi All, Can someone tell me where I can find the definition for struct proc. I see it being used in so many places, but I can't seem to find the definition. Thanks __________________________________ Do you Yahoo!? New and Improved Yahoo! Mail - 100MB free storage! http://promotions.yahoo.com/new_mail _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Little more about mbufs
by Matthew Jaffa 24 Jun '04

24 Jun '04

Hi, I have read chapter 2 about mbufs in TCP/IP books by Stevens. I am trying to get a little more understanding. My NKE is intercepting POST and GET Requests, normally these are within the range of 200 - 1500 in size so that it is easy to get all of the information from the mbuf by just doing char * data = mtod(mbuf,char*); But I now am dealing with POSTS that are huge, when I submit a huge POST form data is big. The first POST Request I intercept however is of the size > 2051, but when I look at outgoing traffic using ethereal each one is made sure to be split into chunks less than 2051, so how come the mbuf's m_pkthdr.len is greater then 2051 when it is only storing 2051, does this mean that there is another mbuf chained to it that I must then get the remaining data for that POST that has m_pkthdr.len > 2051? Thanks, Matt _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0