December 2003 - Darwin-kernel

Finder copy chunk size
by Sam Vaughan 03 Dec '03

03 Dec '03

When doing a copy to/from our file system, the Finder issues the writes/reads in tiny 12k chunks. Since it sets the VNOCACHE_DATA flag, we do direct I/O to the disk to get the best performance. The problem is that we could provide vastly better performance if the chunk sizes were much larger. Chunks of 4MB or more would be ideal, but even increasing to 128k would improve performance by up to a factor of 8. Does the Finder have any logic to decide what size transfers to issue? Does it call any VFS layer hooks that we could implement? Thanks, Sam _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Network Kernel Extensions
by Matt Jaffa 03 Dec '03

03 Dec '03

Hello, I am looking for a pdf file that Mac OS X use to have on their site, the address was http://developer.apple.com/techpubs/macosx/Darwin/Extensions/NKE.pdf The document is called Inside Mac OS X:Network Kernel Extensions Is there anywhere else i can find this document. Thanks, Matt On Dec 2, 2003, at 3:01 AM, Quinn wrote: > Greetings > > I'm trying to find out why developers are patching the kernel system > call dispatch table (sysent). If your product patches sysent, can you > please drop me a line explaining what your product does and why it > patches sysent. > > S+E > -- > Quinn "The Eskimo!" > <http://www.apple.com/developer/> > Apple Developer Technical Support * Networking, Communications, > Hardware > _______________________________________________ > darwin-kernel mailing list | darwin-kernel(a)lists.apple.com > Help/Unsubscribe/Archives: > http://www.lists.apple.com/mailman/listinfo/darwin-kernel > Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

more on API docs
by Jeff Nathan 02 Dec '03

02 Dec '03

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The API docs within the xnu source distribution are man pages that have been converted to HTML documents. For those using traditional development tools (vim and emacs in particular) It would be particularly handy if these were actually available as man pages. Anyone else share my opinion? - -Jeff - -- http://cerberus.sourcefire.com/~jeff (gpg/pgp key id 6923D3FD) "Great spirits have always encountered violent opposition from mediocre minds." - Albert Einstein -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (Darwin) iD8DBQE/zPZOEqr8+Gkj0/0RAtRyAKCaNKzBzonM1i23QSueHFvAdLj+xACfac4b 1LndSMyHH7ZKA8lKmQe8Hpk= =yHVq -----END PGP SIGNATURE----- _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Clock Resolution on MAC Os X
by Johnsy George 02 Dec '03

02 Dec '03

Hi , Could anyone please tell me what's the minimum clock resolution(timertick settings)which could be set by an application running on MAC OS X.Is there anyway we could acheive a granularity in milliseconds? Thanks in advance. Regards, johnsy _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: cluster_read()
by Quinn 02 Dec '03

02 Dec '03

At 21:57 +1000 2/12/03, Chris Bergmann wrote: Or if you still think that filters are the way to go then by all means say so and i'll go play with them instead :-) Definitely not, given that you're already knee deep in BSD code already. The cluster_xxx calls are the fast path interface to Mac OS X's unified buffer cache (UBC). They allow a file system to easily support vop_read/write/pagein/pageout for data that should be cached (typically the contents of files). These routines eventually call back into the file system's vop_cmap routine to map file offsets to disk offsets, and vop_strategy routine to actually kick off the I/O. A file system can also do I/O via the more common buffer I/O routines (bread/brelse/bwrite). [You can learn about these in the "The Design and Implementation of the 4.4 BSD Operating System" book, which is required reading for anyone working on the BSD portitions of our kernel.] AFAIK, a UBC-aware file system only ever uses meta_bread, which is used to read/write meta data (that is, catalogue data, as opposed to file data). The buffer I/O routines also eventually do the I/O via the file system's vop_strategy routine. Any snapshot implementation will need to take both data and meta data writes into account. It might be education for you to get the 10.2 xnu/bsd source and compare it to the 10.2.3 xnu/bsd source. 10.2.3 contains our first cut at journalling. While our journalling implementation only journals meta data, it would be very educational to look at the changes we made to support it. [I must do this myself one day. Would that I had unbounded amounts of time available.] S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: cluster_read()
by Chris Bergmann 02 Dec '03

02 Dec '03

I'm looking into the possibility of snapshots on a hfs+ volume. I've got a duplicate HFS+ filesystem loaded as a kext, now i need to intercept writes to the filesystem. while i guess i could use a filter scheme, i'm going to have to hook a number of other places in the hfs+ source anyway, so if possible i'd rather keep the changes to a single kext for now, and intercept the writes prior to them leaving the hfs+ source (ie. write a wrapper cluster_write function which does its own stuff and then calls through to the real thing.) cluster_write(vp, uio, oldEOF, newEOF, headOff, tailOff, devblocksize, flags) okay, as far as i see it (i'm still not used to reading kernel source ;-) what we're doing here is writing some data [uio] to some offset [uio->uio_offset] and possibly padding before and after [headOff], [tailOff]. I'm not sure why it needs the vnode pointer at this level, i'm assuming this has something to do with the UBC, but since i've yet to see any decent docs on that either... ;-) If you've got any further info on the parameters to cluster_write() then i'd appreciate a quick summary. Otherwise i guess i'll have to keep going the hard way. Or if you still think that filters are the way to go then by all means say so and i'll go play with them instead :-) thx, chris On 02/12/2003, at 20:35, Quinn wrote: > At 20:20 +1000 2/12/03, Chris Bergmann wrote: >> Anyone know where i could find docs/details on these: >> >> cluster_read() >> cluster_write() > > They're currently not documented. The only details are in the source. > >> (I'm trying to intercept the lowest-level calls from the hfs+ source >> to the underlying partition- are these the right functions to be >> looking at? this is for my own purposes with a custom-built hfs+ >> kext, not for shipping code...) > > Hmmm, are you looking for the pattern of block reads/writes? If so, > you may be better off working at the IOMedia level. This is discussed > in the "Developing a Filter Scheme" section of "Inside Mac OS X: > Writing Drivers for Mass Storage Devices". > > <http://developer.apple.com/documentation/DeviceDrivers/Conceptual/ > MassStorage/07_Media_Example/chapter_7_section_1.html> > > S+E > -- > Quinn "The Eskimo!" > <http://www.apple.com/developer/> > Apple Developer Technical Support * Networking, Communications, > Hardware > _______________________________________________ > darwin-kernel mailing list | darwin-kernel(a)lists.apple.com > Help/Unsubscribe/Archives: > http://www.lists.apple.com/mailman/listinfo/darwin-kernel > Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: virtual memory
by Steve Dekorte 02 Dec '03

02 Dec '03

On Dec 2, 2003, at 1:53 AM, Quinn wrote: Within any normal user task, Mac OS X supports 4 GB of VM. By default, chunks of this address space will be allocated to other things (like the frameworks), so the limit to how much VM you can /allocate/ is less. Thanks Quinn. That's the info I was looking for. -- Steve _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: cluster_read()
by Quinn 02 Dec '03

02 Dec '03

At 20:20 +1000 2/12/03, Chris Bergmann wrote: Anyone know where i could find docs/details on these: cluster_read() cluster_write() They're currently not documented. The only details are in the source. (I'm trying to intercept the lowest-level calls from the hfs+ source to the underlying partition- are these the right functions to be looking at? this is for my own purposes with a custom-built hfs+ kext, not for shipping code...) Hmmm, are you looking for the pattern of block reads/writes? If so, you may be better off working at the IOMedia level. This is discussed in the "Developing a Filter Scheme" section of "Inside Mac OS X: Writing Drivers for Mass Storage Devices". <http://developer.apple.com/documentation/DeviceDrivers/Conceptual/MassStora…> S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

cluster_read()
by Chris Bergmann 02 Dec '03

02 Dec '03

Anyone know where i could find docs/details on these: cluster_read() cluster_write() (I'm trying to intercept the lowest-level calls from the hfs+ source to the underlying partition- are these the right functions to be looking at? this is for my own purposes with a custom-built hfs+ kext, not for shipping code...) thx, chris _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Patching sysent
by Quinn 02 Dec '03

02 Dec '03

Greetings I'm trying to find out why developers are patching the kernel system call dispatch table (sysent). If your product patches sysent, can you please drop me a line explaining what your product does and why it patches sysent. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0