site_archiver@lists.apple.com Delivered-To: darwin-kernel@lists.apple.com I hadn't thought of that, and that's an excellent suggestion (thanks also extended to Brian and Matt). The source file data is only read once, so at least I can eliminate that from being cashed. Come to think of it, that might improve my overall performance since the reading the source file blocks also competes with the caching of my database file blocks. -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.a... On Jun 2, 2007, at 8:17 PM, Sam Vaughan wrote: On 01/06/2007, at 5:04 AM, James Bucanek <subscriber@gloaming.com>: There's an even better reason why F_NOCACHE will probably improve your code's performance. Provided that you do as Quinn suggests with respect to alignment, it will cause the I/O to go directly to/ from your user space buffer, removing the additional copy that happens in the buffered I/O case. This is why it's called "direct I/ O" on most other platforms, where the O_DIRECT flag is used at file open time to enable it. Direct I/O is not necessarily the best idea for anything for which you are not guaranteeing rate monotonic data flow. If you use mmap(), then the memory will not need obe paged out (if you read it using F_NOCACHE, then the pages become dirty pages in your process address space, and may be forced out to swap). If you sequentially access the pages after they are mmap()'ed, then you will trigger clustered I/O read-ahead, which will cause the pages to be faulted in, up to a 1M ubc descriptor at a time. By doing it this way, you will also signal that the pager can scavenge behind by having the pages flagged that way by the pager, which will put them first on the list to be shot, should you hit memory pressure. You can also issue aan fcntl(fd, F_RDADVISE, ...) to force a read ahead; if you then touch (read a byte in from) the last page in the range, the touch will block until the buffer is no longer busy, which means all your pages will be available. For a backup utility, you'd be much better off being an fsevents clients (search on <http://developer.apple.com>), and then reacting to events only for files which are actually changed since the last backup. This email sent to site_archiver@lists.apple.com