Re: About efficiently reading and writing files...
Re: About efficiently reading and writing files...
- Subject: Re: About efficiently reading and writing files...
- From: "Erik M. Buck" <email@hidden>
- Date: Mon, 13 Dec 2004 18:58:59 -0500
I find mmap() to be the fastest I/O path for block devices on almost every
Unix like system. This is true for both serial sequential access and random
access.
Here is information for Mac OS X: http://www.osxfaq.com/man/2/mmap.ws
I have not specifically benchmarked on Mac OS X.
The theory behind mmap() is that it avoids a buffer copy within the OS.
Normally, I/O is handled by device drivers which read/write OS file buffers.
User programs then read/write the file buffers to _copy_ to or from user
address space.
mmap() uses the operating system's virtual memory subsystem which is based
on memory management hardware. Pages of data are paged in and out from a
block device (disk, Ethernet, whatever) on demand automatically just like
the system automatically handles the swap file to page other types of
application data in and out. Data that is paged in is assigned to the user
address space in the hardware page table which is part of the memory
management hardware. Similarly, when an application writes to mmaped
memory, the page is marked as "dirty" and asynchronously written back to the
block device.
In my experience, mmapped files can be accessed sequentially at RAM speeds
because the system automatically pages in "the next few" pages of data
before I need it. This is generally an order of magnitute faster than any
other IO technique as long as you don't need to explicitly wait for data
synchronization to permanent media like you might have to do for a shared
database.
Note: Cocoa classes like NSData and NSString may use mmap() under the
surface for large data sets.
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Cocoa-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden