Re: Speed-optimized file copy
Re: Speed-optimized file copy
- Subject: Re: Speed-optimized file copy
- From: Stevo Brock <email@hidden>
- Date: Tue, 19 Jul 2011 14:32:27 -0700
So this certainly begs the question:
On OS X 10.6 (and let's throw in 10.7), if the kernel sees a number of read() and write() calls coming in from different threads, does it automatically reorder them in any way, or does it always service them in order?
Mark Day had said this a few weeks ago about another file system reading question:
Are you giving the file system and the disks enough to allow them to schedule the disk heads? Consider using async dispatch blocks ("man dispatch") or multiple threads to read from several files at once, which gives the disks a better chance at arranging the I/Os in order (to help with seek delays). Overlapping I/O and computation can help you get better throughput from the drive, and may help reduce the affects of rotational delays.
I was working with the understanding that sprinkling the read() and write() calls out over a number of threads, interleaved with processing, was enough to allow the system to optimize those calls. So, would this be accurate?
-Stevo
On Jul 19, 2011, at 10:38 AM, Wim Lewis wrote:
>
> On 18 Jul 2011, at 1:25 PM, Stevo Brock wrote:
>> The idea is that with 4 concurrent operations executing, the OS is getting fed a lot of disk operation requests such that it can reorder the requests to maximize file system efficiency. The deployment is OS X 10.6 minimum so we can use any tech available.
>>
>> What are ways that we could improve the efficiency of this system?
>
> I haven't tested on Darwin, but on some systems you can get a significant efficiency improvement using the aio_* interface (POSIX asynchronous IO). Not only does this let the kernel see multiple outstanding requests and retire them out-of-order, but the particular semantics of aio require less work from the VM system to guarantee correctness. Darwin's different, presumably, but it might be worth looking into.
>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Filesystem-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden