• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: Speed-optimized file copy
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Speed-optimized file copy


  • Subject: Re: Speed-optimized file copy
  • From: Stevo Brock <email@hidden>
  • Date: Tue, 19 Jul 2011 14:32:27 -0700

So this certainly begs the question:

On OS X 10.6 (and let's throw in 10.7), if the kernel sees a number of read() and write() calls coming in from different threads, does it automatically reorder them in any way, or does it always service them in order?

Mark Day had said this a few weeks ago about another file system reading question:

Are you giving the file system and the disks enough to allow them to schedule the disk heads?  Consider using async dispatch blocks ("man dispatch") or multiple threads to read from several files at once, which gives the disks a better chance at arranging the I/Os in order (to help with seek delays).  Overlapping I/O and computation can help you get better throughput from the drive, and may help reduce the affects of rotational delays.

I was working with the understanding that sprinkling the read() and write() calls out over a number of threads, interleaved with processing, was enough to allow the system to optimize those calls.  So, would this be accurate?

-Stevo


On Jul 19, 2011, at 10:38 AM, Wim Lewis wrote:

>
> On 18 Jul 2011, at 1:25 PM, Stevo Brock wrote:
>> The idea is that with 4 concurrent operations executing, the OS is getting fed a lot of disk operation requests such that it can reorder the requests to maximize file system efficiency.  The deployment is OS X 10.6 minimum so we can use any tech available.
>>
>> What are ways that we could improve the efficiency of this system?
>
> I haven't tested on Darwin, but on some systems you can get a significant efficiency improvement using the aio_* interface (POSIX asynchronous IO). Not only does this let the kernel see multiple outstanding requests and retire them out-of-order, but the particular semantics of aio require less work from the VM system to guarantee correctness. Darwin's different, presumably, but it might be worth looking into.
>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Filesystem-dev mailing list      (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden

 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

References: 
 >Speed-optimized file copy (From: Stevo Brock <email@hidden>)
 >Re: Speed-optimized file copy (From: Wim Lewis <email@hidden>)

  • Prev by Date: Re: Speed-optimized file copy
  • Next by Date: Re: Speed-optimized file copy
  • Previous by thread: Re: Speed-optimized file copy
  • Next by thread: Re: Speed-optimized file copy
  • Index(es):
    • Date
    • Thread