Re: Speed-optimized file copy
Re: Speed-optimized file copy
- Subject: Re: Speed-optimized file copy
- From: "Quinn \"The Eskimo!\"" <email@hidden>
- Date: Tue, 19 Jul 2011 09:08:09 +0100
On 18 Jul 2011, at 21:25, Stevo Brock wrote:
> What are ways that we could improve the efficiency of this system?
It's hard to optimise stuff like this without actually looking at the code in action. The issue is that there can non-obvious interactions between the components that you'll only discover by profiling. However, my standard hints for this are:
o use page aligned buffers (I'd recommend <x-man-page://3/valloc> in preference to vm_allocate, btw)
o use read offsets and sizes that are a multiple of the page size
o disable caching on both source and destination
o optimise for seeks
The last part is the tricky one. You typically want to have one 'thread' of execution per spindle. For example, if you're copying between two spindles, you want one producer 'thread' for the source spindle and one consumer 'thread' for the destination. In contrast, if the source and destination are on the same drive, you want to alternate between producing and consuming, with a large buffer in between. The tricky parts of this are:
o what's a spindle -- It's very hard to determine this in the general case.
o large buffer -- You want the buffer to be large enough to amortise the seek cost between producing and consuming, but not too big as to start the system thrashing.
S+E
--
Quinn "The Eskimo!" <http://www.apple.com/developer/>
Apple Developer Relations, Developer Technical Support, Core OS/Hardware
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden