Re: High bandwidth disk management techniques
Re: High bandwidth disk management techniques
- Subject: Re: High bandwidth disk management techniques
- From: kelly jacklin <email@hidden>
- Date: Mon, 2 May 2005 22:33:14 -0700
I think you're absolutely right, which is basically the point.
Spawning multiple threads to do multiple simultaneous reads to the
same disk will result in disk scheduling thrash, as there really is
no way to optimally schedule to a disk without specific knowledge of
disk geometry and onboard caching models, which is not available with
current drives nor the ATA spec, as far as I know.
My point was that you _do_ get a definite win (as we saw in both
CpMac and the Finder) if you can identify which files reside on
unique _physical_ devices (which is hard to do in the face of
pathological situations like disk images, etc., but is completely
doable for typical scenarios), and have a separate thread for each
device you need to hit. This is probably intuitive, as multiple I/Os
can obviously truly be in flight simultaneously for such devices, and
so they can complete in parallel, so having the requests be
simultaneously processed by the I/O system can be desirable and
effective.
And yes, that is the whole reason NCQ was invented (or resurrected
from similar functionality in SCSI-2), but I am not familiar with how
apps would access NCQ; so I'll defer to someone with more hardware
knowledge to elucidate this point. Sounds promising, though...
kelly
On May 2, 2005, at 6:46 AM, Wolfgang Schneider wrote:
Hi Kelly,
does the OS'es disk scheduler have even a chance of optimizing disk
access patterns? I'm speaking
of minimizing the read / write heads' path in order to improve
access time. Since modern hard disks
show only a virtual disk geometry to the outside world, nobody can
know where a particular sector
actually is, resp. how far two physical sectors are away from each
other. That's why NCQ was invented,
no? The disk itself is the only one who knows about these things,
or am I getting this wrong some way?
best wishes,
Wolfgang
Am 02.05.2005 um 15:36 schrieb kelly jacklin:
This is not at all surprising, as each thread is issuing reads
which compete for the scheduling resources of the disk.
When working on filesystem performance issues, at one point we
experimented around with the CpMac tool to do various threaded
models. The one that we found worked best was to single-thread
all access to files on volumes corresponding to the same
_physical_ device, but to have separate threads for each physical
device we encountered.
This was some time ago, back in early 10.2 development, so things
might have changed, and there may be more smarts in the disk
scheduler nowadays, I do not know. If not, then you might take
this approach as well.
kelly
On May 2, 2005, at 5:47 AM, philippe wicker wrote:
Some times ago (OS X 10.2.x) I tried to bench the simultaneous
disk accesses to a number of files. For the little story, these
files were 25 MBytes segment of a MacOS X update. I did that
using two methods. The first one was to sequentially access a
64KByte chunk in each file. The second method was to trigger the
access to each file in a different thread (one thread dedicated
to a given number of files), this method is a variant of
multiple asynchronous reads. The results were really surprising
and unexpected (at least for me): the "threaded" access gave a
twice worse - yes worse - performance on the internal drive than
the "sequential" method, and an equivalent result on an external
FW400 drive.
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
40nexoft.de
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
40apple.com
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden