Re: High bandwidth disk management techniques
Re: High bandwidth disk management techniques
- Subject: Re: High bandwidth disk management techniques
- From: Herbie Robinson <email@hidden>
- Date: Mon, 2 May 2005 04:21:52 -0400
Folks.
I wonder if anyone can offer their experience in efficient
techniques for very disk intensive real time operations.
Our application reads audio from 64 or more files in real time, and
requires consistent performance. The total data throughput is only
about 10 megabytes per second, but unlike a single video file, using
64 audio files has its own seriously taxing load on the disk.
Audio is completely different than video. Video is usually limited
by the data transfer rate (i.e., how fast data actually transfers
to/from the disk after the heads have been positioned). Audio
performance is mostly limited by the disk access time: Getting 64
tracks independent audio streams going on a single disk is very
unlikely to succeed. When we are doing live recordings, we try not
to do more then 12-16 tracks per spindle.
We have noticed that OSX seems to maintain a HUGE disk cache (as
much as 250 MB) which sits between our calls to disk and the actual
disk access (we call once per second, but the disk is only hit
around once every 5 seconds).
Many Unix implementations will bypass the cache if you do reads and
writes in multiples of the page size.
There are various aspects to the challenge - access time,
fragmentation, cache management and overall throughput. I wondered
if anyone has any experience on techniques to optimize this type of
disk access.
Currently our subsystem maintains a 1 second buffer for each audio
file, so we read about 200k once per second per file. This works OK
with lower track counts, but as we approach 64, we start to see
things stalling. We have tested with an internal SATA, and external
SCSI drive, and an XServe RAID. The SCSI drive seems to offer the
best performance, although it is still failing to deliver using our
current data handling techniques.
You are doing about as well as Pro Tools does. You can increase the
track count by increasing the buffer size, but then you also increase
the playback latency (which can be annoying).
I was surprised to find that the RAID didn't help, even with 1GB of
cache. I think the problems is one of startup - getting the
'inertia going' with the cache. If we let playback struggle along,
eventually it does sort itself out on all the drives tested. The
problem we have is getting the throughput established right from the
start of playback.
We do prepare the first second of audio in the buffers before we
start playing, but this doesn't seem to be enough to kick all the
caching into top gear.
I know this is a very specialised and complex question, but I
wondered if anyone who has experience with real time playback of
large numbers of disk streams could offer any general advice.
AFIK, There is no substitute for having more than one spindle.
Allowing the user to tune the buffers will help (the user can make
the latency vs. track count tradeoff to their own tolerance levels).
thanks
Mark Gilbert.
--
email@hidden
Tel: +44 208 340 5677
fax: +44 870 055 7790
http://www.gallery.co.uk
New Product ! - Zen MetaCorder
Location Sound Field Recorder
http://www.metacorder.info
--
-*****************************************
** http://www.curbside-recording.com/ **
******************************************
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden