Re: preferred disk api for high bandwidth sample reading
Re: preferred disk api for high bandwidth sample reading
- Subject: Re: preferred disk api for high bandwidth sample reading
- From: Paul Davis <email@hidden>
- Date: Tue, 5 Jan 2010 13:27:53 -0500
On Tue, Jan 5, 2010 at 1:38 AM, Ross Bencina <email@hidden> wrote:
> Paul Davis wrote:
>>>
>>> On Windows you can set flags to disable OS level caching/buffering when
>>> you
>>> open a file handle (with the restriction of being required to perform IO
>>> in
>>> page aligned blocks).
>>
>> the same thing is possible with unix/posix APIs too.
>
> Can you give some links/keywords please? What do you currently use in
> Ardour? read/write one buffer at a time using posix file io?
O_DIRECT is the open() flag used by several unix systems to avoid the
buffer cache. It is not part of POSIX, and not supported on OS X
AFAIK. Linus has never been a big fan of O_DIRECT but has grudgingly
allowed it to be supported.
this provides some info on how to accomplish a similar effect on OS X:
http://tinyco.de/2008/03/04/hacking-the-unified-buffer-cache.html
i believe that its slightly inaccurate in the way that it describes
the "persistence" of the cache setting, as described here:
http://lists.apple.com/archives/darwin-dev/2009/Oct/msg00165.html
the other thing that can make a huge difference is avoiding meta-data
updates on disk write, since this can cause seeking that would other
be unnecessary. unfortunately, an application can't control this - its
a function of the filesystem mount options. This is usually
accomplished with the noatime mount option, but as noted here:
http://beyondabstraction.net/2008/11/06/noatime-mount-option-in-os-x/
it can be a bit tricky to impose it for all filesystems on an OS X box
because of the automounter (the article does have tips on how to do
that, however). you really do not need access time updates on A/V data
files, and on some disks/filesystems, avoiding this can be a bigger
win than cache design etc.
ardour uses pread/pwrite to drop 256kB chunks onto disk and attempts
to use the same file ordering for read+write, thus attempting to
reduce seeking (i.e. we write file 1,2,3,4 and read file 1,2,3,4 with
the vague hope that the data is on disk as 1,2,3,4) this is all done
from a thread that uses a modified round-robin approach to try to
avoid starvation (this is less necessary than it used to be - ardour
used to support per-track varispeed, meaning that one track could
consume vastly more disk data than another for the same period of
audio time).
i have *absolutely* no doubt that our disk i/o, buffering and caching
scheme could be hugely improved, but on linux, at least, it hasn't
really shown itself to be much of an issue to date, and we haven't
noticed any substantially different behaviour on OS X either.
> Last time I benchmarked this on Windows (admittedly on older hardware,
> perhaps in the 1Ghz era) I saved about 10% CPU usage (from memory with
> 50-100 streams) by disabling the OS level i/o caching. It doesn't make much
> sense to incurr OS-level buffer copying and caching when a
> streaming-sepecific application level buffering/caching layer is needed
> anyway.
depends on the use case. i agree the it would be great to avoid, but
i've never seen anything remotely close to 10% CPU usage from
streaming 50-100 tracks on any unix-ish OS.
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden