Re: Disk streaming library
Re: Disk streaming library
- Subject: Re: Disk streaming library
- From: Paul Davis <email@hidden>
- Date: Fri, 29 Jun 2012 09:12:26 -0400
On Fri, Jun 29, 2012 at 5:04 AM, Philippe Wicker
<email@hidden> wrote:
Yes. We already reached the conclusion that we'll have to cache data around the loop locators. The "keep the first few seconds in RAM" thing is - to my knowledge - the only way to guarantee a low latency (e.g. when a note on is played).
I should probably stress that in addition to relying on the OS for low level disk i/o strategy, Ardour also buffers a big chunk of data for both capture and playback (default: 5 seconds worth). This is necessary to deal with stalls in disk i/o that, while rare, do seem to happen on different platforms. Linux actually seems to be the worst, though admittedly the last time I checked it was many years ago (at that time, disk i/o could stall out for *seconds*). The basic strategy is very simple: a single-reader, single-writer FIFO between the audio callback and disk i/o, with both audio callback and disk i/o occuring in separate threads. We tried using multiple threads for disk i/o a few years back and on *nix like systems we didn't find that it really helped. Instead, the disk i/o thread does its best to ensure reasonably equitable round-robin serving of all the FIFOs by reading min(BLOCKSIZE,FIFO-SPACE) rather than FIFO-SPACE which would allow starvation. In Ardour 3.0, the size of these FIFOs is dynamically variable by the user (it matters more in Ardour which is often being used exclusively for HDR, in which case using huge amounts of RAM to buffer against any chance of a dropout in the to-disk pipeline is entirely reasonable).
> One thing you *do* want is to have as many io-ops in flight at once. Afaik the interfaces for doing this are completely different accross Linux, MacOS and Windows.
Yes, although we also found that the improvements from doing this were nowhere near as large as some of the papers/docs on this had lead us to suspect. It also means using asynchronous I/O APIs which are more complex and not anywhere near as well tested and evaluated as their older cousins. As a result, Ardour specifically does *not* do this, and we still get pretty excellent i/o performance, even on windows (where we are using POSIX API calls, not windows).
>> well, good luck with that. Oracle certainly pulls this off with their
>> databases, but there have been many, many studies where people have
>> tried to do better than the buffer cache and discovered that in real
>> world scenarios, they can't.
>
> Links please?
Ross, I wish I could help, but its been a few years since I was working in a CS department and reading this stuff all the time. I could be wrong now.
Hmmm. The need of data on all the streamed samples is uncorrelated between the samples. I mean that you need to start streaming a sample (or a group of samples) when a note ON (or a similar event) is received. From the plug-in point of view these events are random, unpredictable. Also the rate at which a sample has to be read depends on a lot of parameters (the note value, the pitch modulation…). Disk read commands have their origin in the audio callback because this is where events are processed.
You're not really discussing a streaming engine at all then. You just need a very smart disk caching mechanism, which is related, but different.
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden