Apple

On Thu, Mar 5, 2015 at 5:51 PM, Hamilton Feltman <email@hidden> wrote:

On Mar 5, 2015, at 7:30 AM, Paul Davis <email@hidden> wrote:

Yes, the block size dependence is clear and shared across all platforms (though the shape of the curve is different depending on the platform and the filesystem).

So, with smaller and smaller block sizes, the seek time is creeping up a bit. However, it’s still well within spec of the drive. Also there might be some things unaccounted for in the simplistic calculation, such as adjacent track seeking. But that would only get you closer to the average seek time for all block sizes.

We don't expect small blocksizes to work. We included them in our test runs so that we'd get longer curves to plot that might reveal something non-obvious. We know we have to use a large enough blocksize to even have a hope of handling large track counts; we used to think that 256kB was the right size, but it seems apparent now that for Linux and Windows we should use 1MB and for OS X probably closer to 2MB ... except that this then makes locates (where we explicitly seek then refill a memory buffer because the transport position has changed) really slow without doing clever tricks that are not really all that clever or reliable (e.g. don't read 2MB per track before starting playback, just read (say) 64kB for each track, then go back and finish filling all the buffers - this just defers the point at which you find you don't have enough disk bandwidth).

How is it that the other OS’s can do it faster with caching disabled?

We don't disable caching on other platforms. And unless you modified run-readtest.sh, we aren't disabling it on OS X either in the test (that requires an extra -D flag to be passed to readtest).

I maintain that there must be some form of caching going on.

Would you mind posting the drive model, filesystem, and the output of the test, with the above printf included?

I'm in a 12V DC environment right now, so I can't run my Mac Mini without doing a bit of work, but I'll get that done tomorrow. The machine I have with me is a Mac Mini with Lion, but we've tested with Mavericks, Mountain Lion, Snow Leopard and even Tiger!

However, on OS X, even when you use "optimal" block sizes (which appear to be on the order of 2MB-4MB (roughly twice as large as is necessary on Windows or Linux to get the the disk streaming capacity), the system is *still* not capable of maintaining sustained bandwidth necessary for high track counts, because it drops *way* below that
bandwidth for too long to reasonably buffer. At least that's what we've seen from testing so far.

I think that’s a different problem then. Have you tried higher thread priorities? Why does it seem to work on mine? Or is this test just averaging through it? We could modify the test to print out a list of times and then graph that.

What we see is a big gap between minimum track count (based on the lowest sustained bandwidth) and the maximum track count (based on the highest observed sustained bandwidth). We have graphed several systems in various ways, you can see some of the results here:

http://gareus.org/wiki/hddspeed

There are some others that we've collected and plotted differently. The real issue is not the highest sustained bandwidth - we know we can hit that if we just use a large enough blocksize. The issue is whether we can expect anything even remotely close to that bandwidth to be maintained reliably ... so far the answer has been no, which seems incredibly odd. The behaviour in Ardour itself looks like this: create a new session, add 128 tracks, record 20 minutes of audio, locate back to zero and start playback. It works. For a while. Then a message shows up saying "disk system cannot keep up". Then you start digging ... and digging .. and digging ... and you find that for 8 seconds the disk was wasn't managing more than 5MB/sec. And eventually you end up with readtest.c :)

Thanks for trying the test! Yours are the first set of results we've seen that are close to what you'd expect, which is strange.

No problem, I'm working on some similar things. It’s just a run of the mill mac mini late 2012 model, with 10.10.2 on it.

BTW if anyone wants to try this without the glib dependency compile with:

yes, we were going to add that anyway. I'll merge that and the windows equivalent into the code in git.

References:
	>Re: large scale (audio) file I/O on OS X : help or insight requested (From: Hamilton Feltman <email@hidden>)
	>Re: large scale (audio) file I/O on OS X : help or insight requested (From: Paul Davis <email@hidden>)
	>Re: large scale (audio) file I/O on OS X : help or insight requested (From: Hamilton Feltman <email@hidden>)