Re: large scale (audio) file I/O on OS X : help or insight requested

Subject: Re: large scale (audio) file I/O on OS X : help or insight requested
From: Paul Davis <email@hidden>
Date: Wed, 11 Mar 2015 20:53:49 -0500

robin reports back that your dd-tester gives:

41.1806 MB/sec write
42.6064 MB/sec read

(Western Digital 6400AAV - we can't find specs online)

He also tried it formatted as FAT32, with essentially no change in the results.

On Wed, Mar 11, 2015 at 8:36 PM, Hamilton Feltman <email@hidden> wrote:

Hi Paul,

It’s an external StarTech raid enclosure with Mac OS Extended Journaled. Did I say raid 0? It’s not raid 0, it’s mirrored (raid 1), so the performance should be like a single drive. It’s the only spinning disk I tested.

What is the speed of the drive you tested? This script will give an accurate figure:

#!/bin/sh
dd if=/dev/zero bs=1024k of=_tstfile count=1024 2>&1 | awk '/sec/ {print $1 / $5 / 1048576, "MB/sec write" }'
sudo purge
dd if=_tstfile bs=1024k of=/dev/null count=1024 2>&1 | awk '/sec/ {print $1 / $5 / 1048576, "MB/sec read" }'
rm _tstfile

It would be nice if you included the printf in readtest.c:
printf ("seeks: %llu: bytes: %llu total_time: %f\n", cnt * nfiles, (nfiles * _read), total_time/1000000.0);

so we can pinpoint where the time is being spent.

Regards,
Hamilton

On Mar 11, 2015, at 9:11 AM, Paul Davis <email@hidden> wrote:

We just got around to trying with both F_RDAHEAD and F_NOCACHE. They seem to make no difference at all on the one system we tested. http://pastebin.com/ZKjVA94t

I also just noted that your successful test was on "External 3GB WD Reds on USB 3 in raid 0 " - what filesystem was on the drives? Do I assume you have no spinner that isn't part of a RAID?

On Thu, Mar 5, 2015 at 8:48 PM, Hamilton Feltman <email@hidden> wrote:

We don't disable caching on other platforms. And unless you modified run-readtest.sh, we aren't disabling it on OS X either in the test (that requires an extra -D flag to be passed to readtest).

Caching was disabled, but I had tried a lot of different tests, one of the was to remove lookahead reads and inadvertently it’s still in there. So this explains the more ‘deterministic’ results. But I think you’re going to need this also, since what you’re doing is writing your own file cache, and you don’t want the filesystem to think it knows better.

if (fcntl(fd, F_RDAHEAD, 0) == -1) {
fprintf (stderr, "Cannot set F_RDAHEAD on file #%d\n", n);
}

Then a message shows up saying "disk system cannot keep up". Then you start digging ... and digging .. and digging ... and you find that for 8 seconds the disk was wasn't managing more than 5MB/sec. And eventually you end up with readtest.c :)

Ok, that clarifies the issue drastically. One thing to check is possibly something else hitting the disk, like spotlight. Anything else touching the disk at that seekrate, and the bandwidth is destroyed. You can disable spotlight for that drive (System preferences -> Spotlight -> Privacy). If it’s not spotlight, a utility called fseventer will show system wide file access, but it doesn’t look like mavericks is supported yet. But there has got to be a command line util that can watch fsevent.

Something to try anyway.

 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

Follow-Ups:
- Re: large scale (audio) file I/O on OS X : help or insight requested
  - From: Hamilton Feltman <email@hidden>

References:
	>Re: large scale (audio) file I/O on OS X : help or insight requested (From: Hamilton Feltman <email@hidden>)
	>Re: large scale (audio) file I/O on OS X : help or insight requested (From: Paul Davis <email@hidden>)
	>Re: large scale (audio) file I/O on OS X : help or insight requested (From: Hamilton Feltman <email@hidden>)
	>Re: large scale (audio) file I/O on OS X : help or insight requested (From: Paul Davis <email@hidden>)
	>Re: large scale (audio) file I/O on OS X : help or insight requested (From: Hamilton Feltman <email@hidden>)
	>Re: large scale (audio) file I/O on OS X : help or insight requested (From: Paul Davis <email@hidden>)
	>Re: large scale (audio) file I/O on OS X : help or insight requested (From: Hamilton Feltman <email@hidden>)

Prev by Date: Re: large scale (audio) file I/O on OS X : help or insight requested
Next by Date: Re: large scale (audio) file I/O on OS X : help or insight requested
Previous by thread: Re: large scale (audio) file I/O on OS X : help or insight requested
Next by thread: Re: large scale (audio) file I/O on OS X : help or insight requested
Index(es):
- Date
- Thread