• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: large scale (audio) file I/O on OS X : help or insight requested
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: large scale (audio) file I/O on OS X : help or insight requested


  • Subject: Re: large scale (audio) file I/O on OS X : help or insight requested
  • From: Hamilton Feltman <email@hidden>
  • Date: Thu, 05 Mar 2015 15:51:21 -0800

On Mar 5, 2015, at 7:30 AM, Paul Davis <email@hidden> wrote:

Yes, the block size dependence is clear and shared across all platforms (though the shape of the curve is different depending on the platform and the filesystem).


I modified the test to print out the total time and number of seeks:

printf ("seeks: %llu: bytes: %llu total_time: %f\n", cnt * nfiles, (nfiles * _read), total_time/1000000.0);

WD Red on USB 3
122.768 MB/sec read (128731578 bytes/sec)

256 files 10MB each -> 2684354560 total bytes to read
Time required to stream 2684354560 bytes -> 20.85 seconds

Blocksize 2621440
Number of seeks 1024
Time taken: 27.16 seconds
Time available for seeks: 27.16 - 20.85 = 6.31 seconds
6.31 / 1024 -> 6.2 ms seek time

Blocksize 65536
Number of seeks 40960
Time taken: 365.88 seconds
Time available for seeks: 365.88 - 20.85 = 345.03 seconds
345.03 / 40960 -> 8.42 ms seek time

Blocksize 32768
Number of seeks 81920
Time taken: 928.40 seconds
Time available for seeks: 928.40 - 20.85 = 907.55 seconds
907.55 / 81920 -> 11.078 ms seek time

So, with smaller and smaller block sizes, the seek time is creeping up a bit. However, it’s still well within spec of the drive. Also there might be some things unaccounted for in the simplistic calculation, such as adjacent track seeking. But that would only get you closer to the average seek time for all block sizes.

How is it that the other OS’s can do it faster with caching disabled? I maintain that there must be some form of caching going on.

Would you mind posting the drive model, filesystem, and the output of the test, with the above printf included?


However, on OS X, even when you use "optimal" block sizes (which appear to be on the order of 2MB-4MB (roughly twice as large as is necessary on Windows or Linux to get the the disk streaming capacity), the system is *still* not capable of maintaining sustained bandwidth necessary for high track counts, because it drops *way* below that 
bandwidth for too long to reasonably buffer. At least that's what we've seen from testing so far.

I think that’s a different problem then. Have you tried higher thread priorities? Why does it seem to work on mine? Or is this test just averaging through it? We could modify the test to print out a list of times and then graph that.


Thanks for trying the test! Yours are the first set of results we've seen that are close to what you'd expect, which is strange. 

No problem, I'm working on some similar things. It’s just a run of the mill mac mini late 2012 model, with 10.10.2 on it.

BTW if anyone wants to try this without the glib dependency compile with:
gcc -o readtest readtest.c

but first replace this:
#include <glib.h>

with this:
#if __APPLE__
#import <mach/mach_time.h> // for mach time
#import <sys/syslimits.h> // for PATH_MAX
typedef int64_t gint64;
// From gmain.c
gint64 g_get_monotonic_time (void)
{
static mach_timebase_info_data_t timebase_info;


if (timebase_info.denom == 0)
{
/* This is a fraction that we must use to scale
* mach_absolute_time() by in order to reach nanoseconds.
*
* We've only ever observed this to be 1/1, but maybe it could be
* 1000/1 if mach time is microseconds already, or 1/1000 if
* picoseconds.  Try to deal nicely with that.
*/
mach_timebase_info (&timebase_info);


/* We actually want microseconds... */
if (timebase_info.numer % 1000 == 0)
timebase_info.numer /= 1000;
else
timebase_info.denom *= 1000;


/* We want to make the numer 1 to avoid having to multiply... */
if (timebase_info.denom % timebase_info.numer == 0)
{
timebase_info.denom /= timebase_info.numer;
timebase_info.numer = 1;
}
else
{
/* We could just multiply by timebase_info.numer below, but why
* bother for a case that may never actually exist...
*
* Plus -- performing the multiplication would risk integer
* overflow.  If we ever actually end up in this situation, we
* should more carefully evaluate the correct course of action.
*/
mach_timebase_info (&timebase_info); /* Get a fresh copy for a better message */
fprintf (stderr, "Got weird mach timebase info of %d/%d.  Please file a bug against GLib.",
timebase_info.numer, timebase_info.denom);
}
}


return mach_absolute_time () / timebase_info.denom;
}
#else
#include <glib.h>
#endif


 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

  • Follow-Ups:
    • Re: large scale (audio) file I/O on OS X : help or insight requested
      • From: Paul Davis <email@hidden>
References: 
 >Re: large scale (audio) file I/O on OS X : help or insight requested (From: Hamilton Feltman <email@hidden>)
 >Re: large scale (audio) file I/O on OS X : help or insight requested (From: Paul Davis <email@hidden>)

  • Prev by Date: Re: Undeclared identifier kAudioUnitProperty_NickName
  • Next by Date: Re: large scale (audio) file I/O on OS X : help or insight requested
  • Previous by thread: Re: large scale (audio) file I/O on OS X : help or insight requested
  • Next by thread: Re: large scale (audio) file I/O on OS X : help or insight requested
  • Index(es):
    • Date
    • Thread