Re: How to read files from disk directly?
Re: How to read files from disk directly?
- Subject: Re: How to read files from disk directly?
- From: Ken Hornstein <email@hidden>
- Date: Tue, 05 Jul 2011 09:25:23 -0400
>What I know is that I can read a single 4gb file in about 3 minutes. It
>takes around 20-25 minutes to read 4gb spread among 190,000 files.
It occurs to me that you could write a simple dtrace program to determine
what system calls you're stuck in (or even what kernel functions you're in
the middle of). Might help get a handle on the problem (are you spending
your time in read() system calls, or traversing the filesystem hierarchy?)
It also occurs to me that you could maybe do some (hopefully simpler)
hybrid approaches:
- Read the catalog from the raw disk to get the filesystem hierarchy,
sort based on extent number, then read the files using the standard
APIs() in that order. Might reduce seek times.
- Do the above, but use whatever interface ends up opening files in the
/.vol filesystem to avoid potential overhead of the open() system call
(but don't ask me which interface that is, because I don't know; maybe
one of the Carbon APIs? Not sure that's even possible without some magic).
--Ken
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden