Re: How to read files from disk directly?
Re: How to read files from disk directly?
- Subject: Re: How to read files from disk directly?
- From: Shantonu Sen <email@hidden>
- Date: Tue, 5 Jul 2011 08:51:15 -0700
Reading the catalog would require unmounting the filesystem, with all the caveats that have previously been mentioned. The only savings would be that you wouldn't have to call fcntl(F_LOG2PHYS) on every allocation block in the file. But the tradeoff is that you would need to read the extents overflow file and stitch together extents for fragmented files.
/.vol has more overhead than full paths. The kernel converts file ID lookups to their full path so that permissions/ACL for each path component can be applied.
Shantonu
On Jul 5, 2011, at 6:25 AM, Ken Hornstein wrote:
>> What I know is that I can read a single 4gb file in about 3 minutes. It
>> takes around 20-25 minutes to read 4gb spread among 190,000 files.
>
> It occurs to me that you could write a simple dtrace program to determine
> what system calls you're stuck in (or even what kernel functions you're in
> the middle of). Might help get a handle on the problem (are you spending
> your time in read() system calls, or traversing the filesystem hierarchy?)
>
> It also occurs to me that you could maybe do some (hopefully simpler)
> hybrid approaches:
>
> - Read the catalog from the raw disk to get the filesystem hierarchy,
> sort based on extent number, then read the files using the standard
> APIs() in that order. Might reduce seek times.
> - Do the above, but use whatever interface ends up opening files in the
> /.vol filesystem to avoid potential overhead of the open() system call
> (but don't ask me which interface that is, because I don't know; maybe
> one of the Carbon APIs? Not sure that's even possible without some magic).
>
> --Ken
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Filesystem-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden