• Open Menu Close Menu
  • Apple
  • Shopping Bag
  • Apple
  • Mac
  • iPad
  • iPhone
  • Watch
  • TV
  • Music
  • Support
  • Search apple.com
  • Shopping Bag

Lists

Open Menu Close Menu
  • Terms and Conditions
  • Lists hosted on this site
  • Email the Postmaster
  • Tips for posting to public mailing lists
Re: How to read files from disk directly?
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: How to read files from disk directly?


  • Subject: Re: How to read files from disk directly?
  • From: Shantonu Sen <email@hidden>
  • Date: Tue, 5 Jul 2011 08:51:15 -0700

Reading the catalog would require unmounting the filesystem, with all the caveats that have previously been mentioned. The only savings would be that you wouldn't have to call fcntl(F_LOG2PHYS) on every allocation block in the file. But the tradeoff is that you would need to read the extents overflow file and stitch together extents for fragmented files.

/.vol has more overhead than full paths. The kernel converts file ID lookups to their full path so that permissions/ACL for each path component can be applied.

Shantonu

On Jul 5, 2011, at 6:25 AM, Ken Hornstein wrote:

>> What I know is that I can read a single 4gb file in about 3 minutes. It
>> takes around 20-25 minutes to read 4gb spread among 190,000 files.
>
> It occurs to me that you could write a simple dtrace program to determine
> what system calls you're stuck in (or even what kernel functions you're in
> the middle of).  Might help get a handle on the problem (are you spending
> your time in read() system calls, or traversing the filesystem hierarchy?)
>
> It also occurs to me that you could maybe do some (hopefully simpler)
> hybrid approaches:
>
> - Read the catalog from the raw disk to get the filesystem hierarchy,
>  sort based on extent number, then read the files using the standard
>  APIs() in that order.  Might reduce seek times.
> - Do the above, but use whatever interface ends up opening files in the
>  /.vol filesystem to avoid potential overhead of the open() system call
>  (but don't ask me which interface that is, because I don't know; maybe
>  one of the Carbon APIs?  Not sure that's even possible without some magic).
>
> --Ken
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Filesystem-dev mailing list      (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden

 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list      (email@hidden)
Help/Unsubscribe/Update your Subscription:

This email sent to email@hidden

  • Follow-Ups:
    • Re: How to read files from disk directly?
      • From: Ken Hornstein <email@hidden>
References: 
 >Re: How to read files from disk directly? (From: Ken Hornstein <email@hidden>)

  • Prev by Date: Re: How to read files from disk directly?
  • Next by Date: Re: How to read files from disk directly?
  • Previous by thread: Re: How to read files from disk directly?
  • Next by thread: Re: How to read files from disk directly?
  • Index(es):
    • Date
    • Thread