Re: How to read files from disk directly?
Re: How to read files from disk directly?
- Subject: Re: How to read files from disk directly?
- From: Eric Gorr <email@hidden>
- Date: Sun, 03 Jul 2011 22:16:08 -0400
On Jul 3, 2011, at 9:27 PM, Ken Hornstein wrote:
>> So, the primary question I have is when reading the data from my
>> device directly, how can I determine exactly what data belongs with
>> what file?
>
> I think most people would tell you that this is a terrible idea and to
> forget it. However, I'm not most people :-)
Good :-)
I like it a lot better when people actually answer the question I ask, so thank you.
Performance is very important in my case. I don't have the numbers in front of me, but there is simply no comparison between the speed in which I can read a single 4gb file and 190,000 small files that add up to 4gb.
> I will say, upfront, that I don't think it's a great idea. You're setting
> yourself up for a lot of complexity and honestly I don't think you'll really
> gain a whole lot in terms of performance.
Perhaps, but it would be interesting to find out.
> And I don't think Apple will
> make any guarantees that the filesystem format is stable, so you might
> find your application not working in the future.
But, if I am using public APIs to obtain the information, I would assume it would remain somewhat stable. I would assume that defragmentation applications have to do something similar to this and they seem to remain somewhat stable, but perhaps I am mistaken.
> But, hey, if you're REALLY determined to do this, here's what I can
> tell you:
Well, I am somewhat determined to find out whether or not I will see performance improvements.
> - I _was_ going to say that there isn't an userspace API which lets you turn
> a filename into a list of physical block offsets. But on further
> inspection I realize I was wrong, and there _is_ such an API.
> Specifically, fcntl() has a F_LOG2PHYS option. But I don't think that
> will help you all that much, because to use that you still have to open()
> a file to get a descriptor for that, and you'd have to do a crapload of
> system calls to get all of the physical block offsets and by that point
> you'd probably be better off just reading the damn file yourself.
Nice. I was not aware of the F_LOG2PHYS option of fcntl. What are the other system calls that would get me all of the physical block offsets? Or is that what you describe below?
> - If you want an example of something that does close to what you want,
> look at the source code of fsck_hfs (part of diskdev_cmds). Specifically
> check out what the -B option of fsck_hfs does. You're probably going
> to have incorporate a fair amount of dfalib into your application if
> you go that route, though.
>
> - The canonical code to map files into disk offsets can of course be found
> in the xnu source code, specifically in bsd/hfs (look at at the function
> hfs_vnop_blockmap, which from what I remember calls some functions down
> in the hfscommon subdirectory).
Thank you. I will check this stuff out.
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden