Re: How to read files from disk directly?
Re: How to read files from disk directly?
- Subject: Re: How to read files from disk directly?
- From: Shantonu Sen <email@hidden>
- Date: Sun, 3 Jul 2011 20:51:47 -0700
This is a terrible idea, and you should forget it.
Defragmenting applications use the volume format described as <http://developer.apple.com/library/mac/#technotes/tn/tn1150.html>
This is not the same thing as the logical filesystem that is exposed to userspace. For instance, filesystem compression (introduced in Mac OS X 10.6 Snow Leopard) can prevent you from being able to reconstruct a file's contents by accessing the raw device.
What are you really trying to do?
Shantonu
On Jul 3, 2011, at 7:16 PM, Eric Gorr wrote:
>
> On Jul 3, 2011, at 9:27 PM, Ken Hornstein wrote:
>
>>> So, the primary question I have is when reading the data from my
>>> device directly, how can I determine exactly what data belongs with
>>> what file?
>>
>> I think most people would tell you that this is a terrible idea and to
>> forget it. However, I'm not most people :-)
>
> Good :-)
>
> I like it a lot better when people actually answer the question I ask, so thank you.
>
> Performance is very important in my case. I don't have the numbers in front of me, but there is simply no comparison between the speed in which I can read a single 4gb file and 190,000 small files that add up to 4gb.
>
>> I will say, upfront, that I don't think it's a great idea. You're setting
>> yourself up for a lot of complexity and honestly I don't think you'll really
>> gain a whole lot in terms of performance.
>
> Perhaps, but it would be interesting to find out.
>
>> And I don't think Apple will
>> make any guarantees that the filesystem format is stable, so you might
>> find your application not working in the future.
>
> But, if I am using public APIs to obtain the information, I would assume it would remain somewhat stable. I would assume that defragmentation applications have to do something similar to this and they seem to remain somewhat stable, but perhaps I am mistaken.
>
>> But, hey, if you're REALLY determined to do this, here's what I can
>> tell you:
>
> Well, I am somewhat determined to find out whether or not I will see performance improvements.
>
>> - I _was_ going to say that there isn't an userspace API which lets you turn
>> a filename into a list of physical block offsets. But on further
>> inspection I realize I was wrong, and there _is_ such an API.
>> Specifically, fcntl() has a F_LOG2PHYS option. But I don't think that
>> will help you all that much, because to use that you still have to open()
>> a file to get a descriptor for that, and you'd have to do a crapload of
>> system calls to get all of the physical block offsets and by that point
>> you'd probably be better off just reading the damn file yourself.
>
> Nice. I was not aware of the F_LOG2PHYS option of fcntl. What are the other system calls that would get me all of the physical block offsets? Or is that what you describe below?
>
>> - If you want an example of something that does close to what you want,
>> look at the source code of fsck_hfs (part of diskdev_cmds). Specifically
>> check out what the -B option of fsck_hfs does. You're probably going
>> to have incorporate a fair amount of dfalib into your application if
>> you go that route, though.
>>
>> - The canonical code to map files into disk offsets can of course be found
>> in the xnu source code, specifically in bsd/hfs (look at at the function
>> hfs_vnop_blockmap, which from what I remember calls some functions down
>> in the hfscommon subdirectory).
>
> Thank you. I will check this stuff out.
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Filesystem-dev mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden