Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
- Subject: Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
- From: Michael Alfred Schmidt <email@hidden>
- Date: Mon, 26 Jan 2009 09:41:24 +0100
Hi Terry,
Thank you very much for cour comprehensive
comments that really hit the point! With HFSIOC_RESIZE_VOLUME,
the code works as expected. I will notify the author of 'Mac OS X Internals'
and notify him that the code snippet in his book needs to be modified.
I'm just considering your concern about
using the 'fsctl ()' kernel API call rather than 'diskutil'. For me (just
coming from the Windows world) the option to use an API call that is documented
in the open source community looks much more reliable and future-proof
than calling an executable that might not be present on every machine (because
somebody may have fiddled around with it) or whose calling conventions
might change over time. Maybe I'm wrong here. What's your take on this?
Thanks again,
Michael
| Terry Lambert <email@hidden>
23.01.2009 23:40
|
|
To
| Michael Alfred Schmidt <email@hidden>
|
|
cc
| Shantonu Sen <email@hidden>,
email@hidden
|
|
Subject
| Re: problems with hfs_truncatefs() /
HFS_RESIZE_VOLUME |
|
You have three problems. The first one is the
most severe:
1)
You should really be solving this problem by using popen() on the
diskutil command, rather than trying to replace/rewrite the diskutil
command. If you had started out this way, no matter what changes
we
made in the future, so long as the input and output of that command
didn't change, then your code would have been isolated and continued
to run and be happy. You didn't do this.
2)
The second problem is that you didn't give us all the information
up
front. The most important piece you left out for this particular
type
of problem is that you failed to tell us your code was 64 bit. When
passing size variant parameters to fsctl, ioctl, and fcntl, it's
important to know this. Effectively, these end up with different
definition values for their manifest constants, and when that happens,
they do not match the case statement entries for the 32 bit values, so
unless we've gone out of our way to deal with this issue, many of
these controls will not work from 32 bit code (size variance can also
be a function of structure packing, even if the parameter structure
contains no sized types).
Fortunately for you, the parameter in this case is 64 bit on both 32
and 64 bit platforms, but...
3)
Now the third problem: you are using the wrong command parameter to
accomplish what you are attempting to accomplish.
The command parameter you are using is HFS_RESIZE_VOLUME; the correct
one is HFSIOC_RESIZE_VOLUME:
#define HFSIOC_RESIZE_VOLUME _IOW('h', 2, u_int64_t)
#define HFS_RESIZE_VOLUME IOCBASECMD(HFSIOC_RESIZE_VOLUME)
What does IOCBASECMD() do? It strips the command size out of the
manifest constant, like so:
#define IOCBASEDCMD(x)
((x) & ~(IOCPARM_MASK << 16))
Why? Because when you are calling fsctl, fcntl, or ioctl with a size
variant argument value, this will allow you to have a single case
statement entry which is agnostic as to t whether or not the command
is coming from 32 or 64 bit code. This is useful, in that it allows
us to isolate the bit-ness dependence into the case statement entry.
This is why when it's decoding these, it applies the IOCBASEDCMD()
macro to the command argument, and uses the size-agnostic version of
the call to do the switch statement case entries, and then these
entries go on and cast the data portion of the command buffer to
parameter value pointers of the appropriate type, and dereference them
for input and output.
This works this way because the fsctl, fcntl, and ioctl code in the
kernel copies the parameter data into an internal kernel buffer (size
constrained) and deal with the data as kernel data rather than user
data (among other things, this prevents the race between validation
and use of the data in some cases where it would otherwise have to be
copied in twice). When the operation is done, it copies the data
back
out. Both of these copy operations are predicated on control bits
in
the command parameter (specifically, if IOC_IN is present, the data is
copied in on the way in and if IOC_OUT is present, it's copied out on
the way out).
So given that you made the call and told it that your data length is
0, how much data was copied in? 0 bytes. So what's in the buffer
'data' that's getting dereferenced out? Stack garbage. This
is why
the value is the same each time you call it. Luckily, since these
are
data interfaces behind an authorization check, so you don't have to
worry about this being a security issue; the data being dereferenced
is already in a kernel buffer, so it's not gong to result in a fault
on reference, and since it's no procedural, as long as we range check
our arguments (we do), then you're not going to do anything much
(except get yourself an error, which is what you saw was happening).
--
My suggestion would be to not attempt to use non-public interfaces
unless you understand precisely how they work; even the public
interfaces, as problem #2 demonstrates, can be problematic. Your
best
bet would be to wrap system tools which do what you want, and are well
enough documented and mature enough, that you aren't going to have to
worry much about them changing on you.
If you insist on going forward in writing your own low level code for
this instead of writing intermediate code to wrap the utility, well,
at least now you know why it was failing, why you were seeing the
results you were seeing, and to be more careful about research in the
future.
Cheers,
-- Terry
On Jan 23, 2009, at 7:28 AM, Michael Alfred Schmidt wrote:
> Thanks, I understand. Anyway, my goal is to shrink the volume rather
> than to expand it. So the repartitioning problem shouldn't apply.
>
> As I eventually managed to set up the kernel debugger, I was able
to
> verify the calling parameters of 'hfs_truncatefs()' (in the kernel
-
> hfs_vfsops.c). What I figured there is that the 'newsize' paramter
> always had a constant value, regardless of the various valid input
> parameters that I passed into userland 'fsctl()' for
> 'uiNewSize' (see my code). So I assume that this value gets lost
> somewhwere between userland and kernel. This is why I assume that
I
> might have some kind of thunking problem (maybe between 32-bit/64-
> bit code) with my input pointer. Unfortunately, my kernel debugging
> skills aren't good enough yet to trace the parameter passing to the
> kernel.
>
> Thanks anyway so far!
>
> Michael
>
>
>
> Shantonu Sen <email@hidden>
> 23.01.2009 15:52
>
> To
> Michael Alfred Schmidt <email@hidden>
> cc
> email@hidden
> Subject
> Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
>
>
>
>
>
> Please read my post again.
>
> diskutil is an integrated solution that does both repartitioning and
> filesystem resizing. The "limits" it prints out assume that
it is
> also going to be doing repartitioning.
>
> The 128MB freespace is outside of the partition boundaries. If you
> want to use it (which I don't recommend), you need to repartition
> first to increase the size of your partition to use some of that
> freespace, and then increase the filesystem to match.
>
> Shantonu
>
> Sent from my MacBook
>
> On Jan 23, 2009, at 12:25 AM, Michael Alfred Schmidt wrote:
>
>
> Shantonu,
>
> I'm testing on a MacBook Pro with default factory partitioning, i.e.
> there are 128 MB left free on top of the OS partition (see http://developer.apple.com/technotes/tn2006/tn2166.html#SECPARTITIONINGPOLICY)
> . I verify upfront with 'diskutil resizeVolume <resp. volume device>
> limits' what freedom to expand or shrink I have. Since my goal is
to
> shrink the partition rather than to expand it anyway, I concentrate
> my tests on shrinking (by ~ 10%).
>
> Michael
>
>
> Shantonu Sen <email@hidden>
> 22.01.2009 18:11
>
>
> To
> Michael Alfred Schmidt <email@hidden>
> cc
> mm w <email@hidden>, email@hidden
> Subject
> Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
>
>
>
>
>
>
>
> How do you know it has space to expand? You know you have to
> repartition the disk first, right (so that the container block
> device is large enough).
>
> This fsctl is something that is straightforward to use the same way
> the Disk Utility/diskutil use it. How big is your filesystem? How
> many allocation blocks? What sizes are you requesting?
>
> Shantonu Sen
> email@hidden
>
> Sent from my Mac Pro
>
> On Jan 22, 2009, at 9:06 AM, Michael Alfred Schmidt wrote:
>
>
> I do get ENOSPC in case the code runs into 'hfs_extendfs(): not
> enough space on device', or EINVAL in case the code runs into
> 'hfs_truncatefs(): invalid size'. I get both erros with my test
> program when I only try often enough, i. e. I get both errors for
> the same value. The same behavior (both errors possible) applies to
> any other arbitrary value of uiNewSize. This is why I believe that
> the value is currently somehow lost.
>
> The volume has both sufficient site to expand and to shrink.
>
> Michael
>
>
>
> mm w <email@hidden>
> 22.01.2009 17:50
>
>
> To
> Michael Alfred Schmidt <email@hidden>
> cc
> email@hidden
> Subject
> Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
>
>
>
>
>
>
>
>
>
> you certainly get a ENOSPC message somewhere, bad sample you are using
>
> - yoda
>
> Cheers!
>
> On Thu, Jan 22, 2009 at 8:16 AM, Michael Alfred Schmidt
> <email@hidden> wrote:
> >
> > Hi,
> >
> > I'm trying to shrink an HFS+ partition, just as 'diskutil
> resizeVolume...'
> > does. However, I'm facing problems implementing the
> HFS_RESIZE_VOLUME
> > command via fsctl(), as proposed by Amit Singh on p. 1572 of
'Mac
> OS X
> > Internals'.
> >
> > My code looks as follows:
> >
> > int iRet = 0;
> >
> > u_int32_t uiBlockSize;
> > u_int64_t uiBlockCount, uiCurrentSize, uiNewSize;
> >
> > struct statfs StatFS;
> >
> > // only works if device mounted
> > iRet = statfs ("/etc/", &StatFS);
> > if (iRet < 0)
> > {
> > perror ("statfs failed with");
> >
> > return;
> > }
> >
> > uiBlockSize = StatFS.f_bsize;
> > uiBlockCount = StatFS.f_blocks;
> >
> >
> > printf ("block size:\t %u\n", uiBlockSize);
> > printf ("block count:\t %llu\n", uiBlockCount);
> >
> >
> > uiCurrentSize = uiBlockSize * uiBlockCount;
> > printf ("partition size:\t %llu\n\n", uiBlockSize *
uiBlockCount);
> >
> > uiNewSize = uiCurrentSize; // only test here!!!
> > printf ("new volume size: %llu, hex: %llX\n", uiNewSize,
uiNewSize);
> >
> >
> > iRet = fsctl ("/", HFS_RESIZE_VOLUME, &uiNewSize,
0);
> > if (iRet < 0)
> > {
> > perror ("fsctl failed with");
> >
> > return;
> > }
> >
> >
> > fsctl() fails with either 'hfs_extendfs(): not enough space on
> device' or
> > 'hfs_truncatefs(): invalid size', even if the new size equals
the
> old size
> > (as in the code sample above). It looks like the &uiNewSize
input
> parameter
> > is fully ignored, as other "new sizes" show the same
unpredictable
> results.
> > On the kernel side, the respective code resides in bsd/hfs/
> hfs_readwrite.c.
> > >From looking at that code, I cannot see any obvious mistake
in my
> code. I'm
> > wondering whether there is any thunking (pointer conversion)
> problem with
> > the pointer to u_int64_t in the transition from userland to kernel.
> >
> > My target system is Mac OS X 10.5.x (Intel), compiled with XCode
> 3.1 with
> > default build settings.
> >
> >
> > I'm grateful for any idea on what's wrong with my code!
> > Thanks in advance!
> >
> > Michael
> >
> >
> > _______________________________________________
> > Do not post admin requests to the list. They will be ignored.
> > Darwin-kernel mailing list (email@hidden)
> > Help/Unsubscribe/Update your Subscription:
> >
> > This email sent to email@hidden
> >
>
>
>
> --
> -mmw
>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Darwin-kernel mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Darwin-kernel mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden