Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
- Subject: Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
- From: Terry Lambert <email@hidden>
- Date: Fri, 23 Jan 2009 14:40:30 -0800
You have three problems. The first one is the most severe:
1)
You should really be solving this problem by using popen() on the
diskutil command, rather than trying to replace/rewrite the diskutil
command. If you had started out this way, no matter what changes we
made in the future, so long as the input and output of that command
didn't change, then your code would have been isolated and continued
to run and be happy. You didn't do this.
2)
The second problem is that you didn't give us all the information up
front. The most important piece you left out for this particular type
of problem is that you failed to tell us your code was 64 bit. When
passing size variant parameters to fsctl, ioctl, and fcntl, it's
important to know this. Effectively, these end up with different
definition values for their manifest constants, and when that happens,
they do not match the case statement entries for the 32 bit values, so
unless we've gone out of our way to deal with this issue, many of
these controls will not work from 32 bit code (size variance can also
be a function of structure packing, even if the parameter structure
contains no sized types).
Fortunately for you, the parameter in this case is 64 bit on both 32
and 64 bit platforms, but...
3)
Now the third problem: you are using the wrong command parameter to
accomplish what you are attempting to accomplish.
The command parameter you are using is HFS_RESIZE_VOLUME; the correct
one is HFSIOC_RESIZE_VOLUME:
#define HFSIOC_RESIZE_VOLUME _IOW('h', 2, u_int64_t)
#define HFS_RESIZE_VOLUME IOCBASECMD(HFSIOC_RESIZE_VOLUME)
What does IOCBASECMD() do? It strips the command size out of the
manifest constant, like so:
#define IOCBASEDCMD(x) ((x) & ~(IOCPARM_MASK << 16))
Why? Because when you are calling fsctl, fcntl, or ioctl with a size
variant argument value, this will allow you to have a single case
statement entry which is agnostic as to t whether or not the command
is coming from 32 or 64 bit code. This is useful, in that it allows
us to isolate the bit-ness dependence into the case statement entry.
This is why when it's decoding these, it applies the IOCBASEDCMD()
macro to the command argument, and uses the size-agnostic version of
the call to do the switch statement case entries, and then these
entries go on and cast the data portion of the command buffer to
parameter value pointers of the appropriate type, and dereference them
for input and output.
This works this way because the fsctl, fcntl, and ioctl code in the
kernel copies the parameter data into an internal kernel buffer (size
constrained) and deal with the data as kernel data rather than user
data (among other things, this prevents the race between validation
and use of the data in some cases where it would otherwise have to be
copied in twice). When the operation is done, it copies the data back
out. Both of these copy operations are predicated on control bits in
the command parameter (specifically, if IOC_IN is present, the data is
copied in on the way in and if IOC_OUT is present, it's copied out on
the way out).
So given that you made the call and told it that your data length is
0, how much data was copied in? 0 bytes. So what's in the buffer
'data' that's getting dereferenced out? Stack garbage. This is why
the value is the same each time you call it. Luckily, since these are
data interfaces behind an authorization check, so you don't have to
worry about this being a security issue; the data being dereferenced
is already in a kernel buffer, so it's not gong to result in a fault
on reference, and since it's no procedural, as long as we range check
our arguments (we do), then you're not going to do anything much
(except get yourself an error, which is what you saw was happening).
--
My suggestion would be to not attempt to use non-public interfaces
unless you understand precisely how they work; even the public
interfaces, as problem #2 demonstrates, can be problematic. Your best
bet would be to wrap system tools which do what you want, and are well
enough documented and mature enough, that you aren't going to have to
worry much about them changing on you.
If you insist on going forward in writing your own low level code for
this instead of writing intermediate code to wrap the utility, well,
at least now you know why it was failing, why you were seeing the
results you were seeing, and to be more careful about research in the
future.
Cheers,
-- Terry
On Jan 23, 2009, at 7:28 AM, Michael Alfred Schmidt wrote:
Thanks, I understand. Anyway, my goal is to shrink the volume rather
than to expand it. So the repartitioning problem shouldn't apply.
As I eventually managed to set up the kernel debugger, I was able to
verify the calling parameters of 'hfs_truncatefs()' (in the kernel -
hfs_vfsops.c). What I figured there is that the 'newsize' paramter
always had a constant value, regardless of the various valid input
parameters that I passed into userland 'fsctl()' for
'uiNewSize' (see my code). So I assume that this value gets lost
somewhwere between userland and kernel. This is why I assume that I
might have some kind of thunking problem (maybe between 32-bit/64-
bit code) with my input pointer. Unfortunately, my kernel debugging
skills aren't good enough yet to trace the parameter passing to the
kernel.
Thanks anyway so far!
Michael
Shantonu Sen <email@hidden>
23.01.2009 15:52
To
Michael Alfred Schmidt <email@hidden>
cc
email@hidden
Subject
Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
Please read my post again.
diskutil is an integrated solution that does both repartitioning and
filesystem resizing. The "limits" it prints out assume that it is
also going to be doing repartitioning.
The 128MB freespace is outside of the partition boundaries. If you
want to use it (which I don't recommend), you need to repartition
first to increase the size of your partition to use some of that
freespace, and then increase the filesystem to match.
Shantonu
Sent from my MacBook
On Jan 23, 2009, at 12:25 AM, Michael Alfred Schmidt wrote:
Shantonu,
I'm testing on a MacBook Pro with default factory partitioning, i.e.
there are 128 MB left free on top of the OS partition (see http://developer.apple.com/technotes/tn2006/tn2166.html#SECPARTITIONINGPOLICY)
. I verify upfront with 'diskutil resizeVolume <resp. volume device>
limits' what freedom to expand or shrink I have. Since my goal is to
shrink the partition rather than to expand it anyway, I concentrate
my tests on shrinking (by ~ 10%).
Michael
Shantonu Sen <email@hidden>
22.01.2009 18:11
To
Michael Alfred Schmidt <email@hidden>
cc
mm w <email@hidden>, email@hidden
Subject
Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
How do you know it has space to expand? You know you have to
repartition the disk first, right (so that the container block
device is large enough).
This fsctl is something that is straightforward to use the same way
the Disk Utility/diskutil use it. How big is your filesystem? How
many allocation blocks? What sizes are you requesting?
Shantonu Sen
email@hidden
Sent from my Mac Pro
On Jan 22, 2009, at 9:06 AM, Michael Alfred Schmidt wrote:
I do get ENOSPC in case the code runs into 'hfs_extendfs(): not
enough space on device', or EINVAL in case the code runs into
'hfs_truncatefs(): invalid size'. I get both erros with my test
program when I only try often enough, i. e. I get both errors for
the same value. The same behavior (both errors possible) applies to
any other arbitrary value of uiNewSize. This is why I believe that
the value is currently somehow lost.
The volume has both sufficient site to expand and to shrink.
Michael
mm w <email@hidden>
22.01.2009 17:50
To
Michael Alfred Schmidt <email@hidden>
cc
email@hidden
Subject
Re: problems with hfs_truncatefs() / HFS_RESIZE_VOLUME
you certainly get a ENOSPC message somewhere, bad sample you are using
- yoda
Cheers!
On Thu, Jan 22, 2009 at 8:16 AM, Michael Alfred Schmidt
<email@hidden> wrote:
>
> Hi,
>
> I'm trying to shrink an HFS+ partition, just as 'diskutil
resizeVolume...'
> does. However, I'm facing problems implementing the
HFS_RESIZE_VOLUME
> command via fsctl(), as proposed by Amit Singh on p. 1572 of 'Mac
OS X
> Internals'.
>
> My code looks as follows:
>
> int iRet = 0;
>
> u_int32_t uiBlockSize;
> u_int64_t uiBlockCount, uiCurrentSize, uiNewSize;
>
> struct statfs StatFS;
>
> // only works if device mounted
> iRet = statfs ("/etc/", &StatFS);
> if (iRet < 0)
> {
> perror ("statfs failed with");
>
> return;
> }
>
> uiBlockSize = StatFS.f_bsize;
> uiBlockCount = StatFS.f_blocks;
>
>
> printf ("block size:\t %u\n", uiBlockSize);
> printf ("block count:\t %llu\n", uiBlockCount);
>
>
> uiCurrentSize = uiBlockSize * uiBlockCount;
> printf ("partition size:\t %llu\n\n", uiBlockSize * uiBlockCount);
>
> uiNewSize = uiCurrentSize; // only test here!!!
> printf ("new volume size: %llu, hex: %llX\n", uiNewSize, uiNewSize);
>
>
> iRet = fsctl ("/", HFS_RESIZE_VOLUME, &uiNewSize, 0);
> if (iRet < 0)
> {
> perror ("fsctl failed with");
>
> return;
> }
>
>
> fsctl() fails with either 'hfs_extendfs(): not enough space on
device' or
> 'hfs_truncatefs(): invalid size', even if the new size equals the
old size
> (as in the code sample above). It looks like the &uiNewSize input
parameter
> is fully ignored, as other "new sizes" show the same unpredictable
results.
> On the kernel side, the respective code resides in bsd/hfs/
hfs_readwrite.c.
> >From looking at that code, I cannot see any obvious mistake in my
code. I'm
> wondering whether there is any thunking (pointer conversion)
problem with
> the pointer to u_int64_t in the transition from userland to kernel.
>
> My target system is Mac OS X 10.5.x (Intel), compiled with XCode
3.1 with
> default build settings.
>
>
> I'm grateful for any idea on what's wrong with my code!
> Thanks in advance!
>
> Michael
>
>
> _______________________________________________
> Do not post admin requests to the list. They will be ignored.
> Darwin-kernel mailing list (email@hidden)
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
>
--
-mmw
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden