Re: 64-bit problem with VM statistics
Re: 64-bit problem with VM statistics
- Subject: Re: 64-bit problem with VM statistics
- From: Tony Scaminaci <email@hidden>
- Date: Sat, 4 Jun 2005 13:27:11 -0500
On Jun 2, 2005, at 1:43 AM, William Kucharski wrote:
I found your problem, and it's an interesting one.
The problem lies here in your code and the fact that you have your
project set to be compiled in 64-bit mode for G5s:
vm_size_t mac_pagesize;
[ ... ]
host_page_size(host_port, &mac_pagesize);
The issue is that /usr/include/mach/ppc/vm_types.h defines vm_size_t
as:
#if defined(__ppc__)
/*
* For 32-bit PowerPC ABIs, the scalable types were
* always based upon natural_t (unsigned int).
* Because of potential legacy issues with name mangling,
* we cannot use the stdint uintptr_t type.
*/
typedef natural_t vm_offset_t;
typedef natural_t vm_size_t;
#else /* __ppc64__ */
/*
* For 64-bit PowerPC ABIs, we have no legacy name mangling
* issues, so we use the stdint types for scaling these
* types to the same size as a pointer.
*/
typedef uintptr_t vm_offset_t;
typedef uintptr_t vm_size_t;
#endif
What this means is if __ppc__ is defined, vm_size_t will be an
unsigned int (size 4 bytes); if __ppc64__ is defined, vm_size_t will
be a unintptr_t (size 4 bytes on PPC32, 8 bytes on PPC64.)
The problem is that the kernel is NOT a 64-bit executable, and as such
is not compiled with __ppc64__ set, so when you make this call from a
64-bit executable:
host_page_size(host_port, &mac_pagesize);
the Darwin kernel does this:
kern_return_t
host_page_size(
host_t host,
vm_size_t *out_page_size)
{
if (host == HOST_NULL)
return(KERN_INVALID_ARGUMENT);
*out_page_size = PAGE_SIZE;
return(KERN_SUCCESS);
}
I think you probably see the issue here; since the kernel thinks a
vm_size_t is a four byte quantity but your 64-bit app thinks it's an
eight byte quantity, the kernel will fill in the upper four bytes of
the eight byte variable with the system page size and leave the rest
of the eight byte variable untouched. Thus when the kernel tries to
return 0x1000 (4096), your 64-bit app will instead see it as
0x100000000000, or 17592186044416 (!).
Why wouldn't it fill in the lower four bytes instead? I ask this
because when I print out mac_pagesize, I do get 4096 as expected so the
lower bytes must have the correct value.
Now your earlier wacky number of 281299054850211840 bytes doesn't
really calculate out properly, as given this error you would then get
281299054850211840/17592186044416 = 15990 pages, or only 62M free, not
the 4.4G or so you were theoretically expecting; if mac_pagesize were
an automatic I'd blame garbage left over in the "low" four bytes of
the variable, but since it's a global it was initialized as 0 so there
IS no garbage in the low four bytes.
Thus, two workarounds would be:
1) Compile 32-bit only
This would defeat the purpose of running a 64-bit executable - to test
more than 4 GB of memory. The memory address pointers have to be 64
bits if they're going to be able to access all the memory a user has
installed. John's machine for example has 15 GB installed so the main
pointer returned by malloc has to be 64 bits. The memory space gets
divided into two buffers, each 8 GB, so both these pointers must also
be 64 bit to address each buffer space. If I distribute the app as
32-bit only, then no one will be able to test more than 4GB on any
machine. There seem to be quite a few people out there with 4.5 GB or
more installed in their G5's and they're going to need to test it all
for reliability with 64-bit apps.
2) Declare mac_pagesize as a natural_t or unsigned int rather than a
vm_size_t. (Obviously this would mean that mac_pagesize wouldn't
agree with the man pages, but you've got to do what works.)
This is definitely an option if the 64-bit compiler will allow it. Even
worse, I could just set the page size to 4096 and be done with it. Not
a good idea in case it ever changes but it seems to be 4K on most
modern OS's these days.
This is why my earlier attempt worked fine; I didn't specify that gcc
should specifically compile my app as a 64-bit application.
Yep, I covered this with John Caldwell as well. He used gcc 3.3 which
doesn't compile 64-bit.
The question is, what SHOULD happen in this case, and should there be
an ADC tech note warning of this issue?
According to the Darwin 64-bit conversion guide
(<http://developer.apple.com/documentation/Darwin/Conceptual/
64bitPorting/index.html>), "Only libSystem and the Accelerate
framework are available to 64-bit developers. Higher level frameworks
are 32-bit only." But is a call to host_page_size() really considered
a "higher level framework?"
I'd consider host_page_size() to be part of libSystem if anything. If
we can't address more than 4 GB of memory, what's the point of having a
64-bit OS and a PPC G5? At the very least, Tiger should be able to get
at all of the user's installed memory assuming the user runs a 64-bit
app. How can the kernel handle more than 4 GB of address space if it's
not 64-bit?
There's got to be a simple workaround to this problem. I can't be the
only person trying to use more than 4GB of memory at one time. Any
application that bills itself as 64-bit has to address these issues.
I don't know the answer to that, so I'll leave it to the Apple folks
who frequent this list to sort out (Quinn would probably be the expert
here.)
William Kucharski
email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden