Re: Problem using .zerofill / -segaddr to create very large segments
Re: Problem using .zerofill / -segaddr to create very large segments
- Subject: Re: Problem using .zerofill / -segaddr to create very large segments
- From: Jay Reynolds Freeman <email@hidden>
- Date: Tue, 22 Sep 2009 17:29:03 -0700
On Sep 22, 2009, at 3:22 PM, Terry Lambert wrote:
> You have a LISP runtime which you are using internal
> compilation to generate code for language extensions and ...
A little simpler, actually -- what is in my big memory-mapped block
is just S-expressions, linked by pointers. No processor core's program
counter is ever set to an address in that block. The Lisp system is
an interpreter, and all code that actually runs is all in the normal
text segments of the various processes that get to look at the shared
block. Sometimes the content of an S-expression tells those processes
what code to run, but the actual running code is all in their own
text segments.
> You need to grab a big chunk in the middle of your address space,
> and not in your heap.
It would help if I knew where the heap *was*. My impression is that
the heap is relocatable, so presumably the linker or loader can simply
put it in some likely location to avoid such other segments as I have
claimed for my own use.
> If at all possible, you also need to arrange your code generation
> such that your normal access pattern isn't sparse in the memory
region
> you allocate.
This I try to do!
> Ideally, you'd fix the runtime to support PIC code generation so you
> can relocate segments for locality of reference to keep the address
> space access patterns non-fragmented over time (via feedback).
You are thinking relocatable binary files, I believe. Think "linked
lists" and "structs with pointers to other structs", instead -- that is
what is in this big memory-mapped block.
> If you perform an mmap() very early on of zero-fill pages, then you
> can do it as a .constructor or as part of an early constructor, or
> very early in your runtime, and that will ensure that you get the
common
> mapping that you want.
I believe you, but this is fundamentally a static-linking problem,
and
I keep thinking that is precisely what the linker is supposed to do!
Using
run-time operations to solve a static linking problem just does not
sound
like the right thing to do ...
> You really do not want to use all those .zerofill's.
At the very least I am going to file a bug about .zerofill not
accepting
64-bit section sizes. I suspect that was an oversight in going to 64-
bit
code. However, the .zerofills appear to be doing their job; keep
reading.
> I haven't seen the output of your otool -l, but I rather suspect that
> each one might be getting it's own LC_SEGMENT_64. You may also be
> generating LC_SEGMENT's and linking them in instead (which gets you
> a 32 bit limit).
I ran size -l on the executable (inside the app wrapper) and get the
following:
I will spare you all the printout; in what follows, "__WSS" is the big
segment
that I create.
Segment __PAGEZERO: 4294967296 (vmaddr 0x0 fileoff 0)
Segment __TEXT: 528384 (vmaddr 0x100000000 fileoff 0)
Section __text: 352852 (addr 0x100004a90 offset 19088)
Section __symbol_stub1: 714 (addr 0x10005ace4 offset 371940)
[...]
total 509262
Segment __DATA: 163840 (vmaddr 0x100081000 fileoff 528384)
Section __nl_symbol_ptr: 1152 (addr 0x100081000 offset 528384)
Section __la_symbol_ptr: 952 (addr 0x100081480 offset 529536)
[...]
total 163017
Segment __WSS: 171798691840 (vmaddr 0x10000000000 fileoff 606208)
Section __Wss000: 1073741824 (addr 0x10000000000 offset 0)
Section __Wss004: 1073741824 (addr 0x10040000000 offset 0)
Section __Wss008: 1073741824 (addr 0x10080000000 offset 0)
[...]
Section __Wss159: 1073741824 (addr 0x127c0000000 offset 0)
total 171798691840
Segment __LINKEDIT: 131072 (vmaddr 0x12800000000 fileoff 606208)
total 176094482432
<end of printout>
That was for my .s file with 160 1-GByte sections. When I add another
.zerofill, making 161 1-G-byte sections, I get a size -l output that
appears to be exactly the same, except it correctly lists 161 sections
in Segment __WSS, for a total segment size of 172872433664.
The first executable -- the one with 160 sections -- loads and runs
fine.
The second gives errors early in the load about not being able to load
certain classes. This with *NO* other changes to anything.
-- Jay Reynolds Freeman
---------------------
email@hidden
http://web.mac.com/jay_reynolds_freeman (personal web site)
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden