site_archiver@lists.apple.com Delivered-To: darwin-dev@lists.apple.com On Sep 22, 2009, at 3:22 PM, Terry Lambert wrote:

You have a LISP runtime which you are using internal compilation to generate code for language extensions and ... A little simpler, actually -- what is in my big memory-mapped block is just S-expressions, linked by pointers. No processor core's program counter is ever set to an address in that block. The Lisp system is an interpreter, and all code that actually runs is all in the normal text segments of the various processes that get to look at the shared block. Sometimes the content of an S-expression tells those processes what code to run, but the actual running code is all in their own text segments. You need to grab a big chunk in the middle of your address space, and not in your heap. It would help if I knew where the heap *was*. My impression is that the heap is relocatable, so presumably the linker or loader can simply put it in some likely location to avoid such other segments as I have claimed for my own use. This I try to do! Ideally, you'd fix the runtime to support PIC code generation so you can relocate segments for locality of reference to keep the address space access patterns non-fragmented over time (via feedback). You are thinking relocatable binary files, I believe. Think "linked lists" and "structs with pointers to other structs", instead -- that is what is in this big memory-mapped block. You really do not want to use all those .zerofill's. I haven't seen the output of your otool -l, but I rather suspect that each one might be getting it's own LC_SEGMENT_64. You may also be generating LC_SEGMENT's and linking them in instead (which gets you a 32 bit limit). Segment __PAGEZERO: 4294967296 (vmaddr 0x0 fileoff 0) Segment __TEXT: 528384 (vmaddr 0x100000000 fileoff 0) Section __text: 352852 (addr 0x100004a90 offset 19088) Section __symbol_stub1: 714 (addr 0x10005ace4 offset 371940) [...] total 509262 Segment __DATA: 163840 (vmaddr 0x100081000 fileoff 528384) Section __nl_symbol_ptr: 1152 (addr 0x100081000 offset 528384) Section __la_symbol_ptr: 952 (addr 0x100081480 offset 529536) [...] total 163017 Segment __WSS: 171798691840 (vmaddr 0x10000000000 fileoff 606208) Section __Wss000: 1073741824 (addr 0x10000000000 offset 0) Section __Wss004: 1073741824 (addr 0x10040000000 offset 0) Section __Wss008: 1073741824 (addr 0x10080000000 offset 0) [...] Section __Wss159: 1073741824 (addr 0x127c0000000 offset 0) total 171798691840 Segment __LINKEDIT: 131072 (vmaddr 0x12800000000 fileoff 606208) total 176094482432 <end of printout> That was for my .s file with 160 1-GByte sections. When I add another .zerofill, making 161 1-G-byte sections, I get a size -l output that appears to be exactly the same, except it correctly lists 161 sections in Segment __WSS, for a total segment size of 172872433664.

-- Jay Reynolds Freeman --------------------- Jay_Reynolds_Freeman@mac.com http://web.mac.com/jay_reynolds_freeman (personal web site) _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.appl...

If at all possible, you also need to arrange your code generation

such that your normal access pattern isn't sparse in the memory region

you allocate. If you perform an mmap() very early on of zero-fill pages, then you

can do it as a .constructor or as part of an early constructor, or

very early in your runtime, and that will ensure that you get the common

mapping that you want. I believe you, but this is fundamentally a static-linking problem, and

I keep thinking that is precisely what the linker is supposed to do! Using run-time operations to solve a static linking problem just does not sound like the right thing to do ... At the very least I am going to file a bug about .zerofill not accepting 64-bit section sizes. I suspect that was an oversight in going to 64- bit code. However, the .zerofills appear to be doing their job; keep reading. I ran size -l on the executable (inside the app wrapper) and get the following: I will spare you all the printout; in what follows, "__WSS" is the big segment that I create. The first executable -- the one with 160 sections -- loads and runs fine. The second gives errors early in the load about not being able to load certain classes. This with *NO* other changes to anything. This email sent to site_archiver@lists.apple.com