| Hi Regis,
You are welcome.
Good luck.
On 7 Jan 2015, at 11:10 am, Regis Duchesne <email@hidden> wrote:
Thanks for the additional info, Christopher!
--
Regis "HPReg" Duchesne
Technical Lead for Fusion & Mac OS Guest
VMware, Inc.
On Jan 6, 2015, at 7:47 PM, Christopher Chandler <email@hidden> wrote:
Hi Regis,
Replying to your comments:
1) SDKs and compilers:
Yes. Basically I try to use the correct compiler version(s) with the correct SDKs. The downside of this is that of course you lose bug fixes that have been made in newer compiler versions, and if you’re trying to use newer language features (etc.) you need to partition your code into libraries etc. The easiest way I’ve found of doing this is to use an Xcode config file, which allows you to specify the particular SDK, compiler etc. Then you can swap in and out tools depending on the build target, and over-ride the new defaults in the .config file (if necessary) in specific targets
2) DYLD_LIBRARY_PATH:
That’s definitely neater than hack-rebuilding the code. To some extent, I was investigating how hard it would be to do it, and as I said, I needed something quick & dirty quickly. As I’m sure you’re aware, sometimes making DYLD_LIBRARY_PATH work can be a bit awkward especially as some applications watch this for security threats etc.
3) Source code changes. I grep-ed the source and found quite a few instances of map_fd() (and mmap() strangely - it’s as though Apple started using mmap() in the newer features but never went back and fixed the older code). In order to link ld.NEW for my purposes, I only needed to fix the ones in ld.c and pass1.c. In all cases, map_fd() is used to read in an existing file opened read-only into memory in one go. No changes are written back to the file. The memory needs to be writable, presumably so that the linker can update fields etc.
4) Replace map_fd() with mmap().
Here’s the map_fd() prototype, according to http://www.cs.cmu.edu/afs/cs/project/mach/public/doc/published/mapfiles87.ps
map_fd(fd, offset, addr, find_space, numbytes)
int fd;
vm_offset_t offset;
vm_offset_t *addr;
boolean_t find_space;
vm_size_t numbytes;
For example: /Darwin/ADC/Common/Xcode-3_1_4/cctools-698.1/ld/ld.c, line 1020 (or so - )
if((fd = open(argv[i+3], O_RDONLY, 0)) == -1)
system_fatal("Can't open: %s for %s %s %s",
argv[i+3], argv[i], argv[i+1], argv[i+2]);
if(fstat(fd, &stat_buf) == -1)
system_fatal("Can't stat file: %s for %s %s %s",
argv[i+3], argv[i], argv[i+1], argv[i+2]);
/*
* For some reason mapping files with zero size fails
* so it has to be handled specially.
*/
if(stat_buf.st_size != 0){
#if 0 /* CJEC, 24-Dec-14: map_fd() has been deprecated in OS X 10.10.x */
if((r = map_fd((int)fd, (vm_offset_t)0,
(vm_offset_t *)&(sect_spec->file_addr),
(boolean_t)TRUE, (vm_size_t)stat_buf.st_size)
) != KERN_SUCCESS)
mach_fatal(r, "can't map file: %s for %s %s %s",
argv[i+3], argv[i], argv[i+1], argv[i+2]);
#else
sect_spec -> file_addr = mmap (NULL, stat_buf.st_size, PROT_READ | PROT_WRITE, MAP_FILE | MAP_PRIVATE, fd, 0);
if (sect_spec -> file_addr == NULL)
system_fatal ("%s %s{%u}: Can't mmap file: %s for %s %s %s", __PRETTY_FUNCTION__, __FILE__, __LINE__, argv [i+3], argv [i], argv [i+1], argv [i+2]);
#endif
}
else{
sect_spec->file_addr = NULL;
}
close(fd);
For your solution, the good news is that map_fd() seems to be always used in the same way. So you could implement a specific solution that satisfies this particular set of map_fd() options (addr == NULL, and find_space == true), and throw some kind of fatal error if it map_fd() is called in any way differently.
Good luck - I’m sure it will work.
Regards,
Chris
On 5 Jan 2015, at 12:04 pm, Regis Duchesne <email@hidden> wrote:
Christopher,
For those of you who are still trying to support older OS X releases, you might like to know that I’ve investigated this further. The good news is that Xcode 6 still works with the older plugin formats, build tool configuration files etc. so follow the same approach with Xcode 6 as I indicated with Xcode 5 previously http://lists.apple.com/archives/darwin-kernel/2014/May/msg00017.html My environment now incudes Xcode 6.1.1 on top of 5.0.3, 4.6.2, and 3.2.6, and I can build on all SDKs and platforms from OS X 10.4
Since Apple does not ship earlier SDKs with later Xcode versions, I'm afraid this technique may not work in the long run: newer compilers may require you to edit older SDKs in order to compile.
Instead I simply install multiple versions of Xcode on the same box. But that only works well because I don't use Xcode projects, I use the compilers/linkers directly within make files (SCons files actually).
There is one major new gotcha. It seems that Apple has removed the deprecated map_fd() system call from OS X 10.10.1 (and probably 10.10.0 but I have not tested it) and so /Xcode-3_2_6/usr/bin/ld_classic, the 32 bit old linker, crashes when you attempt to use it to link a 10.4/10.5 compatible kernel extension. I managed to work around this by downloading the Xcode 3.1.4 cc-tools source from http://www.opensource.apple.com/release/developer-tools-314/ and hacking the environment and makefiles enough to be able to build ld.NEW under OS X 10.6.8, 32 bit mode and Xcode 3.2.6.
I have been thinking about doing this a different way: create a new map_fd.dylib that only provides the map_fd() symbol, implemented in terms of a mmap() call. Then invoke the Apple binary by setting DYLD_LIBRARY_PATH and/or DYLD_INSERT_LIBRARIES to make use of that new map_fd.dylib.
The advantage is that there is nothing to recompile from open source: you keep using the official Apple ld_classic binary. And you can use the same map_fd.dylib for other programs that need the map_fd symbol.
(Virtual machines are a wonderful thing!)
Yes, and it is precisely to keep virtual machines (really VMware Tools) running that I need to be able to build 32-bit kexts on a 10.10 host!
Once I was able to build the ld_build target in cctools-698.1/ld/Makefile, I modified ld.c and pass1.c replacing the map_fd() calls with mmap (…PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_FILE..,). This successfully links a 10.4 compatible kernel extension under OS X 10.10 using Xcode 6/5/4/3 together.
To implement my own map_fd() function, I would be interested in the exact modification you did (that will save me a bit of time). I.e. what was the exact map_fd() call code you replaced, and with what exact mmap() call did you replace it.
Thanks,
--
Regis "HPReg" Duchesne
Technical Lead for Fusion & Mac OS Guest
VMware, Inc.
|