Re: Does Mac OS X support interior pointers?

On Sep 8, 2009, at 11:29 PM, John Engelhart wrote:

Bill, the point was this: __strong GC pointers will end up in /usr/lib code. For example:

const char *p = [@"String Me!" UTF8String];
printf("%s", p);

Since using a __strong GC pointer requires special hand holding, that pointer, where ever it may go, needs to be coded so that that special hand holding takes place.

That is a different issue, though. In that case, you are explicitly taking a GC controlled bit of memory and passing it to something explicitly non-GC. IN particular, it is an issue for which interior pointers are orthogonal.

The tools could do a better job of warning when that particular boundary is crossed.

The problem, though, is not one -- directly -- of interior pointers. It is of pointer escaping. Once a pointer escapes from scanned memory -- and a call to a non-GC aware API is a form of escape, most often harmless -- all bets are off unless the escape point is marked with a "hey, collector, don't collect this pointer until I say it is OK for you to collect this pointer".

That is done via CFRetain/CFRelease or via -disableCollectorForPointer: and -enableCollectorForPointer:. There probably should be synonymous API in objc/objc-auto.h, frankly (<rdar://problem/7208041>).

So, given:

const char *p = [@"String Me!" UTF8String];

printf("%s", p);

Or:

const char *p = NSAllocateCollectable(...);

someRandomNonGCCLibraryFunc(p);

The problem is the same.

Once 'p' crosses the boundary into that API, all bets are off as far as GC is concerned. In particular, the function might store away the pointer somewhere. It might XOR it with something prior to storing it (a surprisingly common thing to do). Or it might decide to store it in some non-pointer aligned fashion.

Effectively, ever GC'd application is running dual mode. All of the CF and Objective-C bits are running GC'd and all of the pure-C goop is running non-GC'd. At the boundary, you need to match impedance. That means from GC -> Non-GC, you need to tell the collector that outside forces are colluding to keep the pointer alive. For Non-GC -> GC, it means you might need to drop a free() into your otherwise GC clean code.

Interior pointers, in this context, only extend the range of the problem to include situations such as printf() where the function does not internally keep the pointer around.

Or, are you saying that it is illegal to pass __strong GC pointers to any stdlib function? Anything at all that a user might use in /usr/lib? Are you saying that NSOperation sanitizes every pointer that it passes to /usr/lib/icucore when it encounters a regex op?

No -- I'm saying that every boundary between GC'd code and non-GC'd code is a point of impedance mismatch that needs to, potentially, be considered. Most of the time, "it just works".

Most of the time, it is quite natural that the design/architecture of a body of code is such that a particular set of state -- set of objects -- survives across such call boundaries without a second thought from the developer.

None of this is to say that the problems being discussed aren't real. They are and figuring out how to support interior pointers in an efficient fashion is obviously desired.

Do you want help? I'm actually being serious. I can think of a couple of ways to do this quickly. I like a good challenge.

Is the problem that you're using a hash table? And you have no way to go from hash table -> "ranged" look up efficiently?

Can you point me to the appropriate open source auto.tar.gz file to look at? I think the last version put out was 71?

The code will be available Real Soon Now (There is a placeholder for autozone-141 on the open source page -- http://www.opensource.apple.com/release/mac-os-x-106/).

If you have patches or test cases, file a bug! (Common refrain, I know) http://bugreport.apple.com/

b.bum

References:
	>Does Mac OS X support interior pointers? (From: John Engelhart <email@hidden>)
	>Re: Does Mac OS X support interior pointers? (From: Quincey Morris <email@hidden>)
	>Re: Does Mac OS X support interior pointers? (From: John Engelhart <email@hidden>)
	>Re: Does Mac OS X support interior pointers? (From: Hamish Allan <email@hidden>)
	>Re: Does Mac OS X support interior pointers? (From: John Engelhart <email@hidden>)
	>Re: Does Mac OS X support interior pointers? (From: Hamish Allan <email@hidden>)
	>Re: Does Mac OS X support interior pointers? (From: John Engelhart <email@hidden>)
	>Re: Does Mac OS X support interior pointers? (From: Hamish Allan <email@hidden>)
	>Re: Does Mac OS X support interior pointers? (From: John Engelhart <email@hidden>)
	>Re: Does Mac OS X support interior pointers? (From: Bill Bumgarner <email@hidden>)
	>Re: Does Mac OS X support interior pointers? (From: John Engelhart <email@hidden>)