Re: What should we do when headers are missing in 10.2?
Re: What should we do when headers are missing in 10.2?
- Subject: Re: What should we do when headers are missing in 10.2?
- From: Andrew Gallatin <email@hidden>
- Date: Wed, 11 Sep 2002 13:35:33 -0400 (EDT)
Jim Magee writes:
>
>
On Wednesday, September 11, 2002, at 11:49 AM, Andrew Gallatin wrote:
>
> I could make all uses APPLE_API_PRIVATE headers go away in my driver
>
> (OS-Bypass cluster interconnect --
>
> http://www.myri.com/scs/macosx.html) if somebody would just fix the
>
> VM system so that it wouldn't block a user process indefinatly if said
>
> process attempts to free memory locked via IOMemoryDescriptor's
>
> prepare function. But I couldn't seem to get anybody interested in
>
> changing this behaviour in the past...
>
>
I think I even exchanged some mail with you on this subject last time.
Yes, I think you did. To refresh, the problem happens when an app has
registered memory for os-bypass IO via an ioctl into our driver. The
memory is locked via prepare. In the normal course of operations, a
task will deregister the memory (and our driver will unprepare it,
thus unlocking it) prior to freeing it. The problem occurs when a
buggy applications neglects to deregister the memory prior to freeing
it. This is a bug in the application (they've violated our API). But
the resulting behaviour (process is unkillable, sleeping waiting for
memory to be unprepared; our driver won't unprepare the memory because
it has no idea that it should) is sub optimal.
>
Doing what you ask would require "prepare" to create and hold its own
>
reference to the underlying memory objects rather than "glom-ing" on to
>
the user's reference (leaving the user free to release his own with
>
impunity). This could be done, but it would slow prepare down even
>
more in its current form (and it is in dire need of a performance boost
>
already). In fact, we are in the process of investigating possible
>
enhancements to the implementation. The leading design does involve
>
burrowing all the way down to the object, so we do have the opportunity
>
to think about taking our own references again. This would allow
>
unprepare to be much more efficient, and subsequent prepares in the
>
same object could also be quicker. But is also allows a possible
>
redress of your issue.
Excellent!
Also, memory locking on MacOSX is currently slower than on any other
unix-like platform we support. It would be good to see it sped up.
>
The real question is: Does anything depend upon that portion of the
>
address space being locked down while I/O is in progress? It would be
>
difficult to see how they could, because the prepare gets released
>
before their notion of an I/O operation is marked as complete. But
>
it's just something to investigate.
Nothing depends on that part of the address space being locked down.
However, the physical memory (formerly) backing the portion of the
apps virtual address space which it has just freed cannot be recycled.
Otherwise we risk either corrupting memory if those pages end up
belonging to some other task. Ideally, the memory would be removed
from the app's address space when it called free(), and the physical
pages would held in limbo, only freed back to the system when we
unprepared it and some reference count dropped to zero.
This is what our linux driver does (by hand, in our driver, since
there is no api for it. 300+ lines of code which change frequently.
Ick).
Actually, I suppose it could be freed if you had a sophisticated IOMMU
and could remap those addresses to the bit bucket when they were
accessed via dma. ;)
>
--Jim
>
>
PS: I think, last we left it, we also talked about changing the timing
>
between file-descriptor closing and VM cleaning. We intentionally
>
clean up the Mach port space (which IOKit UserClient interfaces
>
typically use) before the VM space so that UserClient memory
>
descriptors would be freed before attempting to free the memory in an
>
exiting process. Closing file descriptors should happen even before
>
that (to allow file-descriptor-based drivers to release their holds on
>
memory). While this may not be necessary, if we do the work above, it
>
is still the cleaner way to go.
This like what I was asking for... Has the behaviour changed? I
haven't looked into this since I wrote my reaper thread work around,
which uses APPLE_API_PRIVATE apis.
The problem I was running into earlier was due to the process's VM
space being cleaned up _prior_ to its file-descriptors being closed.
The effect would be similar to what I talk about above -- the VM
cleanup code frees registered memory, leading to a deadlock. Closing
the file-descriptors first avoids the deadlock, because our driver
deregisters the app's memory at this point.
Thanks!
Drew
_______________________________________________
darwin-kernel mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/darwin-kernel
Do not post admin requests to the list. They will be ignored.