Re: How To?: Kernel Memory Mapped to User Space
Re: How To?: Kernel Memory Mapped to User Space
- Subject: Re: How To?: Kernel Memory Mapped to User Space
- From: Andrew Gallatin <email@hidden>
- Date: Thu, 6 Jun 2002 14:09:24 -0400 (EDT)
Godfrey van der Linden writes:
>
As Jim said it is usually better to map user memory into the kernel
>
and have the driver wire and unwire it. You don't really have to
>
worry about the app aborting without your being aware of it as the
>
act of wiring means that the memory stays valid until the DRIVER is
>
ready to get rid of it.
Unless you're wiring things for short bounded amounts of time, then
you *do* need to worry about it. If the app attempts to free the
wired memory, it will deadlock because the VM system is waiting for
the driver to unwire the memory. But the driver has no idea the VM
system wants it to unwire the memory.
Take a look at vm_map_delete() in System/xnu/osfmk/vm/vm_map.c
The comments explain things clearly:
<...>
/*
* All our DMA I/O operations in IOKit are currently done by
* wiring through the map entries of the task requesting the I/O.
* Because of this, we must always wait for kernel wirings
* to go away on the entries before deleting them.
*
* Any caller who wants to actually remove a kernel wiring
* should explicitly set the VM_MAP_REMOVE_KUNWIRE flag to
* properly remove one wiring instead of blasting through
* them all.
*/
flags |= VM_MAP_REMOVE_WAIT_FOR_KWIRE;
<...>
Note that VM_MAP_REMOVE_WAIT_FOR_KWIRE is set unconditionally.
Later on when trying to remove an entry whose wired count is nonzero:
if (flags & VM_MAP_REMOVE_WAIT_FOR_KWIRE) {
s = entry->vme_start;
entry->needs_wakeup = TRUE;
vm_map_entry_wait(map, interruptible);
At this point the interruptible flag is false (if it came through
vm_deallocate, which it almost certainly did), so the process is
unkillable.
As far as I can tell, OS-X gives a driver no way to know that an app
has shot its foot off by attempting to free memory you've wired for it.
The only way to recover from this is to have the driver unwire the
memory through operator intervention.
This is a serious problem for our software -- GM driver for
Myrinet (low-latency, high-bandwidth cluster interconnect,
http://www.myri.com) where user processes "register" (wire) large
parts of their address space for unbounded amounts of time for the
purpose of zero-copy OS-bypass networking.
The problem only happens when users violate our API which says you
must "deregister" (unwire) memory before freeing it. But its still a
nasty situation & nearly impossible for a hapless user to
debug. (They'll just complain their app "works fine on Linux" &
potentially not use OS-X.).
I really wish I could get somebody at Apple interested in fixing this
by providing some way for a driver to detect this situation. (Setting
a flag you can get to off the proc struct like Solaris does would be
nice)
Thanks,
-- Drew Gallatin (direct email gallatin AT myri DOT com)
Member of Technical Staff
Myricom Inc.
1575 Yauger Rd., Suite #34
Mount Vernon, OH. 43050
(740) 397-8629
_______________________________________________
darwin-kernel mailing list | email@hidden
Help/Unsubscribe/Archives:
http://www.lists.apple.com/mailman/listinfo/darwin-kernel
Do not post admin requests to the list. They will be ignored.