Re: shm_open and mmap: Invalid argument?
Re: shm_open and mmap: Invalid argument?
- Subject: Re: shm_open and mmap: Invalid argument?
- From: Tim Hewett <email@hidden>
- Date: Mon, 21 Feb 2005 10:57:23 +0000
Ethan,
On 21 Feb 2005, at 06:21, Ethan Tira-Thompson wrote:
Thanks to everyone for their insight... I've gotten things working,
but there may be more here for you to work on amongst yourselves
(assuming there's a few kernel hackers on here...) Actually, I think
documentation or a code sample was the real shortcoming, and that's
not limited to Darwin (I did extensive web searching before winding up
here)
So, ftruncate does seem to be key in the creation of a shared memory
region - after shm_open, before mmap. However, it should only be
called on a *new* region at its creation... it will return an error if
you're connecting to a pre-existing region. That seems reasonable
enough, but I found this aspect to be rather undocumented. (I suppose
the size/len argument of mmap is supposed to be used to map a
subsection of a file into memory, so you have to use ftruncate to
pre-size the shared memory region, and then can selectively map
portions of the region into the process. I initially expected mmap to
do the region sizing itself...)
When your processes attach to the shared region how do they
know the size? Is it an application-wide constant? The call to
mmap has to have the size (since you can choose not to map
all the region). I've found that fstat() gets it fine on an inherited
shared memory descriptor from the parent.
The next step is shm_unlink. It seems it should *not* be called right
after the creation in order to request dereference down the road (i.e.
shm_open, ftruncate, mmap, close, shm_unlink) shm_unlink will remove
the name immediately, thus blocking other processes from accessing it
since it's on the way out. However, I suppose it's fine to call
immediately following the last attach you are expecting, to guarantee
that a sudden crash will cause it to be removed.
It is beginning to look like my approach with shm_unlink is going
to be a problem. My circumstance is that a controlling process
creates the shared segment and spawns short-lived children which
use the shared segment. Those children ought to be able to use
the inherited file descriptor that the parent received from shm_open()
when creating the region. They ought not to have to call shm_open()
themselves to attach to the region, and so the region ought not to
have to exist in its name space (is there a "shm_ls" feature so you
can see what's in the name space??). This should mean that the
process creating the region can unlink it immediately and just use
common inheritance for subprocesses to attach to it.
The shm_open man page certainly suggests that this should be
possible, i.e. "The new descriptor is set to remain open across
execve system calls" (though it directly contradicts itself earlier
stating "The new file descriptor will have the FD_CLOEXEC flag set")
but this suggests inheritability of the file descriptor and it would be
natural for that to happen anyway.
In my case the file descriptor has the FD_CLOEXEC flag set on it
explicitly immediately after its creation, so that it isn't inherited
across exec unnecessarily, however for the children that use the
shared region this descriptor is dup'd, which implicitly causes this
flag to be cleared. The descriptor is definitely being inherited
across exec anyway since the call to fstat() on it is successful and
returns expected results. It is the mmap() which is the problem
returning EINVAL, and it seems to me that this is not what should
happen.
The safest general behavior (if you don't know when an attach is the
last one) seems to be to wait until the last reference is being
removed, and then call shm_unlink. This is directly analogous to
using shmctl to delete a region under SysV. (initially I had called
it on *every* dereference, so I'm not completely sure what happens if
it's not called for the last dereference, or if I the errors I got
were for the first call or the last call (i.e. did the first call
succeed and following ones fail, or did all fail except the last?))
This brings up an interesting problem however. If you shm_open a
bunch of regions, and never unlink them, where do they go when the
process gets shutdown? My experience from the intial email implies
they stick around (blocking further attempts to make new regions of
the same name). Is there a system limit on these resources? Is it
possible to get a list of current regions? Is it possible to clean
them out without rebooting (yes if we can get the list of names, but
otherwise...)? I could see some havoc being caused by a program which
leaks regions, using random or unpredictable names, which can then
never be reclaimed. Surely that's not the case? (say it ain't so!)
It seems to me that the man page makes it clear that the shared
memory will persist even when all processes using it have terminated.
I figure that this means that the swap space will just go on getting
bigger
as more of these segments are left around. The man page mentions
that the segments don't persist across a reboot though so there is a
way out. I agree that it would be useful to see what exists to
then have an opportunity to tidy up, but that doesn't appear to be
possible. Hence why I want to have my region automatically removed
when all processes detach from it (even if they crash) by calling
shm_unlink as soon as the region is created and relying on inheritance
of the descriptor (rather than the name of the region) to have the
subprocesses attach.
[large chunk snipped]
Regards,
Tim.
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Darwin-kernel mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden