site_archiver@lists.apple.com Delivered-To: darwin-kernel@lists.apple.com Ethan, On 21 Feb 2005, at 06:21, Ethan Tira-Thompson wrote: When your processes attach to the shared region how do they know the size? Is it an application-wide constant? The call to mmap has to have the size (since you can choose not to map all the region). I've found that fstat() gets it fine on an inherited shared memory descriptor from the parent. It is beginning to look like my approach with shm_unlink is going to be a problem. My circumstance is that a controlling process creates the shared segment and spawns short-lived children which use the shared segment. Those children ought to be able to use the inherited file descriptor that the parent received from shm_open() when creating the region. They ought not to have to call shm_open() themselves to attach to the region, and so the region ought not to have to exist in its name space (is there a "shm_ls" feature so you can see what's in the name space??). This should mean that the process creating the region can unlink it immediately and just use common inheritance for subprocesses to attach to it. The shm_open man page certainly suggests that this should be possible, i.e. "The new descriptor is set to remain open across execve system calls" (though it directly contradicts itself earlier stating "The new file descriptor will have the FD_CLOEXEC flag set") but this suggests inheritability of the file descriptor and it would be natural for that to happen anyway. In my case the file descriptor has the FD_CLOEXEC flag set on it explicitly immediately after its creation, so that it isn't inherited across exec unnecessarily, however for the children that use the shared region this descriptor is dup'd, which implicitly causes this flag to be cleared. The descriptor is definitely being inherited across exec anyway since the call to fstat() on it is successful and returns expected results. It is the mmap() which is the problem returning EINVAL, and it seems to me that this is not what should happen. [large chunk snipped] Regards, Tim. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.a... Thanks to everyone for their insight... I've gotten things working, but there may be more here for you to work on amongst yourselves (assuming there's a few kernel hackers on here...) Actually, I think documentation or a code sample was the real shortcoming, and that's not limited to Darwin (I did extensive web searching before winding up here) So, ftruncate does seem to be key in the creation of a shared memory region - after shm_open, before mmap. However, it should only be called on a *new* region at its creation... it will return an error if you're connecting to a pre-existing region. That seems reasonable enough, but I found this aspect to be rather undocumented. (I suppose the size/len argument of mmap is supposed to be used to map a subsection of a file into memory, so you have to use ftruncate to pre-size the shared memory region, and then can selectively map portions of the region into the process. I initially expected mmap to do the region sizing itself...) The next step is shm_unlink. It seems it should *not* be called right after the creation in order to request dereference down the road (i.e. shm_open, ftruncate, mmap, close, shm_unlink) shm_unlink will remove the name immediately, thus blocking other processes from accessing it since it's on the way out. However, I suppose it's fine to call immediately following the last attach you are expecting, to guarantee that a sudden crash will cause it to be removed. The safest general behavior (if you don't know when an attach is the last one) seems to be to wait until the last reference is being removed, and then call shm_unlink. This is directly analogous to using shmctl to delete a region under SysV. (initially I had called it on *every* dereference, so I'm not completely sure what happens if it's not called for the last dereference, or if I the errors I got were for the first call or the last call (i.e. did the first call succeed and following ones fail, or did all fail except the last?)) This brings up an interesting problem however. If you shm_open a bunch of regions, and never unlink them, where do they go when the process gets shutdown? My experience from the intial email implies they stick around (blocking further attempts to make new regions of the same name). Is there a system limit on these resources? Is it possible to get a list of current regions? Is it possible to clean them out without rebooting (yes if we can get the list of names, but otherwise...)? I could see some havoc being caused by a program which leaks regions, using random or unpredictable names, which can then never be reclaimed. Surely that's not the case? (say it ain't so!) It seems to me that the man page makes it clear that the shared memory will persist even when all processes using it have terminated. I figure that this means that the swap space will just go on getting bigger as more of these segments are left around. The man page mentions that the segments don't persist across a reboot though so there is a way out. I agree that it would be useful to see what exists to then have an opportunity to tidy up, but that doesn't appear to be possible. Hence why I want to have my region automatically removed when all processes detach from it (even if they crash) by calling shm_unlink as soon as the region is created and relying on inheritance of the descriptor (rather than the name of the region) to have the subprocesses attach. This email sent to site_archiver@lists.apple.com