site_archiver@lists.apple.com Delivered-To: darwin-dev@lists.apple.com -- Terry Here is the line in /etc/rc I changed: I am on this version of Darwin: Hope this was helpful. Norm Green GemStone Systems Inc. http://www.gemstone.com/smalltalk/ -----Original Message----- From: Jay Reynolds Freeman [mailto:jay_reynolds_freeman@mac.com] Sent: Friday, July 13, 2007 1:03 PM To: darwin-dev@lists.apple.com list Subject: need lots of shared memory (with shmget) in a Cocoa app This topic overlaps Cocoa and Darwin, bear with me a minute: I would like to make some serious use of interprocess shared memory in Cocoa applications. I don't mean pipelines or their equivalents; my project involves having multiple different applications all making asynchronous reads and writes to different parts of a large (Gigabyte) area of shared memory. (Details later, for the curious.) I would be using shmget, shmat, and so on, as low-level primitives to obtain and access shared memory. I have figured out that OS X doesn't support that much shared memory right out of the box. (BTW,I am running 10.4.10 on a 13-inch Intel Macbook with 1 GByte of physical memory.) I have done enough research to know that some of the parameters that control shared memory are in /etc/rc, which is read at boot-time, and I even managed to find some documentation on the web for what kern.sysv.shm<whatever> means. (The documentation I found was in Spanish, which I don't read or speak, but fortunately the Google translation was usable -- the search experience was sufficiently Monty-Pythonesque that I found myself lifting my feet in case of killer rabbits.) And I found a lot of comments from people who had tried to get shared memory to work on the Mac, and had not been as successful as they had hoped. I hope that is enough homework to warrant an appeal to this group for further pointers. Specifically, 1) Is there any thorough documentation anywhere about how shared memory on the Mac actually works? I need all the background information I can get, and I certainly need to know whether there are hard or soft range limits for the kern.sysv.<whatever> parameters, and what happens if you crowd those limits. Just man pages aren't near enough, and googling and searching XCode documentation hasn't turned up anything interesting yet. 2) What am I likely to break elsewhere in the Mac architecture if I start increasing shared-memory limits? 3) How do the layers of software architecture upon which a Cocoa application runs interact with the shared memory stuff? I haven't tried messing with /etc/rc yet, but in my application, I could not successfully create even a 1 MByte shared-memory segment. I suspect that something else is using enough of the default shared-memory resources that there weren't enough left for my test, and I need to know how much to allow for any such uses. (If this question should go to another group, tell me and I will take it there.) If you are still reading and are curious, what I have is a functioning Lisp system (Wraith Scheme -- see my web site if you are terminally curious (URL below)); I think I have a way to add an access-rights system to it such that multiple copies of the Lisp interpreter can interact with the same heap without stepping on each others' toes. The heap is the big shared memory object I spoke of. (Lisps eat memory as if there were no tomorrow, 1 GByte is not a lot...) Using just separate threads, one for each Lisp interpreter, isn't appropriate, because each interpreter has substantial amounts of truly private data. Just fork (without exec) won't work on a Mac app, at least, not if you want both parent and child to have a functioning GUI. Small numbers of cooperating parallel Lisp processes would be fun, and would be a natural fit to present and likely near-future Mac processor hardware. Enough blathering. Thanks for any hints or suggestions you may have. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/norm.green%40gemstone.com This email sent to norm.green@gemstone.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/tlambert%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.appl... You are significantly better off using memory sharing via mmap or via Mach primitives than you would be using System V shared memory. The reasoning behind the default imitations have to do with it taking wired memory, which doesn't really happen for mmap or by passing around address spaces between processes using Mach primitives. This also has a number of other benefits, depending on the approach you choose to use: (1) If you use mmap, the data is backed by a file, and will persist over crashes or reboots, which would make it easier to pick up where you left off, if your application supports it; this approach would be ideal for, for example, a ray tracing or other restartable computation model (would require an intention log to know where you are at in the ob). (2) If you use Mach primitives, the size limitation can be overcome by passing region descriptors between 32 and 64 bit applications (wffectively, your 32 bit applications could have a 512M-1G window on a much larger virtual region maintaind by a 64 bit app, as one example). (3) Resource tracking cleanup in either of these approaches (i.e. what you have to do to recover from a crash) is much easier than in System V or POSIX shared memory, since the objects persist past the lifetime of your programs; with SysV or POSIX, you either have to know the id (e.g. use ftok()) and release the memory every time you start up, ignoring the error if it isn't there already). (4) You don't eat wired memory or kernel virtual address space; for a 1G shared memory object, this might actually mean using SysV/POSIX ends up being impossible (in order to persist as a rendezvous, the SysV and POSIX objects pretty much have to end up allocated in a persistent address space that can be mapped out into user processes; hence this limitation). The downside of mmap is that pages will get written to the backing store file under memory pressure; if you don't need the data to be persistent across reboot/program crash, then this ends up costing disk I/O;. On the other hand, given the size of the regions being contemplated here, it's either push it to a backing file or push it to swap: either way you're going to be hitting the disk, unless you have a huge amount of physical RAM, so this might not be an issue for you (enough RAM = no memory pressure = no disk I/O for dirty pages). Note that the above is pretty much true for most OSs (but most OSs don't support Mach primitives for sharing). My personal recommendation is to use mmap; if you want to use the Mach approach, then you will likely want to buy Amit Sing's book to get the documentation necessary for you to be able to write the code. On Jul 13, 2007, at 1:13 PM, Norm Green wrote: Jay, I recently ported our Smalltalk virtual machine and application server to Darwin. Like LISP, Smalltalk uses substantial amounts of heap and shared memory. I had no problems getting 10 MB of shared memory after I increased the default limits in /etc/rc and rebooted. sysctl -w kern.sysv.shmmax=104857600 kern.sysv.shmmin=1 kern.sysv.shmmni=32 kern.sysv.shmseg=8 kern.sysv.shmall=104857600 Darwin Kernel Version 8.9.3: Fri Apr 27 14:50:07 PDT 2007; root:xnu-792.19.5~2/RELEASE_I386 I didn't locate any docs on the subject, I just made the changes above and it worked. -- Jay Reynolds Freeman --------------------- Jay_Reynolds_Freeman@mac.com http://web.mac.com/jay_reynolds_freeman (personal web site) This email sent to tlambert@apple.com This email sent to site_archiver@lists.apple.com