site_archiver@lists.apple.com Delivered-To: darwin-kernel@lists.apple.com That is because of me. Consider fixing your code. Here's the whole story for the default case of "not a custom stack": HOWEVER: On smaller stacks instead of larger: -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel@lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.a... On Mar 2, 2008, at 1:15 PM, Ken Mankoff wrote: On Sun, 2 Mar 2008, Terry Lambert wrote: On Mar 1, 2008, at 1:57 PM, Ken Mankoff <mankoff@gmail.com> wrote: On Sat, 1 Mar 2008, Michael Smith wrote: On Mar 1, 2008, at 11:20 AM, Ken Mankoff wrote: I'm running out of stack space with an application I need to run. I have "ulimit -s 64000" in my .bash_profile. The highest value I can set is 65532. At 65533 it reports, "/usr/bin/ulimit: line 4: ulimit: stack size: cannot modify limit: Invalid argument" I read somewhere that the kernel limits it to 64MB. I figure this is the right place to get a definitive answer and further information. Is it 64 really the limit? How would I allow multiple processes to each have 128 or 256MB each? I'm comfortable recompiling a kernel if need be. Thanks for any advice, First suggestion; fix the app so that it uses less than 64MiB of stack. Hard, but realistically that much stack usage points to a design/algorithmic flaw (Resource consumption that scales 1:1 with some aspect of your dataset directly constrains the dataset that you can manage vs. the availability of the resource in question.) I can't do this. This is 100 000 + lines that is a global climate model. Second suggestion: you have hit the hard limit; you need to be root to increase it. You can either run the shell as root, or write a setuid root wrapper for your application that uses setrlimit(2) to bump both the hard and soft limits up before starting the application. I tried the 'ulimit' command w/ sudo. Won't go above 65532. Even with OK thank you all for the feedback. We haven't actually run into a 64MB limit but are concerned we might. It sounds unlikely based on the Oracle comments, but I think I have a pretty good idea of how to approach it should we require more. The issue is the imposition of resource limits, and where our stacks get allocated. Unlike other UNIX systems, you don't start out with them growing down towards the heap. As Shantonu noted, you can ask for a larger stack segment to be allocated for you at link time; but there will be serious limitation on what you can do after that. For a "custom stack", the defaults are what you tell it, and there is no guard page, so unless you allocate one of those as well, then you can fall off the end (preferably also at link time). Basically, we have to preallocate the space because it could get stomped, and we have to stomp part of it so that if you run off the end you get a warning, and then how much of that we stomp is based on the default soft limits set via setrlimit(); this is required for POSIX conformance, but the only meaning that has is that the default has to be less than the maximum possible: it doesn't effect where the maximum sits. Unfortunately, the default stack size due to administrative limits means we take whatever number that was, and thunk a guard page in at the default offset. You effectively have the address space mapping for the full 64MB, since things could have been allocated into the guard space because of where the stack is initially located. Our memory layout is not like other UNIX systems. We have things that allocate memory in the middle of the address space, and at the end, and the video drivers allocate frame buffer, texture, and other memory in the middle as well. Further, the programs environment and argument list is preallocated out of the available stack space, and the stack starts at an offset after that (this is fairly well documented in kern_exec.c in the xnu sources, which describes the stack layout in detail). But it means that if you have a huge environment, argument list, or combination, you will have less stack. If you ask for a custom stack, and the you, or code you call, *ever* modify the stack resource limits via setrlimit(), you will minimally get a guard page plopped at the end of the 64MB, maximally at the smaller default, and you will be stuck with a maximum 64MB stack thereafter. This limit is set at compile to to MAXSSIZ - PAGE_SIZE ... 64MB - 1 page. The resource limits code doesn't really know anything about the limits you may or may not impose. Clearly, from this, you should also be very careful setting a custom stack size of under 64MB, if you choose to set one, unless you also set your guard page and never touch setrlimit() for the stack size there. If you pick one that is too small, even if you do the right thing with guard pages and so on (my recommendation would be to add a one pages section to the binary after the stack section, and map it no read/write, in fact), you also risk running out of stack space from a larger environment or argument list. This email sent to site_archiver@lists.apple.com