February 2003 - Darwin-kernel

Re: NPROC / MAXUPRC
by Justin Walker 28 Feb '03

28 Feb '03

On Friday, Feb 28, 2003, at 07:35 US/Pacific, Pelle Johansson wrote: fredagen den 28 februari 2003 kl 07.42 skrev Justin Walker: On Thursday, Feb 27, 2003, at 20:19 US/Pacific, Douglas Stetner wrote: So is there an apple bug report open on this? I don't believe there is, but regardless, I don't think that there's a bug to report. It's been discussed here, as I recall, and the mechanism described. I am currently running at about 93 processes and do not look forward to having to build a new kernel to keep myself going. If you had to, that would be a bug, but your shell gives you the means to change this (at least for most available shells). Check your shell's man page for 'ulimit' (or 'limit'). Actually, there does seem to be a bug. limit -h reports maxproc as unlimited, but as soon I try to set the soft limit, the hard limit is set to 100. If I su to root, I can set the hard limit as high as 532. When I then su back to the user, limit -h reports 532, but when I try to set the soft limit, the hard limit is again lowered to 100. It seems that as soon as a normal user calls setrlimit, the hard limit is lowered to 100. Shouldn't I be able to set the hard limit as long as it is lower than the current one, regardless of the default? At least that's what setrlimit(2) seems to say. You're correct. The code permits a root process to 'up' the per-user process count beyond the system-imposed maximum. A 'normal' process is limited to CHILD_MAX==100 processes, period. That limit is hard-coded. One can debate whether this limit is reasonable. It's worth filing a bug report against the kernel to this effect, to start the debate :-}. I tried it with both bash and tcsh, so it doesn't seem to be a shell bug. Definitely independent of the shell you use. It's the way the kernel works. I should have looked at the code before commenting. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | Some people have a got a mental | horizon of radius zero, and call | it their point of view | -- David Hilbert *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: NPROC / MAXUPRC
by Pelle Johansson 28 Feb '03

28 Feb '03

fredagen den 28 februari 2003 kl 07.42 skrev Justin Walker: On Thursday, Feb 27, 2003, at 20:19 US/Pacific, Douglas Stetner wrote: So is there an apple bug report open on this? I don't believe there is, but regardless, I don't think that there's a bug to report. It's been discussed here, as I recall, and the mechanism described. I am currently running at about 93 processes and do not look forward to having to build a new kernel to keep myself going. If you had to, that would be a bug, but your shell gives you the means to change this (at least for most available shells). Check your shell's man page for 'ulimit' (or 'limit'). Actually, there does seem to be a bug. limit -h reports maxproc as unlimited, but as soon I try to set the soft limit, the hard limit is set to 100. If I su to root, I can set the hard limit as high as 532. When I then su back to the user, limit -h reports 532, but when I try to set the soft limit, the hard limit is again lowered to 100. It seems that as soon as a normal user calls setrlimit, the hard limit is lowered to 100. Shouldn't I be able to set the hard limit as long as it is lower than the current one, regardless of the default? At least that's what setrlimit(2) seems to say. I tried it with both bash and tcsh, so it doesn't seem to be a shell bug. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: NPROC / MAXUPRC
by Brian Tabone 28 Feb '03

28 Feb '03

Douglas, One trick you could try is to set you kernel max processes high, then create a group of users that will then run your application. Assuming you can spread the work out, you could run your application under each user in the group, so that in effect you get (Nusers * userlimit) process slots available to you. It's a bit kludgy, but it might get you past your problem until you can cook up a custom kernel. I would imagine though, that there are scheduler efficiency reasons as to why apple set the limit at 100 and you may find a precipitous drop off of performance above that level. That's just my guess though -Brian T On Thursday, February 27, 2003, at 10:19 PM, Douglas Stetner wrote: So is there an apple bug report open on this? I am currently running at about 93 processes and do not look forward to having to build a new kernel to keep myself going. This needs to be tuneable in some fashion (I don't care if I need to reboot), but to have no option other than rebuilding my kernel (and from what I have seen on this list, correct me if I am wrong, I cannot even get up to date source out of the cvs repository!). If nothing else, Apple should do a build themselves with maxuproc at say 500, and provide that kernel as a download somewhere... Doug -- Douglas Stetner <dstetner(a)bigpond.net.au> "If we knew what it was we were doing, it would not be called research, would it?" - Albert Einstein _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: NPROC / MAXUPRC
by Justin Walker 28 Feb '03

28 Feb '03

On Thursday, Feb 27, 2003, at 20:19 US/Pacific, Douglas Stetner wrote: So is there an apple bug report open on this? I don't believe there is, but regardless, I don't think that there's a bug to report. It's been discussed here, as I recall, and the mechanism described. I am currently running at about 93 processes and do not look forward to having to build a new kernel to keep myself going. If you had to, that would be a bug, but your shell gives you the means to change this (at least for most available shells). Check your shell's man page for 'ulimit' (or 'limit'). Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | When LuteFisk is outlawed | Only outlaws will have | LuteFisk *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: NPROC / MAXUPRC
by Douglas Stetner 28 Feb '03

28 Feb '03

So is there an apple bug report open on this? I am currently running at about 93 processes and do not look forward to having to build a new kernel to keep myself going. This needs to be tuneable in some fashion (I don't care if I need to reboot), but to have no option other than rebuilding my kernel (and from what I have seen on this list, correct me if I am wrong, I cannot even get up to date source out of the cvs repository!). If nothing else, Apple should do a build themselves with maxuproc at say 500, and provide that kernel as a download somewhere... Doug -- Douglas Stetner <dstetner(a)bigpond.net.au> "If we knew what it was we were doing, it would not be called research, would it?" - Albert Einstein _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: NPROC / MAXUPRC (Cameron Knowlton)
by Justin Walker 28 Feb '03

28 Feb '03

On Thursday, Feb 27, 2003, at 14:53 US/Pacific, Peter Dyballa wrote: Hello! I remember that in old BSD based SunOS 4.x there was a way to set kernel variables at runtime using adb. I do remember that once I had to enlarge something like NPROC, the system-wide no. of processes. In Mac OS X we have sysctl, IMHO a modern and easy to use tool to achieve the same. And: "sysctl -a" shows: kern.maxproc = 532 After reading the man page ("man 8 sysctl") I tried: pete 156 /\ sysctl -w kern.maxproc=999 kern.maxproc: Operation not permitted pete 157 /\ sudo sysctl -w kern.maxproc=999 Password: kern.maxproc: 532 -> 999 pete 158 /\ sysctl -a | grep proc kern.maxproc = 999 pete 159 /\ But it does succeed to raise the shell's maxproc value above 100! Do I have to reboot? Logging out and in did not change anything ... You apparently missed this minor point in the 'sysctl' man page: To set the maximum number of processes allowed in the system to 1000, one would use the follow request: sysctl -w kern.maxproc=1000 Note that 'maxproc' does not affect the maximum per-user process count. Check back in the archives for further discussion in this thread, and the answer to your question. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | When LuteFisk is outlawed | Only outlaws will have | LuteFisk *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: NPROC / MAXUPRC (Cameron Knowlton)
by Peter Dyballa 27 Feb '03

27 Feb '03

Hello! I remember that in old BSD based SunOS 4.x there was a way to set kernel variables at runtime using adb. I do remember that once I had to enlarge something like NPROC, the system-wide no. of processes. In Mac OS X we have sysctl, IMHO a modern and easy to use tool to achieve the same. And: "sysctl -a" shows: kern.maxproc = 532 After reading the man page ("man 8 sysctl") I tried: pete 156 /\ sysctl -w kern.maxproc=999 kern.maxproc: Operation not permitted pete 157 /\ sudo sysctl -w kern.maxproc=999 Password: kern.maxproc: 532 -> 999 pete 158 /\ sysctl -a | grep proc kern.maxproc = 999 pete 159 /\ But it does succeed to raise the shell's maxproc value above 100! Do I have to reboot? Logging out and in did not change anything ... -- Greetings Pete _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Greets and VFS question
by Quinn 27 Feb '03

27 Feb '03

At 12:42 +0000 27/2/03, Pev wrote: Anyone have any suggestions? I fear I may be missing something obvious! Alas, you're not. This is quite a commonly asked question, and I will turn the following into a DTS Q&A. Note that this hasn't been reviewed yet, so right now it's not Apple gospel but *draft* Apple gospel (-: ---------------------------------------------------------------- Q: Does Apple support the development of external file systems (VFS plug-ins) on Mac OS X? Where can I find information about developing such a file system? A: The answer depends on whether you're writing a leaf file system or a stacking file system. o Leaf File Systems -- Apple does support the development of leaf file systems for Mac OS X. Unfortunately, there is no documentation that specifically covers Mac OS X's VFS plug-in architecture. However, the following resources should be helpful. - Mac OS X's VFS architecture is derived from 4.4BSD. "The Design and Implementation of the 4.4BSD Operation System" (ISBN: 0-201-54979-4) includes a good overview of VFS on that system. As there are key differences between 4.4BSD and Mac OS X (for example, the Unified Buffer Cache and additional VFS entry points to support HFS semantics), you can not consider this book canonical. - For general information about Mac OS X kernel programming, see the documentation on the Apple developer web site. <http://developer.apple.com/techpubs/macosx/Darwin/index.html> - The source for a number of existing file systems is available in Darwin (the open source underpinnings of Mac OS X). <http://www.apple.com/darwin/> - There are a few DTS Q&As that cover VFS. <http://developer.apple.com/qa/indexes/fl-a.html> - If you have technical questions, there are a variety of mailing lists for free support and DTS for paid support. <http://www.lists.apple.com/darwin-kernel> <http://developer.apple.com/programs/techsupport.html> IMPORTANT: As with all development in the BSD portions of the Mac OS X kernel, binary compatibility is not guaranteed for VFS plug-ins. We probably won't document the VFS layer properly until we have the binary compatibility story sorted out. o Stacking File Systems -- Apple does not support the development of stacking VFS plug-ins on Mac OS X. We've taken this position because, in our opinion, it's not possible to create a stacking VFS plug-in that: a) works reliably, b) does not severely impinge on system performance, and c) has any hope of binary (or even source-level) compatibility with future systems. Earlier Apple documentation stated that stacking VFS plug-ins could be used to solve certain problems, such as file-level encryption and virus checking. We even went so far as to publish an example of this, the Politically Correct file system (PCFS), that ran on pre-release builds of Mac OS X. Subsequent experience has shown that this technique does not work properly and we no longer support it. Thus, we have removed (or soon will) all references to stacking VFS from our documentation <rdar://problem/3143262> <rdar://problem/3143919> and we will not publish an update to PCFS. The existing PCFS source code is not useful because it has not been updated to support UBC. If you have a problem that can only be solved via a stacking file system, please let us know by mailing <dts(a)apple.com>. While we can't support your development, we are interested in gathering information about what developers need and why. This feedback will guide our long-term approach to this question. Finally, there are a number of third party developers who are working in this realm. If you are absolutely determined to develop a stacking VFS plug-in, you will find some help on the Internet. ---------------------------------------------------------------- S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Greets and VFS question
by Pev 27 Feb '03

27 Feb '03

Hi, Firstly can I just say hello ("Hello!") to everyone as I've just joined the list. I have a question about filesystem plugins. I'm considering writing a filesystem support plugin for a relatively obscure CD-ROM filesystem format, but I don't seem to be able to find much documentation about them. I've read the few pages of overview in the Kernel Programming pdf book, and scoured the apple and ADC websites as well as google, but the only things I can come up with are the msdosfs and cddafs source code tarballs from the darwin archives. Is this really all there is as its a bit daunting to try and figure out the interface by reverseing the example code. Anyone have any suggestions? I fear I may be missing something obvious! Best Wishes, Dave Peverley Everything you'll ever need on one web page from News and Sport to Email and Music Charts http://uk.my.yahoo.com _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Maximum number of user processes
by Peter Dyballa 26 Feb '03

26 Feb '03

The easiest way is to check using the getrlimit() system call along with RLIMIT_NPROC, or via the "ulimit -u" built-in in bash or csh. "ulimit -u" in bash, sh, or zsh and "limit | grep proc" in csh, tcsh, and zsh too should easily show up the success of the kernel build. But pay attention to your shell setup files: they can change the kernel set limits (to reduced values)! -- Greetings Pete _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0