November 2003 - Darwin-kernel

ulimit settings
by Brian Tabone 17 Nov '03

17 Nov '03

Joey, I was re-reading your message and noticed you had already tried maxprocperuid, my bad. However, What does bash> ulimit -a Say on your machine, on mine, even though ulimit says unlimited, ulimit -a tells me ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) 6144 file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 100 virtual memory (kbytes, -v) 14336 I think you have to explicitly bump up your ulimit in your shell -Brian T On Nov 16, 2003, at 8:32 PM, Joey Stanford wrote: Howdy! I'm out of options, hoping you guys (and girls) might be able to clue me in. Running 10.3 will all the latest updates on a 2ghz, 2gig Dual G5. Using tcsh but bash has same problem. I keep running out of "resources" whatever that is when I hit about 135 pids (and I have over 100gig of disk left and about 500meg of unused memory when this happens). I get the standard "unable to fork" errors. e.g. 2003-11-14 10:56:20.479 Konfabulator[6491] Exception raised during posting of notification. Ignored. exception: couldn't fork 2003-11-14 16:18:14.697 Mail[1770] Exception raised during posting of notification. Ignored. exception: couldn't fork In term I get just "unable to fork" I bumped up my sysctl using /etc/sysctl.conf: kern.maxproc=2048 kern.maxprocperuid=512 and then unlimited in the shell: [joey@futune]:/Applications/entropy/src> limit cputime unlimited filesize unlimited datasize unlimited stacksize 65536 kbytes coredumpsize unlimited memoryuse unlimited descriptors 10240 memorylocked unlimited maxproc 512 [joey@futune]:/Applications/entropy/src> lsof | wc -l 394 kern.maxfiles = 12288 kern.maxfilesperproc = 10240 Help? Any ideas? Thanks! ---------------------- http://stan4d.net http://entropy.stop1984.com [demime 0.98b removed an attachment of type application/pgp-signature which had a name of PGP.sig] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Increase your uid max proc
by Brian Tabone 17 Nov '03

17 Nov '03

Joey, You need to increase your max proc per uid (do this using systcl). Here is how to do it: bash> sysctl kern.maxprocperuid kern.maxprocperuid = 100 bash> sudo sysctl -w kern.maxprocperuid=200 kern.maxprocperuid: 100 -> 200 Hope that helps, -Brian T On Nov 16, 2003, at 8:32 PM, Joey Stanford wrote: Howdy! I'm out of options, hoping you guys (and girls) might be able to clue me in. Running 10.3 will all the latest updates on a 2ghz, 2gig Dual G5. Using tcsh but bash has same problem. I keep running out of "resources" whatever that is when I hit about 135 pids (and I have over 100gig of disk left and about 500meg of unused memory when this happens). I get the standard "unable to fork" errors. e.g. 2003-11-14 10:56:20.479 Konfabulator[6491] Exception raised during posting of notification. Ignored. exception: couldn't fork 2003-11-14 16:18:14.697 Mail[1770] Exception raised during posting of notification. Ignored. exception: couldn't fork In term I get just "unable to fork" I bumped up my sysctl using /etc/sysctl.conf: kern.maxproc=2048 kern.maxprocperuid=512 and then unlimited in the shell: [joey@futune]:/Applications/entropy/src> limit cputime unlimited filesize unlimited datasize unlimited stacksize 65536 kbytes coredumpsize unlimited memoryuse unlimited descriptors 10240 memorylocked unlimited maxproc 512 [joey@futune]:/Applications/entropy/src> lsof | wc -l 394 kern.maxfiles = 12288 kern.maxfilesperproc = 10240 Help? Any ideas? Thanks! ---------------------- http://stan4d.net http://entropy.stop1984.com [demime 0.98b removed an attachment of type application/pgp-signature which had a name of PGP.sig] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

unable to fork/no resources
by Joey Stanford 17 Nov '03

17 Nov '03

Howdy! I'm out of options, hoping you guys (and girls) might be able to clue me in. Running 10.3 will all the latest updates on a 2ghz, 2gig Dual G5. Using tcsh but bash has same problem. I keep running out of "resources" whatever that is when I hit about 135 pids (and I have over 100gig of disk left and about 500meg of unused memory when this happens). I get the standard "unable to fork" errors. e.g. 2003-11-14 10:56:20.479 Konfabulator[6491] Exception raised during posting of notification. Ignored. exception: couldn't fork 2003-11-14 16:18:14.697 Mail[1770] Exception raised during posting of notification. Ignored. exception: couldn't fork In term I get just "unable to fork" I bumped up my sysctl using /etc/sysctl.conf: kern.maxproc=2048 kern.maxprocperuid=512 and then unlimited in the shell: [joey@futune]:/Applications/entropy/src> limit cputime unlimited filesize unlimited datasize unlimited stacksize 65536 kbytes coredumpsize unlimited memoryuse unlimited descriptors 10240 memorylocked unlimited maxproc 512 [joey@futune]:/Applications/entropy/src> lsof | wc -l 394 kern.maxfiles = 12288 kern.maxfilesperproc = 10240 Help? Any ideas? Thanks! ---------------------- http://stan4d.net http://entropy.stop1984.com [demime 0.98b removed an attachment of type application/pgp-signature which had a name of PGP.sig] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Kernel Extensions
by Justin Walker 16 Nov '03

16 Nov '03

On Friday, November 14, 2003, at 05:21 PM, George Andre wrote: Mike Vannorsdel wrote: You'll have to use the VFS functions (vn_open, vn_rdwr, vn_close, ect). This also requires a good grasp on vnodes, vnode reference counting, and vnode locking. You'll find doing file I/O from within the kernel is very different, and more difficult, than from userland. Your best bet is to find a book or online tutorials on using VFS. Just a word of warning... Even though VN ops function will work, Apple is discouraging such an endeavour, since it will rely on the kernel exporting these specific symbols. As far as I know (someone from Apple might want to correct me if I'm wrong) relying on kernel symbols presence at kext link is a no-no. Plus, if I remember it correctly, to make it work a funnel change must be done; plus a possible break of synchronization. Having said that, the specific kernel symbols we are talking about have 99.9% chance/risk of not being removed/changed... so it's your choice. A word of warning about the word of warning :-} None of the symbols in the kernel can be relied on, unless Apple has committed to supporting their presence and meaning. While most of them are unlikely to change (there's no percentage in making arbitrary changes when you are intending to import updates from the outside), it is possible that they will be made unavailable, in favor of more supportable "accessor functions" or some similar scheme. The odds-makers in Vegas and London have yet to publish their take on this event. Cheers, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | "Weaseling out of things is what | separates us from the animals. | Well, except the weasel." | - Homer J Simpson *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Kernel Extensions
by Matt Jaffa 15 Nov '03

15 Nov '03

Thanks a bunch, it seems i missed this part of the documentation. I'll look into this Matt On Nov 15, 2003, at 4:02 PM, Justin Walker wrote: file:///Developer/Documentation/Darwin/General/KernelProgramming _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Kernel Extensions
by Justin Walker 15 Nov '03

15 Nov '03

On Saturday, November 15, 2003, at 02:34 PM, Matt Jaffa wrote: Marek Kozubal, How do I write a user land daemon that the Kernel KEXT can actually call, and how do I call it from within the Kernel extension? One of the biggest questions I have is, Is there better documentation on how to do everything? The online documentation, just teaches you how to compile, non of the specifics on the capabilities of everything, Is there a book out there that i can buy that will help alot? Between the content on Apple's developer site (http://developer.apple.com) and that in the Developer documentation installed with Developer Tools on your system, you should have more than enough to get you well into developing kernel extensions. Have you looked at this (e.g., file:///Developer/Documentation/Darwin/General/KernelProgramming)? In addition, Apple's Darwin site has a number of example kernel extensions to take a look at. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | If you're not confused, | You're not paying attention *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Can a task on each of 2 processors share 7 GB of physical m emory
by Jim Magee 15 Nov '03

15 Nov '03

You would mmap() parts of the file into N different address spaces and wire them down there to get them all in memory. Then use mmap() in the two process to map the parts you need to access at any given time. However, the PCI-X card comment concerns me. Quinn previously asked if you were wiring because you wanted to do I/O into/out-of the memory and you said no. But here you talk about I/O. On the current G5 systems, there is a limit on the amount of memory that can be prepared for I/O at any given time (so that mappings can be done from the PCI bus address space to main system memory address). So, you will NOT be able to access that memory from PCI-X without doing something special for each I/O. --Jim On Nov 14, 2003, at 8:21 PM, Miller, Larry wrote: It is not clear to me that mmap(2) can let a process get a pointer to a block of physical memory which has been previously wired and contains data that has been loaded oby a PCI-X card. It would be good if after suitable initialization, a file descriptor could refer to say a 7 GB file that is mapped entirely into pages in physical memory. The PCI-X card could load the data into those pages. And then a process on each of the two processord could access whatever part of it they wanted using mmap. Is this possible??? Larry -----Original Message----- From: Jim Magee [mailto:jmagee@apple.com] Sent: Friday, November 14, 2003 4:17 PM To: Miller, Larry Cc: '''darwin-kernel(a)lists.apple.com' ' ' Subject: Re: Can a task on each of 2 processors share 7 GB of physical m emory mmap(2)? On Nov 13, 2003, at 11:22 PM, Miller, Larry wrote: Jim, What would be an efficient remapping technique so a given task can quickly get a window into a desired part of the physical memory? Larry -----Original Message----- From: Jim Magee To: Miller, Larry Cc: ''darwin-kernel(a)lists.apple.com' ' Sent: 11/13/2003 5:37 PM Subject: Re: Can a task on each of 2 processors share 7 GB of physical m emory On Nov 13, 2003, at 11:58 AM, Miller, Larry wrote: By wired, I mean the data that I put in the memory remains at a fixed place in physical memory - it is not paged out. This would probably be equivalent to a fixed mapping from VM to physical memory. Then you probably could simultaneously wire 7GB out of 8GB physical as you request. I heard that OS-X Panther was in some ways a 64-bit OS. In other words, I thought tasks could execute 64-bit addressing load and store machine instructions. Maybe I am mistaken. Also, I maybe mistakenly thought that you could compile a C/C++ program in a way such that you could have 64-bit pointers that are dereferenced. There are lots of things that define a 64-bit support. Using 64-bit addresses to access >4GB of mapped virtual memory in a single task is only one (maybe two - depends upon how you count) such feature (which happens to be absent in Panther/Darwin 7.x). The ability to access 4GB of physical of physical memory in the first place is also one. An additional feature is the ability to access and use 64-bit wide registers in critical functions (and have the kernel preserve that state correctly on interrupts/signals, etc...). These last two ARE in Darwin 7.x. If these things are not available, does anyone one have an idea of when they will become available in the future? As always, we can't make any statements about future Mac OS X plans here or elsewhere. But from a technology point of view, it is down-right inevitable that 64-bit address space support will make it into Darwin - eventually. In the meantime, using multiple address spaces to wire the memory down in, and (re)mapping techniques to gain access to some percentage of the memory at any given time, should work. For most applications, this should be acceptable (performance-wise). It's the cost of getting stuff in and out of physical memory that is the real bottleneck. At first reading, this seems to fit well with the "up to 256MB chunk" model of memory allocation you suggested earlier. --Jim _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Kernel Extensions
by Matt Jaffa 15 Nov '03

15 Nov '03

Marek Kozubal, How do I write a user land daemon that the Kernel KEXT can actually call, and how do I call it from within the Kernel extension? One of the biggest questions I have is, Is there better documentation on how to do everything? The online documentation, just teaches you how to compile, non of the specifics on the capabilities of everything, Is there a book out there that i can buy that will help alot? Thanks, Matt On Nov 14, 2003, at 10:41 PM, Marek Kozubal wrote: On Fri, 14 Nov 2003, Mike Vannorsdel wrote: You'll have to use the VFS functions (vn_open, vn_rdwr, vn_close, ect). This also requires a good grasp on vnodes, vnode reference counting, and vnode locking. You'll find doing file I/O from within the kernel is very different, and more difficult, than from userland. Your best bet is to find a book or online tutorials on using VFS. The proper way to do this is to communicate with a user land daemon to do the file writing ala syslog. You might even be able to configure syslog to write your messages into a differnent file. But Apple's recommended way is to have something in userland do your file IO and use IOKit messages/communication if in the IOKit, use sockets if in a file system (which requires you to switch from BSD to network funnel to do I believe), etc. Quinn will have good insight into this. :) --- Marek Kozubal marek(a)portents.com _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Kernel Extensions
by Marek Kozubal 15 Nov '03

15 Nov '03

On Fri, 14 Nov 2003, Mike Vannorsdel wrote: > You'll have to use the VFS functions (vn_open, vn_rdwr, vn_close, ect). > This also requires a good grasp on vnodes, vnode reference counting, > and vnode locking. You'll find doing file I/O from within the kernel > is very different, and more difficult, than from userland. > > Your best bet is to find a book or online tutorials on using VFS. The proper way to do this is to communicate with a user land daemon to do the file writing ala syslog. You might even be able to configure syslog to write your messages into a differnent file. But Apple's recommended way is to have something in userland do your file IO and use IOKit messages/communication if in the IOKit, use sockets if in a file system (which requires you to switch from BSD to network funnel to do I believe), etc. Quinn will have good insight into this. :) --- Marek Kozubal marek(a)portents.com _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Kernel Extensions
by George Andre 15 Nov '03

15 Nov '03

Mike Vannorsdel wrote: You'll have to use the VFS functions (vn_open, vn_rdwr, vn_close, ect). This also requires a good grasp on vnodes, vnode reference counting, and vnode locking. You'll find doing file I/O from within the kernel is very different, and more difficult, than from userland. Your best bet is to find a book or online tutorials on using VFS. Just a word of warning... Even though VN ops function will work, Apple is discouraging such an endeavour, since it will rely on the kernel exporting these specific symbols. As far as I know (someone from Apple might want to correct me if I'm wrong) relying on kernel symbols presence at kext link is a no-no. Plus, if I remember it correctly, to make it work a funnel change must be done; plus a possible break of synchronization. Having said that, the specific kernel symbols we are talking about have 99.9% chance/risk of not being removed/changed... so it's your choice. George. On Nov 13, 2003, at 12:34 PM, Matt Jaffa wrote: In a Kernel Extension, I am trying to do file reading and writing, how do i go about this, I included fstream but I dont think that this is supported. Any suggestions? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0