February 2008 - Darwin-dev - lists.apple.com

Re: shm_open and EMFILE
by David McGavran 16 Feb '08

16 Feb '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Like I said asking for help always let you find the answer. Debug in XCode 10240 Run in XCode 2200 something like that Finder or LSLaunchApplication 253 Is that correct? Cheers Dave On Feb 16, 2008, at 10:40 PM, David McGavran wrote: Cheers Dave On Feb 16, 2008, at 7:17 AM, Terry Lambert wrote: stdin, stdout, stderr = 3 10237 + 3 = 10240 -- Terry Ok I guess this is good news then :) Thanks! I love it when I am wrong :) Cheers Dave On Feb 15, 2008, at 2:05 AM, Terry Lambert wrote: You are mistaken. man 2 setrlimit _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/dmcgavra%40adobe.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Let me guess XCode sets your limits much higher than running from the finder? Here are my guess: I never enabled the setrlimit code because when I tested it in the debugger it wasn't doing anything, because I was already at max. Ok, I am officially confused. So hopefully I will solve it as soon as I write this email ;) If I launch my app (which is a server) and just loop on creating files I get 10237 which is cool. If I launch my app and have a client connect to it using NSConnection and loop on creating files I get 253, very consistently. Any ideas why that would be? Are there any debugging tricks for this? If I look in activity monitor it is showing the 253 files but not too much else. There are some descriptors used up for normal housekeeping. Also realize that for ach one you have open, that's a file, socket, pipe, etc. that you are not going to be able to open. On Feb 14, 2008, at 11:46 PM, David McGavran wrote: I did a much more simple test and was able to hit 10237 which is much closer to the max of 10240 as I hoped. There must be something else wrong, I will have to dig deeper. On Feb 14, 2008, at 11:43 AM, David McGavran wrote: I would guess this has been asked and answered so any pointers are fine. I tried to check the digest but didn't find the answer. I am trying to setup a lot of mem mapped files. But at or around 240 I get EMFILE which man page says means I have too many open. If I am not mistaken on 10.5 at least we are allowed 12000 or so open files? Why can I only open 240 or so memory mapped files? The limit on the number of files for a process is by default 256, as a soft constraint, and the value of maxfiles, as a hard constraint. The value of maxfiles represents the maximum number of files permitted an individual process. In addition, there is a hard constraint based on whatever your parent process decides to constrain you to; the default value of this is "unlimited" (by which it means "whatever the current sysctl-based hard limits are"). Once lowered the value of the hard limit cannot be reaised except by a process with root privilege. The value of maxfiles is reported by the sysctlbyname() value for "kern.maxfilesperproc"; this in turn is constrained by the the sysctlbyname() value for "kern.maxfiles", which is a compile time system limit. Note that the use of actual file descriptors to implement POSIX shared memory is an implementation detail, so all of the above is potentially subject to change in its relevency if that implementation detail ever changes. What can I do to open more? Also note that other administrative constraints other than number of available file descriptors (such as available kernel memory) will potentially also cause your attempt to ail (e.g. ENOMEM, ENOSPC, SPERM, EACCES, etc.). -- Terry This email sent to dmcgavra(a)adobe.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: shm_open and EMFILE
by David McGavran 16 Feb '08

16 Feb '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Cheers Dave On Feb 16, 2008, at 7:17 AM, Terry Lambert wrote: stdin, stdout, stderr = 3 10237 + 3 = 10240 -- Terry Ok I guess this is good news then :) Thanks! I love it when I am wrong :) Cheers Dave On Feb 15, 2008, at 2:05 AM, Terry Lambert wrote: You are mistaken. man 2 setrlimit _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Ok, I am officially confused. So hopefully I will solve it as soon as I write this email ;) If I launch my app (which is a server) and just loop on creating files I get 10237 which is cool. If I launch my app and have a client connect to it using NSConnection and loop on creating files I get 253, very consistently. Any ideas why that would be? Are there any debugging tricks for this? If I look in activity monitor it is showing the 253 files but not too much else. There are some descriptors used up for normal housekeeping. Also realize that for ach one you have open, that's a file, socket, pipe, etc. that you are not going to be able to open. On Feb 14, 2008, at 11:46 PM, David McGavran wrote: I did a much more simple test and was able to hit 10237 which is much closer to the max of 10240 as I hoped. There must be something else wrong, I will have to dig deeper. On Feb 14, 2008, at 11:43 AM, David McGavran wrote: I would guess this has been asked and answered so any pointers are fine. I tried to check the digest but didn't find the answer. I am trying to setup a lot of mem mapped files. But at or around 240 I get EMFILE which man page says means I have too many open. If I am not mistaken on 10.5 at least we are allowed 12000 or so open files? Why can I only open 240 or so memory mapped files? The limit on the number of files for a process is by default 256, as a soft constraint, and the value of maxfiles, as a hard constraint. The value of maxfiles represents the maximum number of files permitted an individual process. In addition, there is a hard constraint based on whatever your parent process decides to constrain you to; the default value of this is "unlimited" (by which it means "whatever the current sysctl-based hard limits are"). Once lowered the value of the hard limit cannot be reaised except by a process with root privilege. The value of maxfiles is reported by the sysctlbyname() value for "kern.maxfilesperproc"; this in turn is constrained by the the sysctlbyname() value for "kern.maxfiles", which is a compile time system limit. Note that the use of actual file descriptors to implement POSIX shared memory is an implementation detail, so all of the above is potentially subject to change in its relevency if that implementation detail ever changes. What can I do to open more? Also note that other administrative constraints other than number of available file descriptors (such as available kernel memory) will potentially also cause your attempt to ail (e.g. ENOMEM, ENOSPC, SPERM, EACCES, etc.). -- Terry This email sent to site_archiver(a)lists.apple.com

1 0

Re: getcontext() -> Bus error?
by Edd Dawson 16 Feb '08

16 Feb '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Thunderbird 2.0.0.9 (Windows/20071031) Hi Terry, Thanks for your patient response. I really appreciate it. Terry Lambert wrote: > When you call getcontext/makecontext/setcontext, you are hacking the > contents of your context. This includes register and other state, > including the stack register. This makes sense, since these functions > are typically used to implement user space threads schedulers. This is what I'm doing (user space threads). > Then you go and depend on the contents of a value which is either stack > relative, or which has been loaded into a volatile register, and expect > the code to "do the right thing", even though you are saying something > like "load a bunch of stack relative data off an offset of -256 off the > stack" (or whatever), or you are saying "load a bunch of data relative > the the address contained in register such-and-such I just overwrote". > Either of these can now refer to totally different data, but the > compiler didn't know about that as a barrier, so it uses the stale > contents not knowing the contents of what it is using was changed out > from under it. > Either way you look at it, this stuff is going to break under those > conditions. > > Minimally, you should be using a chunk of heap or global data, the > address of which is contained in a variable that has been declared > "volatile" so that (a) it will be passed to the functions in a volatile > register, and (b) after the function returns, it doesn't try to pop data > from the stack back into the register, particularly because your new > stack doesn't have the saved data on it anyway, and even if it did, the > data it wants to pop back only exists on the old stack, which you are no > longer referencing. Ok, thank you. I shall make the appropriate changes. > I think on other systems you are simply lucking out due to internal > implementation, or your compiler is doing different optimizations by > default. > > Using functions like these correctly will generally take a pretty > fundamental understanding of the machine architecture, the assembly code > that will be emitted by your compiler, and the relevant standards > governing the functions behaviour. Kind regards, Edd _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I can see how what I did would be problematic for certain implementations. On the other hand I don't believe I'm doing anything funny as far as the POSIX standard is concerned. Please excuse the naivety of this next question as I'm not well read on the low level details, but running with your example, would absolute offsets not be necessary rather than relatives to ensure POSIX compliance? Well, I was under the impression that the relevant standard was POSIX and that my expectations were in line with its wording. Have I missed something? I thought the whole point of these functions was to provide a layer of abstraction so relative laymen such as myself don't have to worry about compiler/target-architecture details. This email sent to site_archiver(a)lists.apple.com

1 0

Re: shm_open and EMFILE
by Terry Lambert 16 Feb '08

16 Feb '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com stdin, stdout, stderr = 3 10237 + 3 = 10240 -- Terry Ok I guess this is good news then :) Thanks! I love it when I am wrong :) Cheers Dave On Feb 15, 2008, at 2:05 AM, Terry Lambert wrote: You are mistaken. man 2 setrlimit _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… There are some descriptors used up for normal housekeeping. Also realize that for ach one you have open, that's a file, socket, pipe, etc. that you are not going to be able to open. On Feb 14, 2008, at 11:46 PM, David McGavran wrote: I did a much more simple test and was able to hit 10237 which is much closer to the max of 10240 as I hoped. There must be something else wrong, I will have to dig deeper. On Feb 14, 2008, at 11:43 AM, David McGavran wrote: I would guess this has been asked and answered so any pointers are fine. I tried to check the digest but didn't find the answer. I am trying to setup a lot of mem mapped files. But at or around 240 I get EMFILE which man page says means I have too many open. If I am not mistaken on 10.5 at least we are allowed 12000 or so open files? Why can I only open 240 or so memory mapped files? The limit on the number of files for a process is by default 256, as a soft constraint, and the value of maxfiles, as a hard constraint. The value of maxfiles represents the maximum number of files permitted an individual process. In addition, there is a hard constraint based on whatever your parent process decides to constrain you to; the default value of this is "unlimited" (by which it means "whatever the current sysctl- based hard limits are"). Once lowered the value of the hard limit cannot be reaised except by a process with root privilege. The value of maxfiles is reported by the sysctlbyname() value for "kern.maxfilesperproc"; this in turn is constrained by the the sysctlbyname() value for "kern.maxfiles", which is a compile time system limit. Note that the use of actual file descriptors to implement POSIX shared memory is an implementation detail, so all of the above is potentially subject to change in its relevency if that implementation detail ever changes. What can I do to open more? Also note that other administrative constraints other than number of available file descriptors (such as available kernel memory) will potentially also cause your attempt to ail (e.g. ENOMEM, ENOSPC, SPERM, EACCES, etc.). -- Terry This email sent to site_archiver(a)lists.apple.com

1 0

Re: Darwin 9.2 Sources
by Rosyna 15 Feb '08

15 Feb '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Feb 15, 2008, at 1:00 AM, Graham J Lee wrote: On 15 Feb 2008, at 04:14, Rosyna wrote: --- Sincerely, Rosyna Keller Technical Support/Carbon troll/Always needs a hug Unsanity: Unsane Tools for Insanely Great People It's either this, or imagining Phil Schiller in a thong. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I think I could handle New CF, after all, I've seen that thing about the girls and the cup. But having the real CF source with _CFLogvEx (which NSLog calls) was important to me as I wanted to know how it uses asl (x-man-page://asl) but the CFLite source uses fprintf_l(). And for the preference thing, I wanted to know how CF was poisoned with Objective-C. But there are quite a few problems with synching multiple pref domains in an application if one domain is poisoned (has an over released value), then the next call to sync any domain will cause a crash. I was hoping to figure out how to debug which domain poisoned the prefs. Is it just me or is the CF source code significantly different from the CF source used used on Mac OS X? I'm not seeing _CFLogvEx and I'm not seeing the objective-C stuff that's in the CFPreferences now. It's not just you. That's not CF, that's CF-Lite. If CF is sugar water, then CF-Lite is at least wet... just be thankful we don't get New CF ;-) This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to pick which architecture to launch?
by Sean McBride 15 Feb '08

15 Feb '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Organization: Rogue Research On 2/15/08 12:56 PM, Jordan K. Hubbard said: >> If I have an obese binary (4 way), is there a way I can pick which >> architecture to launch? > >If it's possible to launch this from the command line, then you can >use the arch(1) command to specify which architecture to run. It >you're relying on Launch Services to launch the app, then I'm not sure >that there currently is a way. Looks like 'arch' will work great. Thanks Chris and Jordan. -- ____________________________________________________________ Sean McBride, B. Eng sean(a)rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montréal, Québec, Canada _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to pick which architecture to launch?
by Jordan K. Hubbard 15 Feb '08

15 Feb '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Feb 15, 2008, at 9:28 AM, Sean McBride wrote: - Jordan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… If I have an obese binary (4 way), is there a way I can pick which architecture to launch? If it's possible to launch this from the command line, then you can use the arch(1) command to specify which architecture to run. It you're relying on Launch Services to launch the app, then I'm not sure that there currently is a way. This email sent to site_archiver(a)lists.apple.com

1 0

Re: How to pick which architecture to launch?
by Chris Sarcone 15 Feb '08

15 Feb '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

How to pick which architecture to launch?
by Sean McBride 15 Feb '08

15 Feb '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Organization: Rogue Research Hi all, If I have an obese binary (4 way), is there a way I can pick which architecture to launch? For example, with Xcode, if I build an OCUnit test, it runs the test under as many architectures as the host machine allows. I'd like to do something similar with my own custom automated testing. I build my executable as obese, then I'd like to launch it 3 times: as i386, as x86_64, as ppc (in Rosetta). I am aware of LSArchitecturePriority and similar Info.plist keys but that would mean changing my executable, which I'd really rather not do. I want to test the executable as is. Thanks, -- ____________________________________________________________ Sean McBride, B. Eng sean(a)rogue-research.com Rogue Research www.rogue-research.com Mac Software Developer Montréal, Québec, Canada _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Darwin 9.2 Sources
by Graham J Lee 15 Feb '08

15 Feb '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 15 Feb 2008, at 04:14, Rosyna wrote: Graham. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Is it just me or is the CF source code significantly different from the CF source used used on Mac OS X? I'm not seeing _CFLogvEx and I'm not seeing the objective-C stuff that's in the CFPreferences now. It's not just you. That's not CF, that's CF-Lite. If CF is sugar water, then CF-Lite is at least wet... just be thankful we don't get New CF ;-) This email sent to site_archiver(a)lists.apple.com

1 0