Hello Mr. Quinn, Thanks a lot for your quick & helpful answer! I came to the same conclusions after my first extended safari through the xnu-kernel source, but as i am mostly from a Linux background - used to the way how Linux handles locking/smp/preemption et al. - and quite new to OS-X, i needed some confirmation regarding xnu's locking et al. So, am i right that the kernel_funnel has this effect on all BSD syscalls, even on uniprocessor machines? Yes. Ah ja, needs to be on uniprocessor as well, because of the preemptible kernel functions, which pose the same requirements as really parallel (==smp) threads... Does this mean, that the kernel funnel is automatically released if a thread sleeps in the kernel, Yes. See the routine thread_block_reason in the following file. <http://www.opensource.apple.com/darwinsource/Current/xnu-517.3.7/osfmk/kern/sched_prim.c> [This link requires an APSL <http://www.opensource.apple.com/apsl/> account.] but that it is not released, if a thread gets preempted by a realtime thread -- which would cause the realtime thread to get delayed/blocked on the lock, as soon as it does a BSD syscall -- basically some kind of priority inversion? Yes. Let me just say on behalf of Apple that we recognise that funnels are bad. We're actively working to eliminate them and introduce finer grained locking to the BSD portions of the kernel. However, with current versions of Mac OS X, they are unavoidable. I understand. But as introducing fine grained locking in an existing kernel is quite a time consuming and incremental endeavour, we probably shouldn't expect this to be completed in the near future? Can i assume that it will take rather a couple of months or years instead of a couple of weeks or months? That would make perfect sense to me, but why doesn't the same thing happen to the gettimeofday() syscall that i use for timing and the usleep() when it gets back from sleep? My guess (didn't have time to check) is that these routines are implemented on top of Mach primitives, not BSD. Non-BSD syscalls don't need to take any funnels. I've checked now: usleep is implemented on top of the Mach-API and gettimeofday is one of a few BSD-calls that doesn't need the funnels. Is there a way to avoid this, e.g. by only using Apples Frameworks or Mach API's instead of Posix stuff? Virtually all frameworks use BSD at some point (you can't access the file system or the network without it), so you can't draw the general conclusion that Apple frameworks won't end up blocking on a funnel. However, there are some frameworks that are safe. For example, all of Mach and all of I/O Kit don't need the funnel. What about things like OpenGL, collecting keyboard responses or mouse button presses? Do the frameworks use Unix device files for communication with the device drivers (-> ioctls/read/write/mmap -> BSD calls -> block on funnel) or do they use the Mach-API? The main concern is not file i/o but querying the mouse and keyboard for user responses and OpenGL graphics output with well controlled timing, see below for reason. Our current -- ugly -- solution is to kill the update() process during realtime parts of the application. Is there something better, e.g. the equivalent of SCHED_FIFO scheduling on other Unices? The best solution is to avoid doing BSD stuff in your realtime thread. For example, let's say you're playing an audio file, reading from disk and playing via Core Audio (which calls you back in a real-time thread). The best way to do this is to separate the disk read and audio play into two threads, insert a buffer between the two, and make the disk thread non-realtime. If you can explain exactly what your realtime thread does, I can perhaps offer some alternative suggestions. Avoiding BSD stuff in this case is difficult: The application is called Psychtoolbox for MacOS-X (see http://www.psychtoolbox.org/osx.html ). It is an extension to/on top of Mathworks Matlab. It is used for programming psychophysical experiments on human visual perception in Matlab -- basically presentation of visual stimuli via OpenGL and collection of keyboard and mouse responses with controlled timing. Its very popular under vision scientists on the old OS-9 Macs and Windows and they are porting it to OS-X at the moment. On OS-9 they achieved "good enough" realtime behaviour by blocking processor interrupts and deferred actions and switching off virtual memory. In the OS-X implementation, "realtime" behaviour is realized by promoting the priority of the Matlab thread to THREAD_CONSTRAINT_POLICY during the experiments, but the content of the realtime part of an experiment can be any kind of mixture of OpenGL rendering, sound output, keyboard/mouse/hid checks, computation and blocking for a defined amount of time. The only thing it probably doesn't need to do in the realtime part is file i/o and large scale memory allocation. As a thread needs to block for some time on each iteration of the experiment loop, in order to not get demoted by the scheduler to normal timesharing mode due to overusage of cpu, it can always happen that some other thread does syscalls inbetween, which take the funnel lock. Another problem: Even if they could avoid any BSD syscalls in their code, the Matlab interpreter loop contains periodic calls to sigsetjmp(), which in turn calls sigprocmask() - which is a BSD syscall and causes the delays on the kernel funnel. As Mathworks seems to be not too enthusiastic about rewriting Matlab to avoid this function, the psychtoolbox guys have a problem. The whole history of the bug can be found under: http://www.psychtoolbox.org/osxTiming.html But anyway, knowing the root cause of the problem may help them to find a solution or workaround -- even if it boils down to killing the update process and calming down other sources of noise in the system. And a last question: Is there an equivalent to mlockall() to prevent paging for a realtime app? My stock answer to this question is that, if the system is short enough on memory that it's paging out parts of your app that you're touching at realtime, things are sufficiently messed up that the wiring wouldn't have helped. Coming from Linux - maybe the VM implementation is completely different in behaviour on OS-X - i would assume that you get delays even if you have enough memory, due to things like copy-on-write and zerofill on demand allocation of memory pages on first access and "faulting in" executable code, when a code path is executed the first time after application startup. On Linux, a mlockall(MCL_CURRENT | MCL_FUTURE) command would guarantee that all parts of the adress space are in core when the call returns. However, this appropriateness of this answer depends on what the negative effect of missing your realtime goal is. For example, glitching audio is not too bad on the overall scale of things, but failing to lower the rods on your nuclear reactor is bad (sufficiently bad that you shouldn't be using Mac OS X to control it). Messing up a few trials in an experiment is not that catastrophic, although too much unwanted delays can pose problems for fMRI experiments AFAIK, but how did you know that i have a nuclear reactor ;) Any more helpful comments are appreciated and thanks again. ciao, -mario -- ********************************************************************* Mario Kleiner Max Planck Institute for Biological Cybernetics Spemannstr. 38 72076 Tuebingen Germany e-mail: mario.kleiner@tuebingen.mpg.de office: +49 (0)7071/601-734 fax: +49 (0)7071/601-616 www: http://www.kyb.tuebingen.mpg.de/bu/people/kleinerm/index.html ********************************************************************* _______________________________________________ darwin-kernel mailing list | darwin-kernel@lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.