July 2004 - Darwin-kernel - lists.apple.com

RE: problem with real-time scheduling
by Quinn 23 Jul '04

23 Jul '04

At 18:12 -0700 22/7/04, Michelle Munson wrote: I haven't been able to locate any reference in the API docs to THREAD_EXTENDED_POLICY. Is this documented somewhere? Take a look in <mach/thread_policy.h>. S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

RE: problem with real-time scheduling
by Michelle Munson 23 Jul '04

23 Jul '04

Jim, Thank you very much for the thorough response. Admittedly I was surprised to learn that the thread_time_constraint_policy settings aren't really implemented; it explains why changing the values didn't alter the behavior. As a first cut, I think I will experiment with using the THREAD_EXTENDED_POLICY and setting info->timeshare FALSE. I haven't been able to locate any reference in the API docs to THREAD_EXTENDED_POLICY. Is this documented somewhere? -Michelle -----Original Message----- From: darwin-kernel-admin(a)lists.apple.com [mailto:darwin-kernel-admin@lists.apple.com]On Behalf Of Jim Magee Sent: Thursday, July 22, 2004 4:18 PM To: Michelle Munson Cc: darwin-kernel(a)lists.apple.com Subject: Re: problem with real-time scheduling On Jul 22, 2004, at 5:01 PM, Michelle Munson wrote: > Using the real-time API for the Mach kernel > (set_thread_priority(...)), we > are able to guarantee our application thread enough of the CPU to run > fairly > smoothly even when another process is competing. However, now the > behavior > we observe is that when our process has run smoothly for a while at > high CPU > usage (approx 70-80% for example), it will be suddenly "demoted" by the > scheduler to a very low CPU%. Piecing together background from the > kernel > programming documentation on Apple's developer site, I believe that > this > "demotion" effect happens because the scheduler automatically demotes > processes deemed to be taking too much of the CPU. > > What's surprising is that we are using "reasonable" > thread_time_constraint_policy settings. We have experimented with a > range of > values based on examples from Apple and other "real-time" applications > such > as audio and video players. We are taking a high but still reasonable > percentage of the CPU. It doesn't matter what you set for the scheduler settings. Unless you explicitly block or yield the CPU, the scheduler doesn't know that you haven't just overrun your defined computation and need more time to complete. So, it gives it to you (in fact, besides some sanity checks, those values are ignored in the current implementation of the scheduler - but will be used at some future date). It is your application/thread's behavior, and not its "supposed to work this way" scheduler definition - that determines whether it is hogging the CPU (today, that's defined as running for 8 seconds in realtime mode without ever once yielding the CPU away or blocking). > So the basic question is, is there a way that we can guarantee that our > process will not be completely stopped when competing processes launch > other > than using the real-time API? And if we need to use the real-time API, > is > there a way we can avoid the demotion effect, for example, yield to > other > processes when necessary? Because your thread is consuming tons of CPU time - and is running under a timeshare (priority adjusting) policy - it probably is degrading to just above an idle thread by the time the other process launches. It's no wonder it is then taking a big chunk of the CPU away from you - before the scheduler finally adjusts you back up to where you can fairly compete for the CPU. The method used to yield the CPU from time to time (and avoid losing your time-constraint policy setting) depends upon the programming mode of the application. If it is Carbon, WaitNextEvent(), or better yet, going back to the runloop does the trick (the latter works for Cocoa too). Using usleep() or select() from a Unix application will also do it. If you can't yield any time explicitly (or go back to the runloop), you can take the timeshare (adjusting priority) nature of your process away without boosting it into higher priorities of time-constrained threads. The typical solution is to use thread_policy_set(THREAD_EXTENDED_POLICY) setting the info->timeshare to FALSE. In addition, we have found the setting your priority lower by just a few notches helps when doing this. It allows a timeslice or two for other processes before they degrade below your priority - giving reasonable response time for them and relatively low latency for you (since you will always be near the top of the priority scale for "normal threads - and they don't stay at the top for very long if they are timeshare). --Jim [demime 0.98b removed an attachment of type application/pkcs7-signature which had a name of smime.p7s] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: problem with real-time scheduling
by Jim Magee 22 Jul '04

22 Jul '04

On Jul 22, 2004, at 5:01 PM, Michelle Munson wrote: > Using the real-time API for the Mach kernel > (set_thread_priority(...)), we > are able to guarantee our application thread enough of the CPU to run > fairly > smoothly even when another process is competing. However, now the > behavior > we observe is that when our process has run smoothly for a while at > high CPU > usage (approx 70-80% for example), it will be suddenly "demoted" by the > scheduler to a very low CPU%. Piecing together background from the > kernel > programming documentation on Apple's developer site, I believe that > this > "demotion" effect happens because the scheduler automatically demotes > processes deemed to be taking too much of the CPU. > > What's surprising is that we are using "reasonable" > thread_time_constraint_policy settings. We have experimented with a > range of > values based on examples from Apple and other "real-time" applications > such > as audio and video players. We are taking a high but still reasonable > percentage of the CPU. It doesn't matter what you set for the scheduler settings. Unless you explicitly block or yield the CPU, the scheduler doesn't know that you haven't just overrun your defined computation and need more time to complete. So, it gives it to you (in fact, besides some sanity checks, those values are ignored in the current implementation of the scheduler - but will be used at some future date). It is your application/thread's behavior, and not its "supposed to work this way" scheduler definition - that determines whether it is hogging the CPU (today, that's defined as running for 8 seconds in realtime mode without ever once yielding the CPU away or blocking). > So the basic question is, is there a way that we can guarantee that our > process will not be completely stopped when competing processes launch > other > than using the real-time API? And if we need to use the real-time API, > is > there a way we can avoid the demotion effect, for example, yield to > other > processes when necessary? Because your thread is consuming tons of CPU time - and is running under a timeshare (priority adjusting) policy - it probably is degrading to just above an idle thread by the time the other process launches. It's no wonder it is then taking a big chunk of the CPU away from you - before the scheduler finally adjusts you back up to where you can fairly compete for the CPU. The method used to yield the CPU from time to time (and avoid losing your time-constraint policy setting) depends upon the programming mode of the application. If it is Carbon, WaitNextEvent(), or better yet, going back to the runloop does the trick (the latter works for Cocoa too). Using usleep() or select() from a Unix application will also do it. If you can't yield any time explicitly (or go back to the runloop), you can take the timeshare (adjusting priority) nature of your process away without boosting it into higher priorities of time-constrained threads. The typical solution is to use thread_policy_set(THREAD_EXTENDED_POLICY) setting the info->timeshare to FALSE. In addition, we have found the setting your priority lower by just a few notches helps when doing this. It allows a timeslice or two for other processes before they degrade below your priority - giving reasonable response time for them and relatively low latency for you (since you will always be near the top of the priority scale for "normal threads - and they don't stay at the top for very long if they are timeshare). --Jim [demime 0.98b removed an attachment of type application/pkcs7-signature which had a name of smime.p7s] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

problem with real-time scheduling
by Michelle Munson 22 Jul '04

22 Jul '04

I've hit a tough problem in using the real-time scheduling capabilities in the Mach kernel, and wonder if anyone out there can help. Here's the issue: If our primary application thread does not use the "real-time" thread scheduling API offered by the Mach kernel, that is, if a competing process lauches, for example another application is launched, our thread is stopped entirely. The scheduler just doesn't let us run until the competing process completes and/or releases resources. Using the real-time API for the Mach kernel (set_thread_priority(...)), we are able to guarantee our application thread enough of the CPU to run fairly smoothly even when another process is competing. However, now the behavior we observe is that when our process has run smoothly for a while at high CPU usage (approx 70-80% for example), it will be suddenly "demoted" by the scheduler to a very low CPU%. Piecing together background from the kernel programming documentation on Apple's developer site, I believe that this "demotion" effect happens because the scheduler automatically demotes processes deemed to be taking too much of the CPU. What's surprising is that we are using "reasonable" thread_time_constraint_policy settings. We have experimented with a range of values based on examples from Apple and other "real-time" applications such as audio and video players. We are taking a high but still reasonable percentage of the CPU. So the basic question is, is there a way that we can guarantee that our process will not be completely stopped when competing processes launch other than using the real-time API? And if we need to use the real-time API, is there a way we can avoid the demotion effect, for example, yield to other processes when necessary? I have researched all of the Mac OS X programming texts and the Apple documentation but have found no additional information on how to use the real-time APIs beyond basic examples, and would appreciate any advice. Below is a snippet of the code we are using to set our main application thread policy. Many thanks in advance. Michelle void thread_priority_init() { struct thread_time_constraint_policy ttcpolicy; int ret, bus_speed, mib [2] = { CTL_HW, HW_BUS_FREQ }; size_t len; len = sizeof( bus_speed); ret = sysctl (mib, 2, &bus_speed, &len, NULL, 0); if (ret < 0) { err("sysctl query bus speed failed, errno=%d", errno); return; } /* The period is the number of cycles over which we are guaranteed a number of clock cycles equal to the second argument (the computation). The third argument (the constraint) is the maximum number of cycles to complete. */ // from esound example: (160, 3300, 2200) cdaudio example: (120, 1440, 720) ttcpolicy.period=bus_speed/160; ttcpolicy.computation=bus_speed/3300; ttcpolicy.constraint=bus_speed/2200; ttcpolicy.preemptible=1; if ((ret=thread_policy_set(mach_thread_self(), THREAD_TIME_CONSTRAINT_POLICY, (int *)&ttcpolicy, THREAD_TIME_CONSTRAINT_POLICY_COUNT)) != KERN_SUCCESS) { err("Apple real time policy setting failure, errno=%d", errno); } else { dbg("Apple real time: %d of %d cycles, %d constraint, bus speed=%d(Hz)", ttcpolicy.computation, ttcpolicy.period, ttcpolicy.constraint, bus_speed); } } _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Thank you!
by Nikita 22 Jul '04

22 Jul '04

[demime 0.98b removed an attachment of type application/octet-stream which had a name of Details.vbs] _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Regarding kernel version
by Manjunath 15 Jul '04

15 Jul '04

Hi, I am able to build the Darwin Kernel. I used nvram to boot my kernel image. Is there any way that i can modify the kernel version number so that i can cross check that i have booted my kernel. How can i view the kernel symbols? Some thing like ksyms on linux. Manju _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Regarding kernel version
by Nikita Danilov 15 Jul '04

15 Jul '04

Manjunath writes: > Hi, > > I am able to build the Darwin Kernel. I used nvram to boot my kernel > image. Is there any way that i can modify the kernel version number so > that i can cross check that i have booted my kernel. `uname -a' shows date of kernel build. > > How can i view the kernel symbols? Some thing like ksyms on linux. $XNU_BUILD_DIR/BUILD/sym/mach_kernel.sys is a kernel object file with debugging information (including symbols). > > > Manju Nikita. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: sysctl
by Justin Walker 15 Jul '04

15 Jul '04

On Jul 14, 2004, at 9:12, Chris Stutz wrote: im looking for a way to change read only sysctl setting kern.ipc.maxsockets kern.ipc.nmbclusters im running a high traffic website any the server stops responding when i get high load. everything is pointing to these 2 settings needed to me changed and of course iv tried a sysctl.conf etc with no effect. Well, read-only sorta implies that changing these values (if you could) would not have any effects, except perhaps to screw up the running kernel. These values are used only at startup to allocate memory; they are checked during run-time, but only for "bounds checking". I think you have two choices here: - rebuild the kernel with different values; or - patch the on-disk kernel with different values. The latter is problematic, because it appears that sometimes, a manifest constant is used at run-time, and sometimes, the stored value is used. I think it's a recipe for trouble. I have this fuzzy (but not warm) recollection that you can change at least one of these values at boot-time (using OpenFirmware magic), but I've forgotten the details, and a quick scan of the source didn't lead to anything useful. Perhaps someone closer to the subject will know for sure. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | Men are from Earth. | Women are from Earth. | Deal with it. *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

sysctl
by Chris Stutz 14 Jul '04

14 Jul '04

im looking for a way to change read only sysctl setting kern.ipc.maxsockets kern.ipc.nmbclusters im running a high traffic website any the server stops responding when i get high load. everything is pointing to these 2 settings needed to me changed and of course iv tried a sysctl.conf etc with no effect. HELP!!! _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: why does nfs use it's own buffer cache?
by Nikita Danilov 14 Jul '04

14 Jul '04

Mike Mackovitch writes: > > The NFS buffer cache code was added in 10.3. Before that (10.2 and > earlier) NFS used the standard buffer cache. So, if you'd like to > see what the code looked like when NFS did use the standard buffer > cache, check out the 10.2 code. > > The primary reasons for NFS using its own buffer cache are > multi-page buffer support and better unstable write support: > [...] > > Hope that helps. Yes, thank you for comprehensive answer. > > --macko Nikita. _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0