August 2009 - Darwin-kernel

Re: Idle priority
by Mike Mimic 05 Aug '09

05 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1249509046; bh=V8C0I+vKWeunselLm/5Ml7w3Y2teBaFBa6YSt69TdcU=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=kP6CodB6bgDkIHUhnPnYc5dw2W4CwoCKhWvFdrA9Gfoc5MBghDCmgmCns9ErNEIHPX+/Ji1PFcYMKhfN0Cas7MvQEq7X3mVz4FItTBWxts2uzgOIZTlRdXUg85Iq1d6EUN2k7QLLVqlrM7+OoHBB1ns8fJ3ReG2Pkso6wzs1dS8= Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=Q+3ZI3nq14UltId2dSzdPVYknuwpqHkYJfVAH66eEvKk9bTOfLmHn2bxWgvdmpILrJ5xm43PrMB8yDovgrosX7vq0C59d/FDIah8V73rDMYUiQn2E36f9AYQKrqGRYqgaQQ5QLBFAw7YK9SZ9o49u07zDS3316hDlNLEX4h6CM4=; Hi! --- On Wed, 8/5/09, Terry Lambert <tlambert(a)apple.com> wrote: > I will point out that the reported real time on the second > one is very close to 20.0722, double the CPU time, which > implied that it didn't actually get any CPU time until the > competing (non-idle priority) process finished, which is > what you were appeared to be asking to have happen. I think you misunderstood the results or I have presented them badly. I have been measuring twice "load" program which represents my normal load for which I do not want to have any slowdown while I run my "idle" program in the background on the idle priority. But results show that there is slowdown. In fact the slowdown is around 100% (real time is 20 seconds instead of 10 seconds) what shows that the "idle" processes have probably not been ran under any decreased priority whatsoever (like my example with nice previous time which showed slight decrease, but still no idle priority decrease). So proper behavior (as I see it) would be that in both cases my "load" processes would take 10 seconds to complete. Whether there are "idle" processes running (the second case) or not (the first case). > To get actual time-on-cpu on a per-process basis you'd need > to examine the thread state structure before the thread > exits. This is why I suggested you use shark to gather > the information, back when this thread started. I doubt that this difference (10 seconds) is the case of me using time instead of Shark. Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: virtual memory limit
by Terry Lambert 05 Aug '09

05 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: virtual memory limit
by Terry Lambert 05 Aug '09

05 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: virtual memory limit
by Roger Herikstad 05 Aug '09

05 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=4h8oXiVKSqbBaINnOHfNwXhdC2093J+vC980fMnhC1I=; b=FctH64iOAfy1I27IeHcoJv760HLp9NavyLUwjL5gYtuGps7ElO8XhvHn9C7F0a/Cq2 iNODK5YSmVOvo2XrkbX9hMCD++4bQ0vYQQ1QNFql80I2ptw/HLHJjdL+Dugq5DcmUjnh fl0TXP3bPNxIn/PLHfIn6TufBngOSmF5PMZiE= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=vpePjdLJC+sz6cfi0/RU03EYRGLsKnrVIahOOzDg3OJRlyl40t1bvIBnd9ieCXMWa/ ZiDHaKZGX6bAUlgwP3hDnszYe/5MRflNt6HOk1d3r2MsP/VAXzVD+ba7spa2W5suAeBA 9KKicE94noavLKEQdgM3pLeUnZqHcW7bChYV0= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: virtual memory limit
by Tom Duffy 05 Aug '09

05 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Do you mean swap? If so, you can turn off the dynamic pager. ™ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On Aug 5, 2009, at 7:57 AM, Roger Herikstad <roger.herikstad(a)gmail.com> wrote: Hi list, I was wondering if there is a way to set a limit on how much virtual memory the operating system should be allowed to allocate? I have a MacPro with 16G of memory running 10.5.6. When using 64 bit applications to analyze big data sets it happens occasionally that the application requests more memory than can be accommodated on the system. As a result, pages are written to disk until there's no more disk space available, which hangs the machine.. Is there a way to change this behavior, such that I would get a memory allocation error instead when (some times accidentally) trying to allocate such huge chunks of memory? Thanks! ~ Roger _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/tduffy %40apple.com This email sent to tduffy(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: virtual memory limit
by Roger Herikstad 05 Aug '09

05 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=DYeaVSoaKoGPlPsKjw9tmFJBRzskEkGhpylYWLYAZLE=; b=qpi4z/GPd7Vum19J0f1j2aBqV044aa00yL5fAutDO9Bm9bm41qfEddh3ke1vMbAA9W GjUxqEHAlcWLGrvJ2TgGY6ByYTHMe8lvn4V/BKU9LQ7nuDGfOlyAsj3Rmk4hQ8bxy9zY jmGlCSugEYgQGFf3A5ZWlpPWxUf/nEhANVejY= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=tx+icPmp8OYVKFkm1HYjzr+MijjwBNYGgbOQ0QOPpKVRN7y6ZAbcy9n3Ix5K2Ft8kQ iFm4nM85miTwyM2Trun/CAOoSFm3dllO71RwqTIg2jy/iT+5F59zOGQ79NuP6VMMnIm7 BYBNA7Q6WJjd+pQhvv/m3WkFH9E/a4DJuqXmE= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

virtual memory limit
by Roger Herikstad 05 Aug '09

05 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=S4uEJ6prexkXRRxsT8pvx3JerM4JQrJ2LH2uuNfu+I0=; b=T93wjl6qI/CrqmjBmp3o0LSoyuqz8Lj56C+mwxKg/dsd0ZbswZ+Ki2dKq0omVSxtxT jQpXId6mEHEGmG0Htwf7YwONFPZ7aSrDCReetbFsdp7Dx81RJPVuh/YoDxbV/YVOLght Ph7ci6tq3Bz4qpDwnAafWlrHNgrLkZ5pf+obg= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=hS4qPgbWIUXjx8SkWi3CVzX5lcR2J2qoSvQhDnGRJmxziCuPlpaSe8kf5Fpvfjaacz 8F9BF4JzfMmLS2NyTpLPJowWDW/dAybgvLvnqfj50ldTA30Yny6xtM9fJPCzg69bMBvp h8zzjmA9VKfYAQAD4Sj9eNvqfl1i0JGhpBCRE= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Idle priority
by Terry Lambert 04 Aug '09

04 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -- Terry Hi! So, any suggestions? Is then this a bug in the scheduler? Mike --- On Fri, 7/31/09, Mike Mimic <ppagee(a)yahoo.com> wrote: From: Mike Mimic <ppagee(a)yahoo.com> Subject: Re: Idle priority To: "Terry Lambert" <tlambert(a)apple.com> Cc: "darwin-kernel(a)lists.apple.com" <darwin-kernel(a)lists.apple.com> Date: Friday, July 31, 2009, 1:54 AM Hi! --- On Thu, 7/30/09, Terry Lambert <tlambert(a)apple.com> wrote: You should probably read: Thanks. I have read it and improved my test program, "nice": #include <mach/mach.h> #include <mach/thread_policy.h> #include <stdio.h> int set_idle() { struct thread_precedence_policy policy; policy.importance = 0; // IDLE_PRI return thread_policy_set(mach_thread_self(), THREAD_PRECEDENCE_POLICY, (thread_policy_t)&policy, THREAD_PRECEDENCE_POLICY_COUNT) == KERN_SUCCESS; } int main(int argc, char *argv[]) { if (!set_idle()) { fprintf(stderr, "set_idle() failed\n"); return 1; } int j; for (j = 0; ; j++) { printf("%d\n", j); // Busy wait int i; for (i = 0; i < 100000000; i++); } return 0; } In documentation it is written about the importance value: While this is a signed 32-bit value, the minimum legal value is zero (IDLE_PRI). threads set to IDLE_PRI will only execute when no other thread is scheduled to execute. Measurements are as follows. In the first case I got such values: - user time - mean: 10.0282 - standard deviation: 0.0008 - real time - mean: 10.0939 - standard deviation: 0.1165 In the seconds case I got such values: - user time - mean: 10.0285 - standard deviation: 0.0005 - real time - mean: 20.0722 - standard deviation: 0.1155 There is even bigger difference. Am I doing something wrong? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… If you think there is a bug there, then you need to file a bug and attach your code examples so that the engineers who work on the scheduler can decide whether or not it's a problem of implementation or of perception. I will point out that the reported real time on the second one is very close to 20.0722, double the CPU time, which implied that it didn't actually get any CPU time until the competing (non-idle priority) process finished, which is what you were appeared to be asking to have happen. However, you can't take the "top" or "time" numbers literally, since they operate on digested resource usage information obtained after the child exits before it is reaped, using BSD rusage() calls. To get actual time-on-cpu on a per-process basis you'd need to examine the thread state structure before the thread exits. This is why I suggested you use shark to gather the information, back when this thread started. On Aug 4, 2009, at 2:39 PM, Mike Mimic wrote: <http://developer.apple.com/documentation/Darwin/Conceptual/KernelProgrammin… > which documents the Mach scheduler interfaces. Mike This email sent to site_archiver(a)lists.apple.com

1 0

Re: Idle priority
by Mike Mimic 04 Aug '09

04 Aug '09

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1249421974; bh=XMO60CzLEt91ZnqkXkKVk5Dr1UyRJYxaHmIVrfemU4k=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=KOV4KElXd71kefQAfCAQZqN24nJOzYmwymg4PKnxSiAwCa9vjBGAeiwSCXJNpUu+i3Co5qg7XKVTBaI66yPCbKSPu1caXTtT9mDrvvOK8b34CLdenDKxnE05DsuMJV4/YdTMrTt+h9BkWZaUYnUKuy5kFDd1a+H6eAgD7OXOl0Q= Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=vsM23mWOsxrlLwPTwU9pBEZkxmDrvJoaLeJXwFjLy5yggMd85k8vjGmVo5iYEFPaQGUgTs/RGRls77mlC5CARwTaE0Mr6aRJzOjlCZ5VzbSxN6t03upbcjP+R3TJKHU69VBsonz71G+1NfNwvEannCYKQK40bL0yyDW02cV/ya4=; Hi! So, any suggestions? Is then this a bug in the scheduler? Mike --- On Fri, 7/31/09, Mike Mimic <ppagee(a)yahoo.com> wrote: > From: Mike Mimic <ppagee(a)yahoo.com> > Subject: Re: Idle priority > To: "Terry Lambert" <tlambert(a)apple.com> > Cc: "darwin-kernel(a)lists.apple.com" <darwin-kernel(a)lists.apple.com> > Date: Friday, July 31, 2009, 1:54 AM > Hi! > > --- On Thu, 7/30/09, Terry Lambert <tlambert(a)apple.com> > wrote: > > You should probably read: > > > > <http://developer.apple.com/documentation/Darwin/Conceptual/KernelProgrammin…> > > > > which documents the Mach scheduler interfaces. > > Thanks. I have read it and improved my test program, > "nice": > > #include <mach/mach.h> > #include <mach/thread_policy.h> > #include <stdio.h> > > int set_idle() { > struct thread_precedence_policy policy; > policy.importance = 0; // IDLE_PRI > > return > thread_policy_set(mach_thread_self(), > THREAD_PRECEDENCE_POLICY, (thread_policy_t)&policy, > THREAD_PRECEDENCE_POLICY_COUNT) == KERN_SUCCESS; > } > > int main(int argc, char *argv[]) { > if (!set_idle()) { > fprintf(stderr, > "set_idle() failed\n"); > return 1; > } > > int j; > for (j = 0; ; j++) { > printf("%d\n", j); > > // Busy wait > int i; > for (i = 0; i < > 100000000; i++); > } > > return 0; > } > > In documentation it is written about the importance value: > > While this is a signed 32-bit value, the minimum legal > value is zero (IDLE_PRI). threads set to IDLE_PRI will only > execute when no other thread is scheduled to execute. > > Measurements are as follows. > > In the first case I got such values: > > - user time > - mean: 10.0282 > - standard deviation: 0.0008 > - real time > - mean: 10.0939 > - standard deviation: 0.1165 > > In the seconds case I got such values: > > - user time > - mean: 10.0285 > - standard deviation: 0.0005 > - real time > - mean: 20.0722 > - standard deviation: 0.1155 > > There is even bigger difference. Am I doing something > wrong? > > > Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0