May 2007 - Darwin-kernel - lists.apple.com

Re: Reload VM hint?
by Michael Smith 30 May '07

30 May '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com On May 30, 2007, at 7:47 AM, James Bucanek wrote: = Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'm hoping this is the correct list. It sounds like a Kernel issue. Feel free to enlighten me if it's not. I'm looking for a way to reload pages that have been paged out by another process. They scenario is pretty simple: I've written an application that uses a lot (100s of MB) of RAM. Users run the application (it's a backup utility). Applications does its thing and terminates. Users now complain that their system is "sluggish and unresponsive." The problem, obviously, is that as the application ran it forced just about everything else in RAM to be paged out to the VM backing files. The OS then spends the next 30 seconds, or more, paging it back in -- a period of time my users describe as "eternity." Is there anyway for my application to send a "hint" or command to the VM manager that will cause it to reload the recently paged out memory pages? I have a supervisory process (a scheduler daemon) that can do this after the process terminates. No, there isn't. The system has no way of telling the difference between pages that were evicted but still relevant and pages that were evicted and not relevant; it has to wait for other applications to need them. Brian's somewhat terse observation might bear a little expansion, though I think he may be on the right path. You say your application is a backup utility, and that it uses hundreds of megabytes of RAM. What do you mean by "uses"? Do you make large allocations inside your application's address space? Why? If you are buffering for a device, you can probably get by with much less buffer than you think; certainly not hundreds of megabytes. As a backup utility, you are probably reading a lot of files; as Brian notes if you apply the F_NOCACHE fcntl to these files immediately after opening them, you will avoid evicting other applications' pages in favour of file cache pages. You should only do this for files you are only reading once; if you read a file more than once in a relatively short period of time, or you write it with the intention of it being read shortly thereafter you're better of letting the system cache it. HTH; feel free to ask more questions. Your users, incidentally, should buy more RAM. 8) This email sent to site_archiver(a)lists.apple.com

1 0

Re: Strange iTune directory/file access paths
by Todd Heberlein 30 May '07

30 May '07

site_archiver(a)lists.apple.com Delivered-To: Darwin-kernel(a)lists.apple.com Thanks! QA1113 exactly answered my questions. Todd _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Reload VM hint?
by Brian Bergstrand 30 May '07

30 May '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 man fcntl Read the F_NOCACHE section. On May 30, 2007, at 9:47 AM, James Bucanek wrote: Greetings, -- James Bucanek Brian Bergstrand <http://www.bergstrand.org/brian/> PGP Key ID: 0xB6C7B6A2 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFGXaUDedHYW7bHtqIRAqv0AKDptN3olIllr5t6tzpoM/3gt3C+IQCfQ656 6DAucGKXI8G/Sd4fUu156lk= =PWfV -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'm hoping this is the correct list. It sounds like a Kernel issue. Feel free to enlighten me if it's not. I'm looking for a way to reload pages that have been paged out by another process. They scenario is pretty simple: I've written an application that uses a lot (100s of MB) of RAM. Users run the application (it's a backup utility). Applications does its thing and terminates. Users now complain that their system is "sluggish and unresponsive." The problem, obviously, is that as the application ran it forced just about everything else in RAM to be paged out to the VM backing files. The OS then spends the next 30 seconds, or more, paging it back in -- a period of time my users describe as "eternity." Is there anyway for my application to send a "hint" or command to the VM manager that will cause it to reload the recently paged out memory pages? I have a supervisory process (a scheduler daemon) that can do this after the process terminates. This email sent to site_archiver(a)lists.apple.com

1 0

Reload VM hint?
by James Bucanek 30 May '07

30 May '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Greetings, -- James Bucanek _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I'm hoping this is the correct list. It sounds like a Kernel issue. Feel free to enlighten me if it's not. I'm looking for a way to reload pages that have been paged out by another process. They scenario is pretty simple: I've written an application that uses a lot (100s of MB) of RAM. Users run the application (it's a backup utility). Applications does its thing and terminates. Users now complain that their system is "sluggish and unresponsive." The problem, obviously, is that as the application ran it forced just about everything else in RAM to be paged out to the VM backing files. The OS then spends the next 30 seconds, or more, paging it back in -- a period of time my users describe as "eternity." Is there anyway for my application to send a "hint" or command to the VM manager that will cause it to reload the recently paged out memory pages? I have a supervisory process (a scheduler daemon) that can do this after the process terminates. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Strange iTune directory/file access paths
by Jonas Maebe 29 May '07

29 May '07

site_archiver(a)lists.apple.com Delivered-To: Darwin-kernel(a)lists.apple.com On 29 May 2007, at 22:49, Todd Heberlein wrote: You may want to read this: http://developer.apple.com/qa/qa2001/qa1113.html Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I have been writing a tool (called Audit Navigator) to analyze Apple's BSM audit trails, and the strange behavior that I am analyzing today is iTunes calling getattrlist() on some interesting file paths such as: /.vol/234881026/7190111 This email sent to site_archiver(a)lists.apple.com

1 0

Re: Strange iTune directory/file access paths
by Ronnie Misra 29 May '07

29 May '07

site_archiver(a)lists.apple.com Delivered-To: Darwin-kernel(a)lists.apple.com The following technical Q&A page should answer your questions. <http://developer.apple.com/qa/qa2001/qa1113.html> Ronnie On May 29, 2007, at 1:49 PM, Todd Heberlein wrote: Sorry to bug this list again. This is a low-priority question... [yavin:/] heberlei% cd /.vol/234881026 [yavin:/.vol/234881026] heberlei% ls -la total 2 dr-xr-xr-x 2 root wheel 64 May 29 10:08 . dr-xr-xr-x 2 root wheel 128 May 29 10:08 .. [yavin:/.vol/234881026] heberlei% But I can still do an "ls" on the file 7190111 and find something: [yavin:/.vol/234881026] heberlei% ls -l 7190111 -rw-rw-r-- 1 root admin 508448 Mar 14 19:36 7190111 [yavin:/.vol/234881026] heberlei% Thanks, Todd _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I have been writing a tool (called Audit Navigator) to analyze Apple's BSM audit trails, and the strange behavior that I am analyzing today is iTunes calling getattrlist() on some interesting file paths such as: /.vol/234881026/7190111 Out of curiosity I went to the /.vol/234881026 directory using the Terminal application to see what other files besides 7190111 were in the directory. The "ls" command showed no files in the directory (except . and ..): In other words, "ls -la" shows no files in the directory, but if I look for a particular file (i.e., "ls -l 7190111") "ls" clearly shows something can be found. Also, interestingly, using the Finder's "Go to Folder..." command, it says the folder /.vol/ 234881026 does *not* exist. Does anyone know why "ls" on the Mac behaves this way (i.e., showing nothing in the directory but being able to do an "ls" in a particular filename in the directory)? Is this common on other UNIX platforms (Linux, Solaris, BSD, ...) outside the machine being root-shelled? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/rgmisra% 40apple.com This email sent to rgmisra(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Strange iTune directory/file access paths
by Todd Heberlein 29 May '07

29 May '07

site_archiver(a)lists.apple.com Delivered-To: Darwin-kernel(a)lists.apple.com Sorry to bug this list again. This is a low-priority question... [yavin:/] heberlei% cd /.vol/234881026 [yavin:/.vol/234881026] heberlei% ls -la total 2 dr-xr-xr-x 2 root wheel 64 May 29 10:08 . dr-xr-xr-x 2 root wheel 128 May 29 10:08 .. [yavin:/.vol/234881026] heberlei% But I can still do an "ls" on the file 7190111 and find something: [yavin:/.vol/234881026] heberlei% ls -l 7190111 -rw-rw-r-- 1 root admin 508448 Mar 14 19:36 7190111 [yavin:/.vol/234881026] heberlei% Thanks, Todd _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… I have been writing a tool (called Audit Navigator) to analyze Apple's BSM audit trails, and the strange behavior that I am analyzing today is iTunes calling getattrlist() on some interesting file paths such as: /.vol/234881026/7190111 Out of curiosity I went to the /.vol/234881026 directory using the Terminal application to see what other files besides 7190111 were in the directory. The "ls" command showed no files in the directory (except . and ..): In other words, "ls -la" shows no files in the directory, but if I look for a particular file (i.e., "ls -l 7190111") "ls" clearly shows something can be found. Also, interestingly, using the Finder's "Go to Folder..." command, it says the folder /.vol/234881026 does *not* exist. Does anyone know why "ls" on the Mac behaves this way (i.e., showing nothing in the directory but being able to do an "ls" in a particular filename in the directory)? Is this common on other UNIX platforms (Linux, Solaris, BSD, ...) outside the machine being root-shelled? This email sent to site_archiver(a)lists.apple.com

1 0

glibc (makecontext) port (user context switching)
by James Shimer 28 May '07

28 May '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Has there been a port of glibc or makcontext, getcontex, swapcontext? I'm trying to port an application that uses these APIs to perform user space context switching. Any ideas of similar threading APIs to create and swap in and out a thread context? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: System.framework/PrivateHeaders
by Thomas Inskip 23 May '07

23 May '07

site_archiver(a)lists.apple.com Delivered-To: darwin-kernel(a)lists.apple.com Dkim-signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:in-reply-to:references:mime-version:content-type:message-id:cc:from:subject:date:to:x-mailer; b=hAOZXpVyKzLzNnGcPWC2sZIThGPn7VGHbe+TaGnvR0iz4xEOAWujDfqD5oiHLiVDlVGNnuYU3lf9dpWX1c538uf7gRlKtWeOolclIWCDQ0ulGV7pfSyhxqdwzMqAqAE/zNGiyqqohoGWODNsgTdBZ2WH7kOa4cgeXLPMkn3F8nY= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:in-reply-to:references:mime-version:content-type:message-id:cc:from:subject:date:to:x-mailer; b=Krb28+bOYx/IK2OcnYw9U3J35/sD32QeNuYNhf+Na6HqaKVZoJoGs8+1zW+Hu6/+laaev7a6zmLn5fmMvvqJd5lVEWmKxLehxUREcU5lFTBc/ZHq0h9Z1bxu5fDmaq0OOX05ajzpK79zHLsdFtAdxeQdrfgGcc17OGGe0xCbfg0= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… This email sent to site_archiver(a)lists.apple.com

1 0

Re: /dev/null and execve()
by Terry Lambert 22 May '07

22 May '07

site_archiver(a)lists.apple.com Delivered-To: Darwin-kernel(a)lists.apple.com Here is the BSM audit record from a PowerPC showing the execve: -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-kernel mailing list (Darwin-kernel(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-kernel/site_archiver%40lists.… On May 21, 2007, at 10:16 PM, Todd Heberlein wrote: I apologize if this is a bonehead question, but why is the Mac doing an execve() on "/dev/null"? header,123,1,execve(2),0,Mon May 21 09:30:23 2007, + 431 msec path,/dev/null path,/dev/null attribute,20666,root,wheel,46807492,0,50331650 subject,heberlei,heberlei,staff,heberlei,staff, 452,384,50331650,0.0.0.0 return,success,0 trailer,123 It's not executing it, it's opening /dev/null so that fd's 0, 1, and 2 are allocated for you, in case you are running an SUID program and forgot to be as careful with your coding as you should have been. If you look at the Darwin sources, you will see this happening in xnu.bsd/kern_exec.c in the function exec_handle_sugid(). This email sent to site_archiver(a)lists.apple.com

1 0