July 2002 - Darwin-kernel - lists.apple.com

Re: How do disk writes work these days?
by Pelle Johansson 30 Jul '02

30 Jul '02

torsdagen den 25 juli 2002 kl 03.15 skrev Wade Tregaskis: Is this still an efficient way of operating, in an increasingly linear [e. g. single-user] OS? I'm just thinking, if you could return from writes faster, the calling process can keep doing other things. My thought is that you could do this without breaking existing apps by performing any error checking [i.e. quickly verifying there is free space, among other things] immediately, returning fail if appropriate, or otherwise returning no error. You could then worry about physically getting the data to disk, seeing as this is the most time intensive part... Side note: I was taught that if you know how large a file you want to write you should use ftruncate() to set the file size before writing. That way the OS can optimize the location of the file on the hard drive. I've noticed Unix application are quite bad at doing this, so perhaps it doesn't apply in some OSes. -- Pelle Johansson <morth(a)morth.nu> _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: How do disk writes work these days?
by Pelle Johansson 30 Jul '02

30 Jul '02

tisdagen den 30 juli 2002 kl 17.15 skrev Quinn: At 16:38 +0200 30/7/02, Pelle Johansson wrote: IMO, a much better way, in programs where it is possible, is to use O_NONBLOCK (in either open() or fcntl()) and select() to check for writable file descriptors. [...] Last I checked file descriptors that refer to files always trigger select to return immediately (at least an Darwin systems). The select system call eventually calls down to a file-system specific routine (eg hfs_select, nfs_select), which always returns true. Take a look... Well, that's quite ok, as long as the writes are indeed not blocking (nor returning EAGAIN). -- Pelle Johansson <morth(a)morth.nu> _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

ps v. top
by David Blanton 30 Jul '02

30 Jul '02

a cfm app shows in ps as LaunchCFMApp in top it shows as app_name where is the app_name found when uap->fname in execve is ../LaunchCFMApp _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: How do disk writes work these days?
by Justin C. Walker 30 Jul '02

30 Jul '02

On Tuesday, July 30, 2002, at 07:38 AM, Pelle Johansson wrote: torsdagen den 25 juli 2002 kl 11.04 skrev Timothy A. Seufert: Wade Tregaskis <wjtregaskis(a)students.latrobe.edu.au> wrote: Wade -- one thing you might want to look in to is that some operating systems implement an asynchronous I/O API in addition to traditional blocking synchronous I/O. For asynch, the difference in the sequence of events (from userland POV) is that the initial I/O call doesn't block, and there is always some mechanism to notify the process of I/O completion. In "classic" MacOS, ISTR this was done by giving the OS a pointer to a callback function, which would be called on error or completion. I think it also set a flag in the descriptor structure used to request the I/O, so you could also determine completion by polling that flag. Is this still an efficient way of operating, in an increasingly linear [e. g. single-user] OS? I'm just thinking, if you could return from writes faster, the calling process can keep doing other things. The classic way of doing this in UNIX systems is to make the application multithreaded. By splitting things into an I/O thread and a (other stuff) thread, you can avoid making the whole program stop every time it needs to do I/O. IMO, a much better way, in programs where it is possible, is to use O_NONBLOCK (in either open() or fcntl()) and select() to check for writable file descriptors. Most programs should in fact be centered around a select() call. If you wish, you can check for SIGIO as well, but it is usually not necessary. O_NONBLOCK and select() are not very useful in writing to files; the kernel will, on the user's thread, copy the data into the kernel (or locate the page in the page buffer, depending on details of the call and the user's buffer layout), and then return to user mode. Writes to a file are (already) essentially asynchronous. FreeBSD (and other Unix/Unix-like OS's) have "asynch I/O" functionality, but for this specific use, it's overly complicated for no real gain. Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | Men are from Earth. | Women are from Earth. | Deal with it. *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: How do disk writes work these days?
by Quinn 30 Jul '02

30 Jul '02

At 16:38 +0200 30/7/02, Pelle Johansson wrote: IMO, a much better way, in programs where it is possible, is to use O_NONBLOCK (in either open() or fcntl()) and select() to check for writable file descriptors. [...] Last I checked file descriptors that refer to files always trigger select to return immediately (at least an Darwin systems). The select system call eventually calls down to a file-system specific routine (eg hfs_select, nfs_select), which always returns true. Take a look... <http://www.opensource.apple.com/cgi-bin/registered/cvs/src/notlive/xnu/bsd/…> <http://www.opensource.apple.com/cgi-bin/registered/cvs/src/notlive/xnu/bsd/…> S+E -- Quinn "The Eskimo!" <http://www.apple.com/developer/> Apple Developer Technical Support * Networking, Communications, Hardware _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: How do disk writes work these days?
by Pelle Johansson 30 Jul '02

30 Jul '02

torsdagen den 25 juli 2002 kl 11.04 skrev Timothy A. Seufert: Wade Tregaskis <wjtregaskis(a)students.latrobe.edu.au> wrote: Wade -- one thing you might want to look in to is that some operating systems implement an asynchronous I/O API in addition to traditional blocking synchronous I/O. For asynch, the difference in the sequence of events (from userland POV) is that the initial I/O call doesn't block, and there is always some mechanism to notify the process of I/O completion. In "classic" MacOS, ISTR this was done by giving the OS a pointer to a callback function, which would be called on error or completion. I think it also set a flag in the descriptor structure used to request the I/O, so you could also determine completion by polling that flag. Is this still an efficient way of operating, in an increasingly linear [e. g. single-user] OS? I'm just thinking, if you could return from writes faster, the calling process can keep doing other things. The classic way of doing this in UNIX systems is to make the application multithreaded. By splitting things into an I/O thread and a (other stuff) thread, you can avoid making the whole program stop every time it needs to do I/O. IMO, a much better way, in programs where it is possible, is to use O_NONBLOCK (in either open() or fcntl()) and select() to check for writable file descriptors. Most programs should in fact be centered around a select() call. If you wish, you can check for SIGIO as well, but it is usually not necessary. -- Pelle Johansson <morth(a)morth.nu> _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: loop device support in darwin?
by Kevin Van Vechten 29 Jul '02

29 Jul '02

Hi Nathan, DiskCopy and the disk image framework it uses are not presently available in Darwin. If you would like to see it available as part of Darwin, please voice your opinion on the "Request for Open Source" thread on the darwin-development(a)lists.apple.com list. -Kevin Nathan T. Hjelm <hjelmn(a)cs.unm.edu> wrote: > I switched to darwin from linux and would have been trying to figure out > how to mount a disk image without using the disk copy tool provided with > macos 10. Is there support for a loop device in the darwin kernel? If > there isn't, is there any plan to add support for it? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

loop device support in darwin?
by Nathan T. Hjelm 29 Jul '02

29 Jul '02

I switched to darwin from linux and would have been trying to figure out how to mount a disk image without using the disk copy tool provided with macos 10. Is there support for a loop device in the darwin kernel? If there isn't, is there any plan to add support for it? _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Mach VM management question
by Pejvan BEIGUI 29 Jul '02

29 Jul '02

Resent with inline attachment... >> Hi guys, >> Thank you, Jim and Justin, for you kind replies >> I have a question regarding the VM management in Mach/Darwin. >> Since I don't like to reboot my mac, and since I use a lot of different >> applications, not at the same time, I'm surprised to see that the VM >> storage >> on my drives keeps going up, and never fall down. >> >> I have a tibook 500MHz, with 512MB of RAM, so I don't think that it's >> normal >> to have so many VM used at the same time on my machine : >> >>> ...[list of swap files deleted]... >> >> So, is there a way to tell the system to update the VM ? Or tell him to >> remove the unused VM swapfiles ? > > It happens automatically as the space is released. But this is > controlled by a user-level daemon process (dynamic_pager). If that > got killed off somehow, you would neither free up any unused ones or > allocate any new ones. While I've never seen that happen, its > something to look for in extremely weird cases. > Ok, it's not that, since my VM goes up very easily ;-) Thanks for the info though :-) >> Plus here's the output of my top : >> >>> [... Top outpout deleted ...] >> >> So what is this kernel_tasks using 400M VSize ? > > The kernel allocates some virtual address space (space in submaps, not > actual memory) for efficient future memory allocation and wiring later > on. Top picks that up and reports it. Kernel leaks would most likely > cause a rise in wired memory. With 512MB of memory, the 59.6MB of > wired memory is not a major concern (some wired data structure counts > scale up with amount of physical memory). > > There are uses of true virtual (pageable) memory within the kernel. > Specifically, some of the (not currently in-use) texture maps for > graphics acceleration are stored in kernel pageable memory. And some > (sent, but not-yet-received) Mach IPC data is stored in pagable kernel > memory. > > There is no way to distinguish between over-commit virtual address > space allocations in the kernel, and true pageable memory allocations > using top. So we can't tell how much of that 400MB of kernel virtual > falls into each of the two categories above. In fact, it isn't > possible to tell that for any other task/process either. Since your > maximum "dirty virtual" memory size of ~1GB (7*76MB swap files + 512MB > of physical memory) is less than the 1.69GB of all virtual allocations > reported by top (even when you remove the 400MB of potentially unused > space in the kernel), it may be correct. > Thanks for the explanation > Your top says there are 55 processes, and you only show 14 of them. Is > there anything else of interest in those other 41 processes? > below, you'll find the outuput of "top -l -n 60" Processes: 57 total, 2 running, 55 sleeping... 158 threads 04:43:27 Load Avg: 0.51, 0.51, 0.57 CPU usage: 7.3% user, 17.9% sys, 74.8% idle SharedLibs: num = 126, resident = 14.8M code, 376K data, 2.20M LinkEdit MemRegions: num = 9525, resident = 124M + 4.51M private, 42.4M shared PhysMem: 59.8M wired, 241M active, 205M inactive, 505M used, 6.41M free VM: 1.95G + 56.4M 264471(0) pageins, 643124(0) pageouts PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE 5979 top 13.0% 0:00.43 1 14 18 420K+ 308K 676K+ 1.82M+ 73 Window Man 2.9% 4:04:41 3 2744 3426 3.65M 26.3M 29.6M 127M 328 Terminal 2.1% 22:18.23 7 131 881 3.26M 8.25M 8.15M 68.9M 289 java 0.7% 19:00.04 26 205 181 700K 272K 728K 171M 5857 Net Monito 0.7% 5:34.53 3 73 134 1.13M 2.86M 2.18M 58.0M 5957 mldonkey 0.7% 2:00.72 1 16 74 14.1M 984K 15.0M 16.0M 5853 DragThing 0.7% 1:59.85 2 81 279 2.86M 4.27M 4.17M 62.2M 5595 imserviced 0.7% 0:26.04 1 22 35 272K 372K 668K 3.07M 0 kernel_tas 0.0% 1:44:06 26 0 - - - 48.0M+ 400M 5583 Proteus 0.0% 37:48.32 5 145 960 40.9M 6.46M 43.2M 233M 291 loginwindo 0.0% 24:29.78 7 151 183 2.02M 3.11M 2.98M 46.7M 327 SystemUISe 0.0% 16:39.92 4 138 237 1.67M 3.82M 2.30M 60.9M 133 configd 0.0% 14:07.55 4 110 173 364K 376K 576K 4.29M 5584 imservices 0.0% 6:03.19 4 32 678 30.6M 900K 30.8M 151M 2431 Dock 0.0% 4:24.34 3 112 209 1.19M 5.41M 2.50M 56.6M 330 MicrosoftM 0.0% 3:32.92 1 77 37 152K 496K 260K 26.1M 3323 Microsoft 0.0% 2:42.81 2 75 143 2.76M 6.36M 3.30M 58.3M 333 MicrosoftK 0.0% 2:34.48 1 53 50 168K 1.25M 276K 38.7M 5594 imserviced 0.0% 2:21.82 1 22 38 1.72M 816K 2.41M 5.74M 71 ATSServer 0.0% 2:09.11 1 37 78 1.01M 2.80M 2.03M 66.7M 277 openmonito 0.0% 2:00.33 1 9 20 28K 308K 104K 1.64M 75 update 0.0% 1:34.06 1 8 13 12K 268K 44K 1.25M 419 SecurityAg 0.0% 0:54.08 4 124 170 1.67M 4.74M 3.67M 59.6M 447 ntpd 0.0% 0:49.48 1 9 15 68K 384K 192K 1.49M 5875 Finder 0.0% 0:47.09 2 88 294 12.3M 8.54M 14.1M 76.9M 229 coreservic 0.0% 0:34.26 3 62 78 784K 380K 732K 4.75M 4894 nmbd 0.0% 0:31.95 1 16 17 120K 520K 412K 2.08M 248 httpd 0.0% 0:26.47 1 9 73 16K 284K 56K 3.45M 2 mach_init 0.0% 0:26.20 1 117 13 40K 284K 92K 1.27M 78 dynamic_pa 0.0% 0:24.56 1 10 15 32K 272K 60K 1.27M 170 sharityd 0.0% 0:21.89 2 11 25 92K 388K 252K 6.59M 257 SecuritySe 0.0% 0:17.97 2 71 24 556K 536K 348K 3.19M 211 lookupd 0.0% 0:17.21 2 28 21 212K 412K 408K 2.39M 107 autodiskmo 0.0% 0:15.43 2 47 19 108K 364K 196K 2.34M 204 netinfod 0.0% 0:10.07 1 8 18 412K 348K 520K 1.79M 321 pbs 0.0% 0:09.08 1 30 44 116K 416K 100K 22.5M 331 Silk 0.0% 0:08.56 1 57 69 624K 2.42M 840K 49.9M 1976 imserviced 0.0% 0:05.68 1 23 32 0K 816K 328K 4.01M 282 openinfo 0.0% 0:03.30 4 16 28 100K 352K 272K 2.33M 159 syslogd 0.0% 0:03.16 1 8 13 40K 276K 88K 1.26M 41 kextd 0.0% 0:03.02 2 16 338 0K 264K 24K 6.44M 270 oplrqb 0.0% 0:02.18 1 9 16 40K 296K 108K 1.82M 267 cron 0.0% 0:01.97 1 9 15 32K 292K 96K 1.52M 996 System Eve 0.0% 0:01.57 1 58 62 532K 2.43M 756K 48.6M 263 sshd 0.0% 0:01.22 1 9 15 76K 420K 176K 1.56M 318 databackup 0.0% 0:00.38 1 14 26 128K 328K 280K 1.88M 286 sh 0.0% 0:00.01 1 16 13 0K 584K 16K 1.69M 275 openexec 0.0% 0:00.96 4 17 29 200K 324K 344K 3.03M 271 www_sv 0.0% 0:00.02 1 16 17 0K 264K 16K 2.04M 379 automount 0.0% 0:00.20 2 11 18 56K 348K 176K 2.14M 250 httpd 0.0% 0:00.01 1 9 73 0K 284K 40K 3.45M 236 inetd 0.0% 0:00.10 1 9 13 0K 264K 24K 1.26M 182 CrashRepor 0.0% 0:00.16 1 61 16 212K 284K 276K 1.57M 5913 tcsh 0.0% 0:00.23 1 24 16 564K 576K 760K 5.82M 5926 tcsh 0.0% 0:00.31 1 24 16 568K 576K 996K 5.82M 5958 telnet 0.0% 0:00.03 1 13 15 100K 340K 332K 1.46M 1 init 0.0% 0:00.11 1 20 12 24K 288K 52K 1.26M 0 idle_threa 66.7% 90:40:40 All the processes are there. Unfortunately, I had to reboot after saving this data, since AppleTalk went nuts ;-) BTW, yes, I have an MS Keyboard, and an MS Mouse, please don't blame me for that ;-) Pejvan _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0

Re: Mach VM management question
by Justin C. Walker 29 Jul '02

29 Jul '02

Sadly, the list manager does not let attachments through, for reasons of sanity and health. You'll have to paste it... Regards, Justin On Monday, July 29, 2002, at 11:04 , Pejvan BEIGUI wrote: Hi guys, Thank you, Jim and Justin, for you kind replies [snip] Your top says there are 55 processes, and you only show 14 of them. Is there anything else of interest in those other 41 processes? As an attachment, you'll find the outuput of "top -l -n 60" All the processes are there. Unfortunately, I had to reboot after saving this data, since AppleTalk went nuts ;-) BTW, yes, I have an MS Keyboard, and an MS Mouse, please don't blame me for that ;-) This probably explains everything :-}. Don't ever use the "microsoft" key... Regards, Justin -- Justin C. Walker, Curmudgeon-At-Large * Institute for General Semantics | It's not whether you win or lose... | It's whether *I* win or lose. *--------------------------------------*-------------------------------* _______________________________________________ darwin-kernel mailing list | darwin-kernel(a)lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.

1 0