On Wednesday, November 13, 2002, at 09:13 AM, Tyler Ward wrote: There are a few things to be noticed here. First of all, it's performing lots of superfluous opening and closing of "outfile.sah". But by lots, that's only one a second or so, completely insignificant. In fact, there are large time periods (up to two seconds) where the program isn't making any syscalls. Consequently it is pretty clear that syscalls aren't the cause of this. While this was running it was still using 35% or more systime, though it was tracing. I'm ktracing setiathome at this very moment, and typically systime (as reported by top) is 2% or less. It never goes much higher than that unless I start doing other things that would be expected to spike system time. (open a Terminal window, etc.) One thing that could be a problem is this..... PID COMMAND %CPU TIME FAULTS ... BSDSYSCAL ... CSWITCH 5169 top 0.6% 0:00.55 2075 22527 419 5168 setiathome 56.4% 1:01.51 13954 17563 29892 OK, I've dropped fields that I don't think are important (generally very low numbers). This is setiathome after running for only a few seconds, maybe 10 or 20. Notice that it's had 13,000 faults and almost 30,000 context switches. This is running at nice = 20. On my system setiathome is getting 200 to 500 faults per second and 10 to 20 context switches per second (judging by top in delta mode). After four minutes of CPU time (maybe 5 minutes of real time) it looks like this... 5168 setiathome 71.0% 4:04.97 45376 .... 68898 .... 113043 Once again with the same fields dropped. Notice that it's had 45,000 faults, 70,000 syscalls, and 113,000 CSwitches. Here, I get: PID COMMAND %CPU TIME FAULTS ... BSDSYSCALL MACHSYSCALL CSWITCH 688 setiathome 98.9% 4:04.61 85313 ... 5322 1279 7006 Nearly double the faults, but 1/13 as many system calls (assuming you recorded BSD syscalls) and 1/16 the context switches. Now one last bit of information. Here's the top header, not in event mode. Processes: 51 total, 4 running, 47 sleeping... 145 threads 12:02:25 Load Avg: 1.65, 1.44, 1.36 CPU usage: 75.5% user, 24.5% sys, 0.0% idle SharedLibs: num = 42, resident = 16.2M code, 1.40M data, 5.38M LinkEdit MemRegions: num = 3759, resident = 84.4M + 5.05M private, 62.0M shared PhysMem: 41.5M wired, 108M active, 205M inactive, 354M used, 93.5M free VM: 1.37G + 34.0M 15734(0) pageins, 14783(0) pageouts The same: Processes: 50 total, 3 running, 47 sleeping... 152 threads 14:31:02 Load Avg: 1.14, 1.06, 1.00 CPU usage: 51.2% user, 4.2% sys, 44.7% idle SharedLibs: num = 82, resident = 23.3M code, 2.38M data, 5.96M LinkEdit MemRegions: num = 4196, resident = 76.1M + 7.11M private, 114M shared PhysMem: 67.4M wired, 89.9M active, 218M inactive, 376M used, 648M free VM: 2.25G + 49.8M 7966(0) pageins, 0(0) pageouts Note that the 44.7% idle percentage is due to the fact that this is a SMP machine and setiathome is single threaded. What does this tell us? Well first of all, there's no shortage of ram, so there's no reason why seti should be having many faults. Not true. You can, for example, have lots of faults due to memory allocation. When a process asks for memory, the OS claims to give it some but is really only giving it virtual address space. Physical pages aren't allocated until the process tries to touch some of the allocated address space and faults. I believe Mach IPC also uses VM tricks, which may be another source of page faults that aren't actually swapfile use. The page faults are probably the cause of the context switches, and together they're probably the cause of much of the system time. Just a guess. On this system, I got more faults and dramatically fewer context switches. This would tend to rule out faults in and of themselves as the cause of excessive context switches. However, swapfile use certainly could cause extra context switches (since a process waiting for a pagein is blocked), and you are getting a lot of that. The other interesting thing is that there are 70,000 syscalls in the listing, but not nearly that many in the ktrace. ktrace didn't pick up more than say ten syscalls per second, meaning that after 5 minutes there should have been no more than 3,000 syscalls, plus whatever happened while starting the program. So... 70,000 - 18,000 - 3,000 = 49,000 syscalls unaccounted for. So, it looks to me like the performance problem comes from two places. The extra syscalls, and the completely unnecessary page faults. I might even go as far as to say that fixing both of these might give all programs a significant boost across the board. I don't know why you have so many syscalls, but that and the actual swapfile access are where I'd focus my attention. If you can, scrounge up some extra memory and retest. You may find that this is really just a matter of running out of physical RAM at some point in time, forcing the system to swap some stuff out. (roughly 57 megabytes cumulative since turning the system on, going by your pageout count) Another thing to try, if you can, is to do a fresh install of 10.2 on another disk and see how it behaves. (i.e., the "maybe there is something misconfigured somewhere" theory) _______________________________________________ darwin-kernel mailing list | darwin-kernel@lists.apple.com Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-kernel Do not post admin requests to the list. They will be ignored.