January 2005 - Darwin-dev - lists.apple.com

Re: Disk Usage Incorrect [solved]
by Eric Ocean 12 Jan '05

12 Jan '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jan 12, 2005, at 1:58 PM, Jonas Maebe wrote: 7407280 /Applications 1103440 /Applications (Mac OS 9) 0 /Cleanup At Startup 1796 /Desktop DB 2580 /Desktop DF 0 /Desktop Folder 770444 /Developer 9320 /Final Cut Pro 4 /Installer Log File 1025168 /Library 8524 /Local 3 /Network 0 /Network Trash Folder 0 /Scripts 1479368 /System 272556 /System Folder 0 /Temporary Items 4 /TheVolumeSettingsFolder 0 /Trash 100710936 /Users 12 /Volumes 2 /automount 3460 /bin 820 /cdb-0.75 84 /command 125092 /cores 36420 /db-4.2.52 du: /dev/fd/3: Bad file descriptor 20047924 /dev 4 /etc 4 /iNode10616414 4 /mach 560 /mach.sym 3776 /mach_kernel 764284 /opt 10576 /package 121492 /private 2148 /sbin 0 /service 60700 /subversion-1.0.3 4 /tmp 1281980 /usr 4 /var Thanks to everyone who chimed in to help. Best, Eric _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… What does `sudo du -ks /*` say? According to this, /Users is taking up a ton of space. I wonder if that number has improved (become more accurate) after the Disk Utility and fsck stuff? Disk Utility corrected a "volume header" problem. I didn't run du again after fixing that, I just ran OmniDiskSweeper, verified that nothing changed there, and assumed the problem was still there. My bad. FWIW, OmniDiskSweeper reports /Users/anthem to be ~850MB when du says it's really ~23GB. Other user folders have similar discrepancies (though not as large). The problem is not caused by lots of files; the space in question is occupied by music, mostly, that was dropped in the ~/Public/Drop Box folder. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Disk Usage Incorrect
by Jerry Pendergraft 12 Jan '05

12 Jan '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On the point of comparing file size to allocated size I did a quick and dirty perl script to stat files and find out what allocated size was for various file sizes. Got a big surprise when this reported more than a fraction of a block of waste for larger files. And the the minimum allocation seems to be 8 blocks even for the 1 byte file. Does darwin set the stat struct variable st_blocks somehow different than the man page describes as /* blockes allocated for file */ ?? Surely something is fishy here? foreach my $filename (@ARGV) { chomp($filename); my ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size, $atime,$mtime,$ctime,$blksize,$blocks) = stat($filename); my $allocsize = $blksize * $blocks; printf("blksize:%d blocks:%d usage:%d\n", $blksize, $blocks, $allocsize); printf("%-14s size:%d alloc:%d waste:%d\n", $filename, $size, $allocsize, $allocsize - $size); } -- Jerry Pendergraft jerry.pendergraft(a)endocardial.com Endocardial Solutions voice: 651-523-6935 1350 Energy Lane, Suite 110 fax: 651-644-7897 St Paul, MN 55108-5254 mobile: 651-491-0163 On Wed, 12 Jan 2005, Jonas Maebe wrote: > > On 12 jan 2005, at 22:49, Eric Ocean wrote: > > > The number of files is 860,137. I haven't run fsck yet; I guess I'll > > do that now. > > fsck does the same as Disk Utility. > > What does > > sudo du -ks /* > > say? > > > Jonas > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-dev/jerry%40endocardial.com > > This email sent to jerry(a)endocardial.com > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Disk Usage Incorrect
by Jonas Maebe 12 Jan '05

12 Jan '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 12 jan 2005, at 22:49, Eric Ocean wrote: fsck does the same as Disk Utility. What does sudo du -ks /* say? Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… The number of files is 860,137. I haven't run fsck yet; I guess I'll do that now. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Disk Usage Incorrect
by Eric Ocean 12 Jan '05

12 Jan '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jan 12, 2005, at 1:18 PM, Clark Cox wrote: My first question would be how many files do you have on that disk? If it's anywhere near 26 million then that may be the issue. If you estimate that, on average, the difference between the physical and logical sizes of a file is 2K, then that could easily add up to 50 GB of wasted space. Maybe you're comparing apples (logical size) to oranges (physical size). Best, Eric _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… If that's not it, have you run fsck or DiskUtility? The number of files is 860,137. I haven't run fsck yet; I guess I'll do that now. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Disk Usage Incorrect
by Eric Ocean 12 Jan '05

12 Jan '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Yes, the disk is fine (according to Disk Utility). Best, Eric _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Have you tried checking your disk with Disk Utility after booting from CD? This email sent to site_archiver(a)lists.apple.com

1 0

Re: Disk Usage Incorrect
by Clark Cox 12 Jan '05

12 Jan '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=pkJBtHBBNQhcXuTeEXxrnfRfHE51+SJAjyMRX+AyLruhxDhp1IqtuTOSz3j1yGk3GmECHHmKtIPcIGhU0sE1pWQKWFCK9J2goBchLQV2Z2h+EUBlY+ONMK5bzARdjyASzVcegpa7+jT+f54gbNoN5Atwlki0g0hmr8njItuCS2M= My first question would be how many files do you have on that disk? If it's anywhere near 26 million then that may be the issue. If you estimate that, on average, the difference between the physical and logical sizes of a file is 2K, then that could easily add up to 50 GB of wasted space. Maybe you're comparing apples (logical size) to oranges (physical size). If that's not it, have you run fsck or DiskUtility? On Wed, 12 Jan 2005 12:36:56 -0800, Eric Ocean <subscriber(a)ampede.com> wrote: > I have an approximately 110 GB drive. Both the Finder and the command > line show that I've used all but about 1.2 GB; however, when I add up > the file sizes individually, they only add up to about 62GB. I've tried > determining this using a number of different methods (du, > OmniDiskSweeper, etc.) and they all report ~110GB of used space, and > ~62GB of actual space used by the files on my system. > > At this point, I don't know how to proceed any further. I either need > to determine what is taking up the remaining 48GB of missing space, or > determine why all of my tools think I'm using 110GB of space. > > I've searched far and wide on Google to no avail; perhaps I'm using the > wrong terms. > > Best, Eric > > P.s. The link is a screenshot (12KB) from OmniDiskSweeper, which > clearly shows the problem: http://www.ampede.com/disk-space.png > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-dev/clarkcox3%40mac.com > > This email sent to clarkcox3(a)mac.com > -- Clark S. Cox III clarkcox3(a)gmail.com http://www.livejournal.com/users/clarkcox3/ http://homepage.mac.com/clarkcox3/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Disk Usage Incorrect
by Michael Hogsett 12 Jan '05

12 Jan '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Mozilla Thunderbird 1.0 (Macintosh/20041206) Best, Eric I may be speaking out of my rear, but I believe the smallest allocation block on an HFS partition increases as the size of the partition increases. On my 80Gbyte disk a 1byte file (yes 1 byte) consumes 4Kbytes. The Finder's 'Get Info' reports '4KB on disk (1 byte)' If your 110 Gbyte drive contains a large number of small files that is a great deal of wasted space. Also note, that this means that a 4097 byte file will consume 8192 bytes (2 4k disk allocation blocks). - Mike _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Eric Ocean wrote: I have an approximately 110 GB drive. Both the Finder and the command line show that I've used all but about 1.2 GB; however, when I add up the file sizes individually, they only add up to about 62GB. I've tried determining this using a number of different methods (du, OmniDiskSweeper, etc.) and they all report ~110GB of used space, and ~62GB of actual space used by the files on my system. At this point, I don't know how to proceed any further. I either need to determine what is taking up the remaining 48GB of missing space, or determine why all of my tools think I'm using 110GB of space. I've searched far and wide on Google to no avail; perhaps I'm using the wrong terms. P.s. The link is a screenshot (12KB) from OmniDiskSweeper, which clearly shows the problem: http://www.ampede.com/disk-space.png This email sent to site_archiver(a)lists.apple.com

1 0

Re: Disk Usage Incorrect
by Jonas Maebe 12 Jan '05

12 Jan '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 12 jan 2005, at 21:36, Eric Ocean wrote: Jonas _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… At this point, I don't know how to proceed any further. I either need to determine what is taking up the remaining 48GB of missing space, or determine why all of my tools think I'm using 110GB of space. Have you tried checking your disk with Disk Utility after booting from CD? This email sent to site_archiver(a)lists.apple.com

1 0

Disk Usage Incorrect
by Eric Ocean 12 Jan '05

12 Jan '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Best, Eric _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I have an approximately 110 GB drive. Both the Finder and the command line show that I've used all but about 1.2 GB; however, when I add up the file sizes individually, they only add up to about 62GB. I've tried determining this using a number of different methods (du, OmniDiskSweeper, etc.) and they all report ~110GB of used space, and ~62GB of actual space used by the files on my system. At this point, I don't know how to proceed any further. I either need to determine what is taking up the remaining 48GB of missing space, or determine why all of my tools think I'm using 110GB of space. I've searched far and wide on Google to no avail; perhaps I'm using the wrong terms. P.s. The link is a screenshot (12KB) from OmniDiskSweeper, which clearly shows the problem: http://www.ampede.com/disk-space.png This email sent to site_archiver(a)lists.apple.com

1 0

Re: Stack trace queryable?
by Sam Vaughan 07 Jan '05

07 Jan '05

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 07/01/2005, at 7:05 AM, Greg Hurrell <greg(a)hurrell.net> wrote: El 04/01/2005, a las 23:27, Sam Vaughan escribió: I haven't looked at the MoreBacktrace code, but you should be able to generate backtraces pretty simply with the following code: Thanks a lot, Sam. The MoreBacktrace sample uses the same assembly to get the stack pointer: #define MoreBTPPCGetStackPointer(result) \ __asm__ volatile("mr %0,r1" : "=r" (result)); One thing that interests me is that both implementations seem to be designed to get the stack trace of the current thread. I wonder what would be necessary to get a stack trace of all threads in a running process. Sam _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… But the code is a lot more complicated than yours. Well it was probably written by someone like Quinn who knew what they were doing and guarded themselves better against all the possible pitfalls ;-) I wrote that quickly, found that it worked and have only used it in the narrow environment of kernel debug code, so be careful. If it's user space threads you're interested in, then you should be able to start with the source code to "top". I seem to recall there are some useful user space APIs being used there to get thread information, although program counter addresses might be asking too much. I guess what you're really looking for is the source code to Sampler.app. I wonder if there's an open source equivalent. I wrote a tool a while ago to get symbolic backtraces on all the running kernel threads on a live system. It's very handy for quickly finding deadlocked kernel threads without setting up dual machine debugging - ideal for deadlocked test machines on the other side of the world. It uses a kext made up of two halves, one compiled against Mach headers and the other against BSD. The BSD side contains the sysctl interface and memory allocation code, and the Mach side walks the running threads and pulls out the program counters and stack traces. It works, but I think it'll become unmaintainable in 10.4. I haven't got around to formally requesting an API for it yet... This email sent to site_archiver(a)lists.apple.com

1 0