November 2009 - Darwin-dev - lists.apple.com

Re: file offset of __DATA, __bss and __common
by Shantonu Sen 05 Nov '09

05 Nov '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com <http://developer.apple.com/mac/library/documentation/DeveloperTools/Concept…> Shantonu Sen ssen(a)apple.com Sent from my Mac Pro On Nov 5, 2009, at 7:57 AM, Rustam Muginov wrote: > Had you tried "otool -lV"? > > On Nov 5, 2009, at 6:26 PM, Joel Reymont wrote: > >> Following is the output from 'size -m -l XUL'. For the life of me I cannot figure out the file offset of the __DATA, __bss and __DATA, __common sections. >> >> I tried to use heuristics to divine the file offsets but I seem to end up in the __LINKEDIT segment :-(. >> >> Do I correctly understand that __DATA, __data = __bss + __common? >> >> Thanks, Joel >> >> --- >> >> Segment __TEXT: 18673664 (vmaddr 0x0 fileoff 0) >> Section __text: 13477360 (addr 0x1470 offset 5232) >> Section __symbol_stub1: 9234 (addr 0xcdba60 offset 13482592) >> Section __stub_helper: 27718 (addr 0xcdde72 offset 13491826) >> Section __const: 1119212 (addr 0xce4ac0 offset 13519552) >> Section __cstring: 436667 (addr 0xdf5eb0 offset 14638768) >> Section __dof_mozilla: 2316 (addr 0xe60870 offset 15075440) >> Section __gcc_except_tab: 72218 (addr 0xe6117c offset 15077756) >> Section __unwind_info: 224100 (addr 0xe72bbc offset 15150012) >> Section __eh_frame: 3299552 (addr 0xea9720 offset 15374112) >> total 18668377 >> Segment __DATA: 1814528 (vmaddr 0x11cf000 fileoff 18673664) >> Section __nl_symbol_ptr: 1136 (addr 0x11cf000 offset 18673664) >> Section __la_symbol_ptr: 12312 (addr 0x11cf470 offset 18674800) >> Section __dyld: 16 (addr 0x11d2488 offset 18687112) >> Section __mod_init_func: 1504 (addr 0x11d2498 offset 18687128) >> Section __const: 1567536 (addr 0x11d2a80 offset 18688640) >> Section __cfstring: 3232 (addr 0x13515b0 offset 20256176) >> Section __objc_data: 1760 (addr 0x1352250 offset 20259408) >> Section __objc_msgrefs: 10208 (addr 0x1352930 offset 20261168) >> Section __objc_selrefs: 368 (addr 0x1355110 offset 20271376) >> Section __objc_classrefs: 1528 (addr 0x1355280 offset 20271744) >> Section __objc_superrefs: 160 (addr 0x1355878 offset 20273272) >> Section __objc_const: 12912 (addr 0x1355918 offset 20273432) >> Section __objc_classlist: 176 (addr 0x1358b88 offset 20286344) >> Section __objc_catlist: 56 (addr 0x1358c38 offset 20286520) >> Section __objc_protolist: 24 (addr 0x1358c70 offset 20286576) >> Section __objc_imageinfo: 8 (addr 0x1358c88 offset 20286600) >> Section __data: 106632 (addr 0x1358ca0 offset 20286624) >> Section __bss: 61412 (addr 0x1372d40 offset 0) >> Section __common: 29384 (addr 0x1381d40 offset 0) >> total 1810364 >> Segment __LINKEDIT: 19382272 (vmaddr 0x138a000 fileoff 20393984) >> total 39870464 >> >> --- >> firefox for android! >> http://wagerlabs.com >> >> _______________________________________________ >> Do not post admin requests to the list. They will be ignored. >> Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) >> Help/Unsubscribe/Update your Subscription: >> http://lists.apple.com/mailman/options/darwin-dev/rmuginov%40gmail.com >> >> This email sent to rmuginov(a)gmail.com > > -- > Sincerely, Rustam Muginov > > > > _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-dev/ssen%40apple.com > > This email sent to ssen(a)apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: file offset of __DATA, __bss and __common
by Rustam Muginov 05 Nov '09

05 Nov '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=roXuL7QAZWgw8CfdJADSC73QR+/w/4wvuq/3+lz2aPE=; b=OGqn14B8dfrHJzRIBlym+6iqpSLuHZkY39xHsLFr/uY0Ei9+XHIDgWmjY7arDGOf5a JKUvxu65rDBc6ko6PweHoAByCyzAxIZ0H/27BZkn82rQVzGbgfsmtshZAKiJI9Qc4NUk W8yXIwRm0MX2SjoY/bVaZCwsSdhb8max67N3g= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=O+VKDSysWpKtx92Lwk5yCowYO5x/UVVCVq5+NxN8MgL+utkwIpv+xkjAdrtKOHuACk tTUEaUAKUv49WhHFfoWecimr05IY0/W8acKGRd4V6ixqoovmrRD724gYJqIgDKuJcRkQ Gtn1hHUqfrWdnF5HCzrU1AlSp/7NXzpSH/KLk= Had you tried "otool -lV"? On Nov 5, 2009, at 6:26 PM, Joel Reymont wrote: Do I correctly understand that __DATA, __data = __bss + __common? Thanks, Joel --- Segment __TEXT: 18673664 (vmaddr 0x0 fileoff 0) Section __text: 13477360 (addr 0x1470 offset 5232) Section __symbol_stub1: 9234 (addr 0xcdba60 offset 13482592) Section __stub_helper: 27718 (addr 0xcdde72 offset 13491826) Section __const: 1119212 (addr 0xce4ac0 offset 13519552) Section __cstring: 436667 (addr 0xdf5eb0 offset 14638768) Section __dof_mozilla: 2316 (addr 0xe60870 offset 15075440) Section __gcc_except_tab: 72218 (addr 0xe6117c offset 15077756) Section __unwind_info: 224100 (addr 0xe72bbc offset 15150012) Section __eh_frame: 3299552 (addr 0xea9720 offset 15374112) total 18668377 Segment __DATA: 1814528 (vmaddr 0x11cf000 fileoff 18673664) Section __nl_symbol_ptr: 1136 (addr 0x11cf000 offset 18673664) Section __la_symbol_ptr: 12312 (addr 0x11cf470 offset 18674800) Section __dyld: 16 (addr 0x11d2488 offset 18687112) Section __mod_init_func: 1504 (addr 0x11d2498 offset 18687128) Section __const: 1567536 (addr 0x11d2a80 offset 18688640) Section __cfstring: 3232 (addr 0x13515b0 offset 20256176) Section __objc_data: 1760 (addr 0x1352250 offset 20259408) Section __objc_msgrefs: 10208 (addr 0x1352930 offset 20261168) Section __objc_selrefs: 368 (addr 0x1355110 offset 20271376) Section __objc_classrefs: 1528 (addr 0x1355280 offset 20271744) Section __objc_superrefs: 160 (addr 0x1355878 offset 20273272) Section __objc_const: 12912 (addr 0x1355918 offset 20273432) Section __objc_classlist: 176 (addr 0x1358b88 offset 20286344) Section __objc_catlist: 56 (addr 0x1358c38 offset 20286520) Section __objc_protolist: 24 (addr 0x1358c70 offset 20286576) Section __objc_imageinfo: 8 (addr 0x1358c88 offset 20286600) Section __data: 106632 (addr 0x1358ca0 offset 20286624) Section __bss: 61412 (addr 0x1372d40 offset 0) Section __common: 29384 (addr 0x1381d40 offset 0) total 1810364 Segment __LINKEDIT: 19382272 (vmaddr 0x138a000 fileoff 20393984) total 39870464 --- firefox for android! http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/rmuginov%40gmail.com -- Sincerely, Rustam Muginov _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Following is the output from 'size -m -l XUL'. For the life of me I cannot figure out the file offset of the __DATA, __bss and __DATA, __common sections. I tried to use heuristics to divine the file offsets but I seem to end up in the __LINKEDIT segment :-(. This email sent to rmuginov(a)gmail.com This email sent to site_archiver(a)lists.apple.com

1 0

file offset of __DATA, __bss and __common
by Joel Reymont 05 Nov '09

05 Nov '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:content-type :content-transfer-encoding:subject:date:message-id:to:mime-version :x-mailer; bh=xbUsC0hDgiVl5BeobnPS9VT5XzJbKjcKAbdZMLnqXlc=; b=N/AafIqHne5XPS3Z2Tkubk/fMr+7I7lvicw+B5LMES44HnlaKbDlzYNhJty+FGccxq 5VATPj1ahddXqhSIWoeD+ttKiE+qbByFFh5rMJYgdENDWbILk0SWNX1bQo18np+k4y4e xQM1Pu/4gF9xQ7fFqWJvB/Smk7UmFzubMgAcs= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:content-type:content-transfer-encoding:subject:date:message-id :to:mime-version:x-mailer; b=WqlUAx8KoxypHuVZB15zZgGI4wYiXfAlW307nUhpV/v72FQ0kleg7kLMSshShNkYiQ kASDG72fSSREVsyDVjatawSqCE5UysJ0hiohpz4lbC1EoO+GtqoROpNs6YaES3CWrZUi sSp8ioAxsBTQKj1cNYVbPEhFCx/G6FI0aYuUo= Do I correctly understand that __DATA, __data = __bss + __common? Thanks, Joel --- Segment __TEXT: 18673664 (vmaddr 0x0 fileoff 0) Section __text: 13477360 (addr 0x1470 offset 5232) Section __symbol_stub1: 9234 (addr 0xcdba60 offset 13482592) Section __stub_helper: 27718 (addr 0xcdde72 offset 13491826) Section __const: 1119212 (addr 0xce4ac0 offset 13519552) Section __cstring: 436667 (addr 0xdf5eb0 offset 14638768) Section __dof_mozilla: 2316 (addr 0xe60870 offset 15075440) Section __gcc_except_tab: 72218 (addr 0xe6117c offset 15077756) Section __unwind_info: 224100 (addr 0xe72bbc offset 15150012) Section __eh_frame: 3299552 (addr 0xea9720 offset 15374112) total 18668377 Segment __DATA: 1814528 (vmaddr 0x11cf000 fileoff 18673664) Section __nl_symbol_ptr: 1136 (addr 0x11cf000 offset 18673664) Section __la_symbol_ptr: 12312 (addr 0x11cf470 offset 18674800) Section __dyld: 16 (addr 0x11d2488 offset 18687112) Section __mod_init_func: 1504 (addr 0x11d2498 offset 18687128) Section __const: 1567536 (addr 0x11d2a80 offset 18688640) Section __cfstring: 3232 (addr 0x13515b0 offset 20256176) Section __objc_data: 1760 (addr 0x1352250 offset 20259408) Section __objc_msgrefs: 10208 (addr 0x1352930 offset 20261168) Section __objc_selrefs: 368 (addr 0x1355110 offset 20271376) Section __objc_classrefs: 1528 (addr 0x1355280 offset 20271744) Section __objc_superrefs: 160 (addr 0x1355878 offset 20273272) Section __objc_const: 12912 (addr 0x1355918 offset 20273432) Section __objc_classlist: 176 (addr 0x1358b88 offset 20286344) Section __objc_catlist: 56 (addr 0x1358c38 offset 20286520) Section __objc_protolist: 24 (addr 0x1358c70 offset 20286576) Section __objc_imageinfo: 8 (addr 0x1358c88 offset 20286600) Section __data: 106632 (addr 0x1358ca0 offset 20286624) Section __bss: 61412 (addr 0x1372d40 offset 0) Section __common: 29384 (addr 0x1381d40 offset 0) total 1810364 Segment __LINKEDIT: 19382272 (vmaddr 0x138a000 fileoff 20393984) total 39870464 --- firefox for android! http://wagerlabs.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Following is the output from 'size -m -l XUL'. For the life of me I cannot figure out the file offset of the __DATA, __bss and __DATA, __common sections. I tried to use heuristics to divine the file offsets but I seem to end up in the __LINKEDIT segment :-(. This email sent to site_archiver(a)lists.apple.com

1 0

Interprocess Locking?
by Duane Murphy 04 Nov '09

04 Nov '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 I'm looking for a good solution for interprocess system wide locking. I have two programs (with multiple users, it could be even more), that need to coordinate handling of a system wide event (disk mounting). I have a solution in mind using named semaphores. Is that the best solution or are there other suggestions or techniques I should look at? Anything I should be aware of using named semaphores? Thanks, ...Duane -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 9.10.0 (Build 500) Charset: US-ASCII wsBVAwUBSvHN/Erg9acQ4r2CAQhNTQgAj8t8/nl7tSSh816wL3B86okhO/8R9r+n E+LdsiMxHYOHvfasXtgbBAkASMBIRNOWM84qCdetrJDL0EIPM4EdzJF3v2TEOWp+ bz7VTSPifhkY5MWjR3DK+fh6ei8AUaCKFef1yHvO4LkOdzp5eYKL/7JWCEyLj9Nz v0lALPpJzUTkPPaaOs4Uj8brUIwSjZ3i0yr2oDOjX0uUdDVxBxqgAsOz4ffRtsit 5I3TEdU3NJhobhiBFDP7/3+WvzkFYUx6pOQRF1Y3DaARtY616o3CPIuYnLio2TPW zcNbcELBQBms6AW662bPUIVakkBHXGiLpqKSNFNlPFJcEz4WZmStVw== =aWEd -----END PGP SIGNATURE----- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Identifying sparse files using getattrlist()
by Mike Mackovitch 04 Nov '09

04 Nov '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Mutt/1.5.20 (2009-06-14) On Wed, Nov 04, 2009 at 09:45:46AM -0600, Peter O'Gorman wrote: > On 11/04/2009 04:12 AM, Alastair Houghton wrote: > >On 4 Nov 2009, at 08:41, rohan a wrote: > > > >>However, for a sparse file this continues to return same sizes for > >>stat size and real size. This is an Apple UFS partition. > > > >Anyway, the answer is that (a) stat() won't return the information you > >want, > > > > Sorry for the stat distraction, I only tried it on this linux system > before making the suggestion: > $ stat sparse (output snipped) > Size: 3221225472 Blocks: 8 > > But I see this morning that when I mount the same file system over > NFS, on my mac: > $ stat -s sparse (output snipped) > st_size=3221225472 st_blocks=6291456 > > Oh well. Yes, this is a known issue in the VFS layer (5633019), and Alastair already pointed it out: On Wed, Nov 04, 2009 at 10:12:42AM +0000, Alastair Houghton wrote: > All of this also explains why stat(), which uses vn_stat() (in > vfs_vnops.c), won't work. stat() asks for va_data_size and > va_total_alloc. UFS doesn't return va_total_alloc, so it's computed > in vnode_getattr() by copying from va_data_size. Hence you're going > to get the same data in both places. Like UFS, NFS also does not return va_total_alloc - only va_data_size and va_data_alloc. HTH --macko _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Identifying sparse files using getattrlist()
by Peter O'Gorman 04 Nov '09

04 Nov '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.4pre) Gecko/20091014 Fedora/3.0-2.8.b4.fc11 Thunderbird/3.0b4 On 4 Nov 2009, at 08:41, rohan a wrote: However, for a sparse file this continues to return same sizes for stat size and real size. This is an Apple UFS partition. Anyway, the answer is that (a) stat() won't return the information you want, Oh well. Peter -- Peter O'Gorman http://pogma.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On 11/04/2009 04:12 AM, Alastair Houghton wrote: Sorry for the stat distraction, I only tried it on this linux system before making the suggestion: $ stat sparse (output snipped) Size: 3221225472 Blocks: 8 But I see this morning that when I mount the same file system over NFS, on my mac: $ stat -s sparse (output snipped) st_size=3221225472 st_blocks=6291456 This email sent to site_archiver(a)lists.apple.com

1 0

Re: Identifying sparse files using getattrlist()
by rohan a 04 Nov '09

04 Nov '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=Zu4sb1EsmqoLDxetjoruorn8mUvbS6rBdIUVY+FEJLw=; b=en5EI5YBQKLi6+iLLYFobB+T0IesUQilz2A2m8BF9iEzcEq9r4/xiEbT8s2n9gX8CA aXwFh0S6MSxQ6sGBiBqIYUPzvOu5SjUY5NIaEMagXQXHobLlimF2QqdQv5x0OzzQMjY+ GS8RaqsVsGSrm1XLB7drL2qWT7K3Zf41ge1c4= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=nEjl+MotnDh9TF0wE8WE/yDIdwRfhciBETxpsvj2F9oyaF2woglKrMHY9xtR3gepfK NgL5unVdF34BtTqlC/nXHTTrbl/YTRSPmS1Km/Bbg8XKj8tgJyC1Mqv0IQa299A/eKOl /eVV74O9foj+IoUH6CbtIk5QbnE88mOCeF4M0= Hi Alastair, Thanks for the help. This worked like a gem :) I am new with this and not much of a kernel guy yet. The man page for getattrlist() says this : A sparse file may have an allocated size on disk that is less than its logical length (that is, ATTR_FILE_ALLOCSIZE < ATTR_FILE_TOTALSIZE ). Hence the confusion. Thanks On Wed, Nov 4, 2009 at 3:42 PM, Alastair Houghton <alastair(a)alastairs-place.net> wrote: > On 4 Nov 2009, at 08:41, rohan a wrote: > >> However, for a sparse file this continues to return same sizes for >> stat size and real size. This is an Apple UFS partition. > > As I said before, the thing to do is to look at the kernel sources. It's > amazing how much of their own time some people are prepared to waste... it > took me less than five minutes to figure out the answer to your question, > and in the meantime I bet you've spent a lot longer than that on it. > > Anyway, the answer is that (a) stat() won't return the information you want, > and (b) you're using the wrong attribute for getattrlist(). You want > ATTR_FILE_DATAALLOCSIZE, not ATTR_FILE_ALLOCSIZE. > > FYI, you can work this out by looking at vfs_attrlist.c (in bsd/vfs), where > you'll see that getattrlist() uses vnode_getattr(), which you can see from > the table in bsd/ufs/ffs/ffs_vnops.c maps to ufs_getattr(). (That's kind of > unsurprising, so you could skip that part if you wanted.) > > Anyway, ufs_getattr() is in ufs_vnops.c (in bsd/ufs/ufs/ufs_vnops.c), and > there you can see it copying the data out of its inode structure. The > potentially interesting lines are: > > VATTR_RETURN(vap, va_data_size, ip->i_din.di_size); > > and > > VATTR_RETURN(vap, va_data_alloc, dbtob((u_quad_t)ip->i_blocks, > devBlockSize)); > > So next we need to look in inode.h to see what the ip->i_din.di_size and > ip->i_blocks fields actually contain. We find that i_blocks is just a > #define: > > #define i_blocks i_din.di_blocks > > So now we're interested in the i_din member of struct inode, which is a > struct dinode. struct dinode is in dinode.h, so we open that up and > discover: > > u_int64_t di_size; /* 8: File byte count. */ > > and > > u_int32_t di_blocks; /* 104: Blocks actually held. */ > > in struct dinode. > > So va_data_alloc is what you're interested in, and if you look back at the > vfs_attrlist.c file, you'll find that at the top there is a list mapping the > getattrlist() constants to the va_ names. > > In case you're wondering how the ATTR_FILE_TOTALSIZE works for ufs when > ufs_getattr() clearly doesn't bother returning it, for that you need to look > in kpi_vfs.c, at vnode_getattr(), where you'll see that the vnode_getattr() > function fills in various things automatically if the underlying filesystem > didn't do it. > > All of this also explains why stat(), which uses vn_stat() (in vfs_vnops.c), > won't work. stat() asks for va_data_size and va_total_alloc. UFS doesn't > return va_total_alloc, so it's computed in vnode_getattr() by copying from > va_data_size. Hence you're going to get the same data in both places. > > I suppose there's an argument that va_total_alloc should be computed from > va_data_alloc rather than va_data_size if the former is defined, but you'd > have to take that up with Apple, and I don't fancy your chances because UFS > is pretty old and largely unused at this point. > > Anyway, it took me longer to write this e-mail than it did to work all that > out, and even if it had taken you twice as long as it took me to find all of > this, it would have been faster for you... > > Kind regards, > > Alastair. > > -- > http://alastairs-place.net > > > > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Identifying sparse files using getattrlist()
by Alastair Houghton 04 Nov '09

04 Nov '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha1; c=relaxed; d=alastairs-place.net; h= subject:mime-version:content-type:from:in-reply-to:date:cc: content-transfer-encoding:message-id:references:to; q=dns/txt; s= aug07; bh=grptblzgJWnZ780pfhBAH+GjBEs=; b=VAf213DGcKLtA2NYGz12mc gVE1FjyKLlTU0zGZe6lAeNryIYPZA7rTfy18kJEXnujkOJdV/gpdbNKyxDFEpbVb frzERJwl6lz2VHveXeRewoLZU2rahIFSHcfrNiP3JHt2cwlAPcJTG7yM31+WIgmm K1wDnWwy4lWx6e4s5u3AI= On 4 Nov 2009, at 08:41, rohan a wrote: However, for a sparse file this continues to return same sizes for stat size and real size. This is an Apple UFS partition. VATTR_RETURN(vap, va_data_size, ip->i_din.di_size); and #define i_blocks i_din.di_blocks u_int64_t di_size; /* 8: File byte count. */ and u_int32_t di_blocks; /* 104: Blocks actually held. */ in struct dinode. Kind regards, Alastair. -- http://alastairs-place.net _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… As I said before, the thing to do is to look at the kernel sources. It's amazing how much of their own time some people are prepared to waste... it took me less than five minutes to figure out the answer to your question, and in the meantime I bet you've spent a lot longer than that on it. Anyway, the answer is that (a) stat() won't return the information you want, and (b) you're using the wrong attribute for getattrlist(). You want ATTR_FILE_DATAALLOCSIZE, not ATTR_FILE_ALLOCSIZE. FYI, you can work this out by looking at vfs_attrlist.c (in bsd/vfs), where you'll see that getattrlist() uses vnode_getattr(), which you can see from the table in bsd/ufs/ffs/ffs_vnops.c maps to ufs_getattr (). (That's kind of unsurprising, so you could skip that part if you wanted.) Anyway, ufs_getattr() is in ufs_vnops.c (in bsd/ufs/ufs/ufs_vnops.c), and there you can see it copying the data out of its inode structure. The potentially interesting lines are: VATTR_RETURN(vap, va_data_alloc, dbtob((u_quad_t)ip->i_blocks, devBlockSize)); So next we need to look in inode.h to see what the ip->i_din.di_size and ip->i_blocks fields actually contain. We find that i_blocks is just a #define: So now we're interested in the i_din member of struct inode, which is a struct dinode. struct dinode is in dinode.h, so we open that up and discover: So va_data_alloc is what you're interested in, and if you look back at the vfs_attrlist.c file, you'll find that at the top there is a list mapping the getattrlist() constants to the va_ names. In case you're wondering how the ATTR_FILE_TOTALSIZE works for ufs when ufs_getattr() clearly doesn't bother returning it, for that you need to look in kpi_vfs.c, at vnode_getattr(), where you'll see that the vnode_getattr() function fills in various things automatically if the underlying filesystem didn't do it. All of this also explains why stat(), which uses vn_stat() (in vfs_vnops.c), won't work. stat() asks for va_data_size and va_total_alloc. UFS doesn't return va_total_alloc, so it's computed in vnode_getattr() by copying from va_data_size. Hence you're going to get the same data in both places. I suppose there's an argument that va_total_alloc should be computed from va_data_alloc rather than va_data_size if the former is defined, but you'd have to take that up with Apple, and I don't fancy your chances because UFS is pretty old and largely unused at this point. Anyway, it took me longer to write this e-mail than it did to work all that out, and even if it had taken you twice as long as it took me to find all of this, it would have been faster for you... This email sent to site_archiver(a)lists.apple.com

1 0

Re: Identifying sparse files using getattrlist()
by rohan a 04 Nov '09

04 Nov '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=3bbRFaEXhyWnvyFkcBM2JdUwQaKnI0iSPwJJ/1/0Bz0=; b=qp3GDC+o4cObB4aiCG8+6ElAgzE87RjTgzRG9G7FZwzdJNKTvAG9O8A0rNYX1qhup/ OHAQ8xZDAOqjN83m2pY9gxo3rzlPtvdEpyMUTN6TSMTo9wHeZxpNxv0vGy/t089efNiw upjgNS0LvLYYuWj8I9RfGJaOwpmiOWJW6TL/c= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=oPDteRV04Q+X8vfa2tiKJ02TcmY47LkWhNOX5U83gPaJ4j4HS7xI1ekAHADHsozU+D /dkMBUzTDeJb/HpPYJeb7tnz/aiP/8mCM4PxnFkXZZ0BYkzXgmpzNYQTA7MDgvxWAYrE HuhOFvJggtTwUYanLY+AO+/o0WoGTUAuceuRk= Hi All, I wrote another program to get the file stat size and its real size using this program : ----------------------------------------------------------------------------------------------------------------- #include<stdio.h> #include<sys/stat.h> int main(int argc, char *argv[]) { struct stat statbuf; if (lstat(argv[1], &statbuf) == 0) { printf("Success\n"); printf("size = %lld\n", statbuf.st_size); printf("No. of blocks = %lld\n", statbuf.st_blocks); printf("Block Size = %d\n", S_BLKSIZE); printf("Real size = %lld\n", statbuf.st_blocks* S_BLKSIZE); } else printf("Failed\n"); return 0; } ----------------------------------------------------------------------------------------------------------------- However, for a sparse file this continues to return same sizes for stat size and real size. This is an Apple UFS partition. Thanks On Tue, Nov 3, 2009 at 11:02 PM, Mike Mackovitch <macko(a)apple.com> wrote: > On Tue, Nov 03, 2009 at 09:55:05PM +0530, rohan a wrote: >> Thanks a lot :) I will try this >> Would st_blksize vary for every file ? > > That's up to each file system. > >> Also is there any macro I can use instead of directly using 512 ? > > $ grep 512 /usr/include/sys/stat.h > #define S_BLKSIZE 512 /* block size used in the stat struct */ > > > --macko > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Identifying sparse files using getattrlist()
by Mike Mackovitch 03 Nov '09

03 Nov '09

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Mutt/1.5.20 (2009-06-14) On Tue, Nov 03, 2009 at 09:55:05PM +0530, rohan a wrote: > Thanks a lot :) I will try this > Would st_blksize vary for every file ? That's up to each file system. > Also is there any macro I can use instead of directly using 512 ? $ grep 512 /usr/include/sys/stat.h #define S_BLKSIZE 512 /* block size used in the stat struct */ --macko _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0