June 2006 - Darwin-dev - lists.apple.com

Re: Metadata support
by Boyd Waters 29 Jun '06

29 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jun 28, 2006, at 10:49 PM, Jordan K. Hubbard wrote: - boyd _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com Actually attempting to test this just now (under 10.4.6) consistently leads to a rsync EXC_BAD_ACCESS crash in acl_from_text and copyfile. Basically this seems to occur anytime I try to rsync a file with both and ACL and EA. If you can send me your data as an attachment, or file a radar and attach the data to it, I'll see that it's looked at. It shouldn't crash, obviously. I used to see this all the time and stopped seeing it with a tip build of 10.4.7. I would be very interested in seeing a unit-test case of this so I could verify the status of this rsync bug on my systems; I used to be able to reproduce this bug but cannot with current 10.4.7 smime.p7s

1 0

Re: Metadata support
by Dan Shoop 29 Jun '06

29 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Please be sure to file these as individual Radar bugs! http://bugreporter.apple.com/ -dhan ------------------------------------------------------------------------ Dan Shoop AIM: iWiring Systems & Networks Architect http://www.ustsvs.com/ shoop(a)iwiring.net http://www.iwiring.net/ 1-714-363-1174 pgp key fingerprint: FAC0 9434 B5A5 24A8 D0AF 12B1 7840 3BE7 3736 DE0B iWiring provides systems and networks support for Mac OS X, unix, and Open Source application technologies at affordable rates. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… At 12:00 AM -0700 6/29/06, Peter Bierman wrote: At 4:35 PM +1000 6/29/06, Q wrote: If you want a couple more things to look into that are broken try these two: It's great to have such excellent analysis, but we need this stuff in our bug tracking system so it doesn't get lost. (Even if that appears to be exactly what the bug tracking system does to externally reported bugs. ;-) It's never seemed to make a difference before, at least wrt rsync. They just get closed and the problems still remain. -- This email sent to site_archiver(a)lists.apple.com

1 0

Re: AfxIsValidAddress equivalent, test valid memory
by Brendan Creane 29 Jun '06

29 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=tX57vxuSt94PWVokSTCmctu3QD1GoQ6Bj4Aulr0cwgE9+6qVtssqd+jYbVmowq6O8dNbz3lySKaPiRslcOUR+vEXRXbrf7lSsITU0AMyWvYWFAD6teRRO3gwt6bM/oxRU0tgBZWa2sBboVo1FfJUp+SklSzvU7Z9kY8KtSbuhQA= ; Oops, forgot to free the malloc'ed memory (never did that before:-) -brendan Boolean memIsValid(vm_address_t m_addr, size_t numOfBytes) { vm_address_t dstPtr = (vm_address_t)malloc(numOfBytes); mach_port_t thisTask = mach_task_self(); vm_size_t sizeRead = numOfBytes; int err; err = vm_read_overwrite(thisTask, m_addr, numOfBytes, dstPtr, &sizeRead); if (err == noErr && sizeRead != numOfBytes) err = -1; free ((void*)dstPtr); // !! return (err == noErr); } _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: AfxIsValidAddress equivalent, test valid memory
by Brendan Creane 29 Jun '06

29 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=OyvRg37YIBv0qXFqtRk55oy/3l60Gthv/9PNg3CuCxT/NZ6VcGlIOEiijPR1TFL2XswhqUmf8xpg76HPBysqb1HBTzxmwlHfJ+pfQ+l4n6RkfmsmnrGobftjiY/SeGkLB649qWQrwQKXidDh/2gSu6zq4+VeWWTXgmPpajdhP3I= ; Hi Alex, I had a similar need for an underlying Win32 function (IsBadReadPtr and it's bretheren). I ended up poaching some ideas from the MoreIsBetter stack backtrace code. I bet there's another more [efficient, elegant, effective, etc] way to accomplish this, but hey, it seems to work. cheers, Brendan Creane Boolean memIsValid(vm_address_t addr, size_t numOfBytes) { vm_address_t dstPtr = (vm_address_t)malloc(numOfBytes); mach_port_t thisTask = mach_task_self(); vm_size_t sizeRead = numOfBytes; int err; err = vm_read_overwrite(thisTask, addr, numOfBytes, dstPtr, &sizeRead); if (err == noErr && sizeRead != numOfBytes) err = -1; return (err == noErr); } > Hi All, > On the PC (Visual Studio), there is a call for > testing valid memory > AfxIsValidAddress( ) - "Tests any memory address to > ensure that it is > contained entirely within the program's memory > space." This function > also takes a read/write parameter. Is there a > similar function on > MacOSX/Xcode? > _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Metadata support
by seebs＠plethora.net 29 Jun '06

29 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com In message <a06010202c0c92a757977(a)[192.168.0.222]>, Peter Bierman writes: >At 4:35 PM +1000 6/29/06, Q wrote: >>If you want a couple more things to look into that are broken try these two: >Please be sure to file these as individual Radar bugs! I'm nearly totally sure that the rsync/tar/pax problems with uchg have been in Radar since spring of 2005. I ran into them working on a project back then and discussed them with people at Apple, anyway. -s _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Metadata support
by Peter Bierman 29 Jun '06

29 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Please be sure to file these as individual Radar bugs! http://bugreporter.apple.com/ -pmb _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… At 4:35 PM +1000 6/29/06, Q wrote: If you want a couple more things to look into that are broken try these two: It's great to have such excellent analysis, but we need this stuff in our bug tracking system so it doesn't get lost. (Even if that appears to be exactly what the bug tracking system does to externally reported bugs. ;-) This email sent to site_archiver(a)lists.apple.com

1 0

Re: Metadata support
by seebs＠plethora.net 29 Jun '06

29 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com In message <46D9A985-8265-40D8-B7C7-0DD81CA5D3B0(a)gmail.com>, Q writes: >a) If chflags +uchg is set on a file copyfile() will replicate this >onto rsync's temporary ._* files, which rsync is then unable to >delete, and are left behind in /tmp. Not unique to rsync; found in several other programs. Reported months ago, too. (Actually, we may have reached a full year.) >b) When acl's are disabled for a filesystem (sudo fsaclctl -p / -d) >the following code works. With acl's enabled (sudo fsaclctl -p / -e) >it fails. The difference is the behaviour of mktemp(3), without >ACL's it will create a temporary file, with ACL's enabled it doesn't, >a situation that copyfile() seems unable to handle. If you were to >use mkstemp(3) instead this problem does not occur. It looks to me as though this is a relative of the bug where certain programs can't copy files with ACL's enabled. The boundary condition seems to be something like "file has resource fork but no ACL". Hmm. Looking at my notes, the bug was that if you have ACLs enabled, and a file has a resource fork, but does not have an ACL, tar and pax choke trying to copy it. But this could be a totally DIFFERENT bug related to the sort of innovative ACL implementation. -s _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Metadata support
by Q 29 Jun '06

29 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:in-reply-to:references:mime-version:content-type:message-id:cc:content-transfer-encoding:from:subject:date:to:x-mailer; b=SLmAwZ0/XT0X6kE6AEh/f9bIHe1FYo43Cu9hHp9dq03eT6PxLf0egiM0V7srKa7/0+uNs1ZHGfJjq7/2vk92k3nfHxFNfxBgsbMecld59ZN/dFa08engL6dMDYg274T9d0t8zje78366IE+zLXzift4bp1Qdny9bnL0jMeWyX/Y= On 29/06/2006, at 2:49 PM, Jordan K. Hubbard wrote: and #include <stdio.h> #include <unistd.h> #include <fcntl.h> #include <err.h> #include <errno.h> #include <string.h> #include "copyfile.h" int main (int argc, char **argv) { char *fname_src = "/tmp"; char *template = "/tmp/test.XXXXX"; char fname[32]; char *fname_tmp; int fd; strncpy(fname, template, sizeof(fname)); fname_tmp = mktemp(fname); return 0; } -- Seeya...Q -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- _____ / Quinton Dolan - qdolan(a)gmail.com __ __/ / / __/ / / / __ / _/ / / Gold Coast, QLD, Australia __/ __/ __/ ____/ / - / Ph: +61 419 729 806 _______ / _\ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Actually attempting to test this just now (under 10.4.6) consistently leads to a rsync EXC_BAD_ACCESS crash in acl_from_text and copyfile. Basically this seems to occur anytime I try to rsync a file with both and ACL and EA. If you can send me your data as an attachment, or file a radar and attach the data to it, I'll see that it's looked at. It shouldn't crash, obviously. - Jordan If you want a couple more things to look into that are broken try these two: a) If chflags +uchg is set on a file copyfile() will replicate this onto rsync's temporary ._* files, which rsync is then unable to delete, and are left behind in /tmp. b) When acl's are disabled for a filesystem (sudo fsaclctl -p / -d) the following code works. With acl's enabled (sudo fsaclctl -p / -e) it fails. The difference is the behaviour of mktemp(3), without ACL's it will create a temporary file, with ACL's enabled it doesn't, a situation that copyfile() seems unable to handle. If you were to use mkstemp(3) instead this problem does not occur. if (fname_tmp && copyfile(fname_src, fname_tmp, NULL, COPYFILE_NOFOLLOW | COPYFILE_PACK | COPYFILE_METADATA)==0 && (fd = open(fname_tmp, O_RDONLY, 0)) >= 0) { printf("Copyfile worked\n"); unlink(fname_tmp); } else { printf("Copyfile failed\n"); printf("fname_tmp = %s\nfname_src = %s\nfd = %d\n", fname_tmp, fname_src, fd); err(errno, "%s", fname_tmp); } This email sent to site_archiver(a)lists.apple.com

1 0

Re: Metadata support
by Jordan K. Hubbard 29 Jun '06

29 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jun 28, 2006, at 9:04 PM, Dan Shoop wrote: On Jun 27, 2006, at 12:17 PM, Dan Shoop wrote: - Jordan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… At 12:10 AM -0700 6/28/06, Jordan K. Hubbard wrote: - rsync -aE doesn't preserve BSD flags, locked flag, modification date of files with resource forks, ACLs (I'm sure there's gazillions of rdars) It also doesn't appear to be able to copy files with both ACLs and resource forks. It appears to only work if you have one or the other. At least that was the last time I checked. That's not true. rsync copies all EAs, of which ACLs are simply examples of. It also copies a couple of "synthetic EAs", namely the old-style resource fork information and the finderInfo. As far as rsync (and tar and anyone else using copyfile()) is concerned, however, it's just a list of EAs scattered across several different namespaces (for various reasons, ACL-containing EAs are logically segregated). If you want to do more digging for yourself, simply check out the source code for copyfile() and the behavior of the getxattr() call. If this was fixed then it was fixed extremely recently as the issue has been well reported extensively. Actually attempting to test this just now (under 10.4.6) consistently leads to a rsync EXC_BAD_ACCESS crash in acl_from_text and copyfile. Basically this seems to occur anytime I try to rsync a file with both and ACL and EA. If you can send me your data as an attachment, or file a radar and attach the data to it, I'll see that it's looked at. It shouldn't crash, obviously. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Metadata support
by Dan Shoop 29 Jun '06

29 Jun '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jun 27, 2006, at 12:17 PM, Dan Shoop wrote: That is good news. That is even better news! -dhan Host Name: Ooblek Date/Time: 2006-06-28 23:31:46.257 -0400 OS Version: 10.4.6 (Build 8I127) Report Version: 4 Command: rsync Path: /usr/bin/rsync Parent: rsync [3908] Version: ??? (???) PID: 3909 Thread: 0 Exception: EXC_BAD_ACCESS (0x0001) Codes: KERN_PROTECTION_FAILURE (0x0002) at 0x00000000 Thread 0 Crashed: 0 libSystem.B.dylib 0x900fc058 acl_from_text + 736 1 libSystem.B.dylib 0x900c5698 copyfile_unpack + 564 2 libSystem.B.dylib 0x900c60c4 copyfile + 1584 3 rsync 0x0000da58 0x1000 + 51800 4 rsync 0x00008744 0x1000 + 30532 5 rsync 0x00003764 0x1000 + 10084 6 rsync 0x0000632c 0x1000 + 21292 7 rsync 0x0000ac6c 0x1000 + 40044 8 rsync 0x0000af90 0x1000 + 40848 9 rsync 0x0000b0b0 0x1000 + 41136 10 rsync 0x0000afd0 0x1000 + 40912 11 rsync 0x0001bd18 0x1000 + 109848 12 rsync 0x0000a5e8 0x1000 + 38376 13 rsync 0x0000bb2c 0x1000 + 43820 14 rsync 0x0000bf98 0x1000 + 44952 15 rsync 0x0000281c 0x1000 + 6172 16 rsync 0x000026c4 0x1000 + 5828 -- -dhan ------------------------------------------------------------------------ Dan Shoop AIM: iWiring Systems & Networks Architect http://www.ustsvs.com/ shoop(a)iwiring.net http://www.iwiring.net/ 1-714-363-1174 pgp key fingerprint: FAC0 9434 B5A5 24A8 D0AF 12B1 7840 3BE7 3736 DE0B iWiring provides systems and networks support for Mac OS X, unix, and Open Source application technologies at affordable rates. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… At 12:10 AM -0700 6/28/06, Jordan K. Hubbard wrote: - rsync -aE doesn't preserve BSD flags, locked flag, modification date of files with resource forks, ACLs (I'm sure there's gazillions of rdars) It also doesn't appear to be able to copy files with both ACLs and resource forks. It appears to only work if you have one or the other. At least that was the last time I checked. That's not true. rsync copies all EAs, of which ACLs are simply examples of. It also copies a couple of "synthetic EAs", namely the old-style resource fork information and the finderInfo. As far as rsync (and tar and anyone else using copyfile()) is concerned, however, it's just a list of EAs scattered across several different namespaces (for various reasons, ACL-containing EAs are logically segregated). If you want to do more digging for yourself, simply check out the source code for copyfile() and the behavior of the getxattr() call. If this was fixed then it was fixed extremely recently as the issue has been well reported extensively. Actually attempting to test this just now (under 10.4.6) consistently leads to a rsync EXC_BAD_ACCESS crash in acl_from_text and copyfile. Basically this seems to occur anytime I try to rsync a file with both and ACL and EA. The creation date information you and maurits have been referring to (as copied by the Finder) is another bit of information in the HFS catalog which isn't currently exposed via stat(), but we have other ways to get at it and will probably end up storing it as another "synthetic EA" for portability purposes, though that's not a firm decision yet. Thanks to everyone for sparking an interesting discussion. It's led us to renew internal discussion on this topic and we're currently engaged in trying to work out ways of making Finder copies and tar/cp/rsync copies more genuinely equivalent as a result. $ rsync -avE xyzzy/fileaclxattr xyzzy2/fileaclxattr building file list ... done fileaclxattr ._fileaclxattr rsync: connection unexpectedly closed (99 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-24/rsync/io.c(359) rsync: connection unexpectedly closed (52 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-24/rsync/io.c(359) Thread 0 crashed with PPC Thread State 64: srr0: 0x00000000900fc058 srr1: 0x000000000000f030 vrsave: 0x0000000000000000 cr: 0x28002222 xer: 0x0000000000000004 lr: 0x00000000900fc040 ctr: 0x00000000901006a0 r0: 0x00000000900fc040 r1: 0x00000000bfffcbf0 r2: 0x0000000000000000 r3: 0x0000000000000000 r4: 0x00000000901a110c r5: 0x00000000bfffcc34 r6: 0x0000000000000001 r7: 0x0000000000000000 r8: 0x0000000000000003 r9: 0x0000000001804e20 r10: 0x0000000001804e30 r11: 0x0000000000000000 r12: 0x00000000901006a0 r13: 0x00000000a0001910 r14: 0x000000009019bd80 r15: 0x000000009019bd80 r16: 0x00000000bfffcc30 r17: 0x000000009019bd80 r18: 0x00000000901a110c r19: 0x00000000ffffffff r20: 0x00000000bfffcc34 r21: 0x000000009019bd80 r22: 0x00000000a0001910 r23: 0x0000000000300aa0 r24: 0x00000000bfffcc2c r25: 0x000000009019bd80 r26: 0x0000000000300a40 r27: 0x0000000000000001 r28: 0x0000000000000000 r29: 0x0000000000300a87 r30: 0x00000000a000186c r31: 0x00000000900fbd80 Binary Images Description: 0x1000 - 0x34fff rsync /usr/bin/rsync 0x8fe00000 - 0x8fe51fff dyld 44.4 /usr/lib/dyld 0x90000000 - 0x901bbfff libSystem.B.dylib /usr/lib/libSystem.B.dylib 0x90213000 - 0x90218fff libmathCommon.A.dylib /usr/lib/system/libmathCommon.A.dylib 0x94efe000 - 0x94f1bfff libresolv.9.dylib /usr/lib/libresolv.9.dylib This email sent to site_archiver(a)lists.apple.com

1 0