March 2006 - Darwin-dev - lists.apple.com

Re: INT3 not calling SIGTRAP handler?
by Cyrus Harmon 14 Mar '06

14 Mar '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com i: 31 i: 32 i: 33 i: 34 i: 35 test-mach-exception.c:100: failed assertion `flag == 0' Abort trap #include <stdio.h> #include <stdlib.h> #include <signal.h> #include <stdarg.h> #include <pthread.h> #include <assert.h> #if defined(__MACH__) #include <mach/mach_types.h> #include <mach/sync_policy.h> #endif thread_port_t main_thread; int exception_count = 0; int flag = 0; kern_return_t catch_exception_raise(mach_port_t exception_port, mach_port_t thread, mach_port_t task, exception_type_t exception, exception_data_t code_vector, mach_msg_type_number_t code_count) { kern_return_t ret; exception_count++; flag = 0; return KERN_SUCCESS; } void * exception_handler_proc(void *arg) { extern boolean_t exc_server(); mach_port_t p = (mach_port_t) arg; mach_msg_server(exc_server, 2048, p, 0); abort(); } mach_port_t setup_mach_exception_port() { static mach_port_t exception_port = MACH_PORT_NULL; kern_return_t ret; pthread_attr_t attr; pthread_t returned_thread = (pthread_t) 0; pthread_attr_init(&attr); ret = mach_port_allocate(mach_task_self(), MACH_PORT_RIGHT_RECEIVE, &exception_port); fprintf(stderr, "mach_port_allocate ret: %d\n", ret); ret = mach_port_insert_right(mach_task_self(), exception_port, exception_port, MACH_MSG_TYPE_MAKE_SEND); fprintf(stderr, "mach_port_insert_right ret: %d\n", ret); ret = thread_set_exception_ports(main_thread, EXC_MASK_BREAKPOINT, exception_port, EXCEPTION_DEFAULT, MACHINE_THREAD_STATE); fprintf(stderr, "thread_set_exception_ports ret: %d\n", ret); pthread_create(&returned_thread, &attr, exception_handler_proc, (void*) exception_port); return exception_port; } void test_trap() { #if defined(__i386__) __asm__ __volatile__ ( "int3\n\t"); #elif defined(__ppc__) __asm__ __volatile__ ( "twle r1,r1\n\t"); #endif } int main(int argc, char** argv) { main_thread = mach_thread_self(); mach_port_t port = setup_mach_exception_port(); int i=0; for(i=0; i < 1000; i++) { flag = 1; test_trap(); assert(flag == 0); fprintf(stderr, " i: %d\n", i); } fprintf(stderr, "Exception count: %d\n", exception_count); return 0; } _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… And, here's a version that tries to work around the problem by using mach exceptions, but the problem persists. Perhaps I'm doing something wrong in the mach exception code, but, like in the SIGTRAP case, it works most of the time, except when it doesn't, e.g.: This email sent to site_archiver(a)lists.apple.com

1 0

Re: db / put / i386 / bug in Darwin? [Addendum]
by Stephane Sudre 14 Mar '06

14 Mar '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 14 mars 06, at 16:53, Stephane Sudre wrote: The data base is initially created on a PowerPC machine. Type of base: DB_HASH key size: 160 bits. data size: 160 bits _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Is there a known bug in Mac OS X (Intel) when it comes to the db APIs and the put call? I'm observing corruptions of the data base whenever put is used. The same code works OK on Mac OS X (PowerPC). There is no issue when reading the data base on the Intel architecture and using the seq or get API. The data is correctly swapped (and this shall not have any incidence anyway). I'm beginning to wonder if there is not a bug on the key size, where you need to swap it. Which would be really strange IMHO. If the data base is created blank on the Intel architecture, put is working fine from what I can see. So it really looks like a compatibility issue between put and a data base which has been created on a PowerPC machine. This email sent to site_archiver(a)lists.apple.com

1 0

db / put / i386 / bug in Darwin?
by Stephane Sudre 14 Mar '06

14 Mar '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com The data base is initially created on a PowerPC machine. Type of base: DB_HASH key size: 160 bits. data size: 160 bits _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Is there a known bug in Mac OS X (Intel) when it comes to the db APIs and the put call? I'm observing corruptions of the data base whenever put is used. The same code works OK on Mac OS X (PowerPC). There is no issue when reading the data base on the Intel architecture and using the seq or get API. The data is correctly swapped (and this shall not have any incidence anyway). I'm beginning to wonder if there is not a bug on the key size, where you need to swap it. Which would be really strange IMHO. This email sent to site_archiver(a)lists.apple.com

1 0

Re: rsync
by Markus Weissmann 14 Mar '06

14 Mar '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Importance: Normal User-agent: SquirrelMail/1.4.6 On Tue, March 14, 2006 05:06, Robert Nicholson wrote: > Does the darwinports one incorporate the guy from The Netherlands > fixes re: Extended Attributes. > no, darwinports rsync is the plain vanilla one; we're currently working on a rsync-ea -or whatever it'll be called- port with EA support that can be installed alongside the vanilla one. -Markus > On Mar 7, 2006, at 4:53 PM, Markus Weissmann wrote: > >> Hi Robert, >> >> On Tue, March 7, 2006 10:40, Robert McAlpine wrote: >>> >>> I've just been hit by the (i guess) problem in rsync with deletion >>> while syncing my darwinports tree. >>> I think it was mentioned a few days ago. I've lost some mail since >>> then. Were there any developments, patches? >>> >>> ... >>> >>> rsync: delete_one: unlink "/opt/local/var/db/dports/sources/ >>> rsync.rsync.darwinports.org_dpupdate_dports/devel/gpgme/files/patch- >>> configure" failed: Directory not empty (66) >>> >>> sent 12033 bytes received 349615 bytes 24941.24 bytes/sec >>> total size is 11769349 speedup is 32.54 >>> rsync error: some files could not be transferred (code 23) at / >>> SourceCache/rsync/rsync-23/rsync/main.c(1161) >>> DEBUG: sync failed doing rsync >>> while executing >>> "dportsync" >>> Error: /opt/local/bin/port: port sync failed: sync failed doing rsync >>> >> >> You can fix this problem by just installing 'rsync' from darwinports; >> this way you do not need to rely on the buggy version from Apple. >> >> >> -Markus >> >> --- >> http://www.mweissmann.de/ >> http://www.opendarwin.org/~mww/ >> >> _______________________________________________ >> Do not post admin requests to the list. They will be ignored. >> Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) >> Help/Unsubscribe/Update your Subscription: >> http://lists.apple.com/mailman/options/darwin-dev/robert% >> 40elastica.com >> >> This email sent to robert(a)elastica.com > --- http://www.mweissmann.de/ http://www.opendarwin.org/~mww/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Why no work on rsync?
by Martin Costabel 14 Mar '06

14 Mar '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Thunderbird 1.5 (Macintosh/20051201) The rsync-24 sources available at: and mirrored at: <http://darwinsource.opendarwin.org/tarballs/other/rsync-24.tar.gz> Thank you. -- Martin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Kevin Van Vechten wrote: <http://www.opensource.apple.com/darwinsource/tarballs/other/rsync-24.tar.gz> rsync-24 is the source to both the PowerPC and Intel versions of SecUpdate2006-002. This email sent to site_archiver(a)lists.apple.com

1 0

[Fwd: Re: On MAC 10.4 RAID type mirror after Rebuilding 100% it wont get mounted automatically..]
by Gaurav Kumar 14 Mar '06

14 Mar '06

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com Organization: hi rick, I am just testing the functionality of RAID of type mirror on 10.4 how it behaves after rebuilding. I create a RAID -1,umount that disk and then fire dd command to clear that disk.After this i rebuild the RAID. According to u i must not umount that RAID and start rebuild. But how can i remove one of the disk of two till i umount. If i fire dd command directly without umounting it gives Resource Busy error. Can u suggest me how can i degrade the RAID-1 without umounting.So that i can stimulate Rebuild test.And after rebuild is complete it must be again mounted. -Gaurav On Mar 13, 2006, at 1:54 AM, Gaurav Kumar wrote: hi all, On MAC OSX 10.4 if i repair a damaged RAID using diskutil command as diskutil repairMirror diskx disky. Rebiulding starts properly and when it is 100% Complete. Diskutil checkRAID shows status of RAID disk as Offline i.e not mounted. After i have to execute diskutil mountDisk to make it mounted. so,suggest me what should i do so that after rebuilding is complete, disk gets automatically mounted on MAC 10.4. -rick __ rick sulack darwin core os group apple computer, inc A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com While on 10.3.3 after completion of diskutil repairMirror the status of disk is mounted. We don't have to expilicitly fire diskutil mountDisk to make it mounted again. In 10.3.2 (7E46) we added support for live mirror rebuilding. 10.4 also has support for live mirror rebuilding. However, most of the RAID code was re-written for 10.4, it supports some new features like RAID 10, 0+1, JBOD, ... For both 10.3.2+ (7E46) and 10.4.x the need to first unmount the mirrored volume is no longer necessary. Releases older than 10.3.2 had to rebuild mirrors unmounted so the code was most likely remounting the volumes prior to 10.3.2 when the rebuild completed. Since you are talking about 10.3.3, my best guess that this was a bug outside of the RAID code that was fixed in 10.4, since a unmounted volume should probably stay that way. The easiest fix might be to not unmount the volume before rebuilding it on 10.4. Unmounting a mirrored volume on 10.4 before rebuilding is not a bad thing though. Live rebuilding was added mostly to allow rebuilding boot volumes. Depending on much normal i/o activity occurring on the volume during the rebuild, the rebuild could take longer if it is mounted. So it is best to try to minimize the extra i/o while rebuilding if you can.

1 0

Re: rsync
by Robert Nicholson 14 Mar '06

14 Mar '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Mar 7, 2006, at 4:53 PM, Markus Weissmann wrote: Hi Robert, I've just been hit by the (i guess) problem in rsync with deletion while syncing my darwinports tree. I think it was mentioned a few days ago. I've lost some mail since then. Were there any developments, patches? ... rsync: delete_one: unlink "/opt/local/var/db/dports/sources/ rsync.rsync.darwinports.org_dpupdate_dports/devel/gpgme/files/patch- configure" failed: Directory not empty (66) sent 12033 bytes received 349615 bytes 24941.24 bytes/sec total size is 11769349 speedup is 32.54 rsync error: some files could not be transferred (code 23) at / SourceCache/rsync/rsync-23/rsync/main.c(1161) DEBUG: sync failed doing rsync while executing "dportsync" Error: /opt/local/bin/port: port sync failed: sync failed doing rsync You can fix this problem by just installing 'rsync' from darwinports; this way you do not need to rely on the buggy version from Apple. -Markus --- http://www.mweissmann.de/ http://www.opendarwin.org/~mww/ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Does the darwinports one incorporate the guy from The Netherlands fixes re: Extended Attributes. On Tue, March 7, 2006 10:40, Robert McAlpine wrote: _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/robert% 40elastica.com This email sent to robert(a)elastica.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: Why no work on rsync?
by Kevin Van Vechten 13 Mar '06

13 Mar '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com The rsync-24 sources available at: and mirrored at: <http://darwinsource.opendarwin.org/tarballs/other/rsync-24.tar.gz> - Kevin On Mar 3, 2006, at 11:11 AM, Kevin Van Vechten wrote: The rsync-23 sources available at: - Kevin On Mar 3, 2006, at 8:13 AM, Rob Braun wrote: Since rsync is a GPL'd project, you can make a formal request for the source, and they should send it to you. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… <http://www.opensource.apple.com/darwinsource/tarballs/other/ rsync-24.tar.gz> rsync-24 is the source to both the PowerPC and Intel versions of SecUpdate2006-002. <http://www.opensource.apple.com/darwinsource/tarballs/other/ rsync-23.tar.gz> rsync-23 is the source to both the PowerPC and Intel versions of SecUpdate2006-001. On Fri, Mar 03, 2006 at 03:29:04PM +0100, Martin Costabel wrote: Sorry, I was not very clear: I would like to see the source of the exact version that Apple shipped with the security update. Sources of the earlier version (rsync-20) are available, I know, and they do what you are describing. This is, after all, the rsync we had in Tiger until two days ago. Or are you saying that the source for rsync-23 is available somewhere on opendarwin CVS? I haven't found it yet. OpenDarwin does not keep a copy of rsync in it's cvs repository. I do not believe Apple keeps a copy of rsync in a public cvs repository. There are several older, hacked up versions of rsync (such as rsync_hfs), but those are not what you are looking for either. Looking at the apple open source website, it appears apple does not usually release source to security update releases. Only software updates that have incremented the Mac OS X version number or developer tools release appear to be routinely released. Rob _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/kevin% 40opendarwin.org This email sent to kevin(a)opendarwin.org This email sent to site_archiver(a)lists.apple.com

1 0

Re: INT3 not calling SIGTRAP handler?
by Cyrus Harmon 13 Mar '06

13 Mar '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thanks again, Cyrus #include <stdio.h> #include <signal.h> #if defined(__MACH__) #include <sys/_types.h> #include <sys/ucontext.h> typedef struct ucontext os_context_t; #endif #define FROB_EIP void sigtrap_handler(int signal, siginfo_t *info, void *void_context) { os_context_t *context = (os_context_t*)void_context; unsigned int trap; unsigned int eip; #if defined(__MACH__) eip = (context->uc_mcontext->ss.eip); #endif #ifdef FROB_EIP *(&context->uc_mcontext->ss.eip) += 2; #endif void install_sigtrap_handler() { struct sigaction sa; sa.sa_sigaction = sigtrap_handler; sa.sa_flags = SA_SIGINFO | SA_RESTART | SA_NODEFER; sigaction(SIGTRAP, &sa, NULL); } void test_trap() { __asm__ __volatile__ ( "int3\n\t" #ifdef FROB_EIP "int $4" /*".byte 0x90" */ /* "nop" */ #endif ); } int counter; int main(int argc, char** argv) { install_sigtrap_handler(); int i=0; counter = 0; for(i=0; i<100000; i++, counter++) { test_trap(); fprintf(stderr, " i: %d, counter: %d\n", i, counter); } return 0; } < _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Here's a slightly modified test case that uses an int $4, instead of 0x09 as the bit that gets skipped over. In this case, I'm not seeing the variable corruption, but, with a reasonable frequency (after 1000-4000 iterations or so) . In this case, we die with a SIGFPE, which seems to be an attempt to execute the int $4 instruction, instead of skipping over it, which it successfully does _most of the time_. It's the times when it doesn't that are the problem. fprintf(stderr, "TRAP! PC: %x, *PC: %x", eip, *((unsigned short*) (eip-1)) ); fprintf(stderr, " fixed-up PC: %x", (context->uc_mcontext- >ss.eip)); } This email sent to site_archiver(a)lists.apple.com

1 0

Re: INT3 not calling SIGTRAP handler?
by Cyrus Harmon 13 Mar '06

13 Mar '06

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thanks, Cyrus #include <stdio.h> #include <signal.h> #if defined(__MACH__) #include <sys/_types.h> #include <sys/ucontext.h> typedef struct ucontext os_context_t; #endif #define FROB_EIP void sigtrap_handler(int signal, siginfo_t *info, void *void_context) { os_context_t *context = (os_context_t*)void_context; unsigned int trap; unsigned int eip; #if defined(__MACH__) eip = (context->uc_mcontext->ss.eip); #endif #ifdef FROB_EIP *(&context->uc_mcontext->ss.eip) += 1; #endif } void install_sigtrap_handler() { struct sigaction sa; sa.sa_sigaction = sigtrap_handler; sa.sa_flags = SA_SIGINFO | SA_RESTART | SA_NODEFER; sigaction(SIGTRAP, &sa, NULL); } void test_trap() { __asm__ __volatile__ ( "int3\n\t" #ifdef FROB_EIP ".byte 0x09" #endif ); } int counter; int main(int argc, char** argv) { install_sigtrap_handler(); int i=0; counter = 0; for(i=0; i<10000; i++, counter++) { test_trap(); fprintf(stderr, " i: %d, counter: %d\n", i, counter); } return 0; } On Mar 12, 2006, at 11:43 PM, Eric Albert wrote: On Mar 12, 2006, at 11:25 PM, Cyrus Harmon wrote: -Eric _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Thanks for the interest in this problem. I have a test case that seems to show some weirdness related to trap handling. It's quite possible that I'm doing something wrong in this test case, but it looks good to me. It can be found below. Ah, here's an interesting thing. If I turn off the EIP frobbing, and only do an INT3, not an INT3 with an extra byte as an error code that my sigtrap handler attempts to skip over, everything is fine. Turn off the #define FROB_EIP to see this behavior. The good news is that, at least from this test case, ITNT3 and sigtrap are reasonably happy, but the bad news is that if one tries to adjust the EIP inside of a sigtrap hanlder, bad things happen. But I would still like to be able to adjust the EIP from within my sigtrap handler. Suggestions? fprintf(stderr, "TRAP! PC: %x, *PC: %x", eip, *((unsigned short*) (eip-1)) ); This may sound a bit sketchy, as it only happens somewhat sporadically, but when it happens, bad things happen. I'm working on an x86 port of a compiler that makes heavy use of trapping in it's runtime environment and things basically seem to work pretty well at this point. My problem is that every now and then, I try to trap with an x86 INT3 (opcode CC) and our SIGTRAP handler takes a look at the byte following the CC, gets the appropriate "arguments" by looking at the instruction data just after the EIP, fixes up the EIP to skip over these arguments and goes on its merry way. This all works well enough, except when it doesn't, which is to say that occasionally it seems that I just walk across the INT3 without the SIGTRAP handler being called, and get a SEGV when the CPU tries to execute the instruction immediately following the INT3, which was supposed to be skipped over by the code that fixes up the EIP in the signal handler. Yes, this is all a bit messy, but the frustrating part is that most of the time this works. Is anyone else seeing instances where INT3 traps aren't being called? I don't think we have any bug reports on this. It'd be great to have one with a reproducible test case, even if it's along the lines of "this shows up about 10% of the time with this test". Any suggestions on where to look in the Darwin sources to see if there's anything amiss? This works fine on Darwin/ppc and, FWIW, {freebsd,linux}/x86 and mostly works on Darwin/x86, but this one thing is causing some rather major problems. Any suggestions or advice on dealing with this would be greatly appreciated. You'd need the kernel source, which I'm afraid isn't available yet for the Intel releases of Mac OS X. You could look at the older Darwin/x86 kernel sources, but enough has changed since then that it wouldn't be helpful. This email sent to site_archiver(a)lists.apple.com

1 0