March 2008 - Darwin-dev - lists.apple.com

Re: high accuracy timing options?
by Andrew Gallatin 27 Mar '08

27 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Aaron Turner writes: > Interesting how gettimeofday() fails to increment around 90% of the > time. I guess technically incrementing 1/10 times while requiring FWIW, I hacked your program to keep some stats, and on a core2 iMac I have access to, I see an average increment of just over 1us with an occasional outlier of a few ms (probably when something stole the CPU from the app). Given that gtod has a 1us granualarity, and the program is not realtime, that is not too bad. > ~13 times to measure a usec is good enough, but it's not giving me a > warm & fuzzy feeling. Running a tight gettimeofday() loop in my > application seems to have good results down to 8usec between events I have no idea what you're trying to do, so its hard to provide any solution that would make you warm and fuzzy. I was mearly suggesting that the accuracy of gettimeofday() on MacOSX is likely good enough for most timing you might want to do.. Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: high accuracy timing options?
by Steve Sisak 27 Mar '08

27 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Anyways, I'm going to try to dig around some more and try to find the HPET timer API. Theoretically, that should be more accurate then what I'm seeing with gtod(). I saw Steve's MPDelayUntil() comment, but it makes no indication what the method is, and it seems unlikely that it would be any better then this. I'm guessing it's just a wrapper around nanosleep(). The example I pointed you to: <http://developer.apple.com/documentation/Carbon/Conceptual/Multitasking_Mul…> Note this code is written in Eudora, not tested. >From the example I gave you (to convert back to seconds): static float HowLong( AbsoluteTime endTime, AbsoluteTime bgnTime ) { AbsoluteTime absTime; Nanoseconds nanosec; absTime = SubAbsoluteFromAbsolute(endTime, bgnTime); nanosec = AbsoluteToNanoseconds(absTime); return (float) UnsignedWideToUInt64( nanosec ) / 1000.0; } u_int32_t count = 0, fail = 0, back = 0, i; struct timeval sleep_until, now, last; // MPDelayUntil(&sleep_until); // would just until 5 sec from start last = UpTime(); do { count++; now = UpTime(); if (now == last) fail ++; else if (SubAbsoluteFromAbsolute(now, endTime) == 0) back ++; last = now; } while (timercmp(&now, &sleep_until, <)); printf("Ran gtod() %lu times in 5 seconds.\n", count); printf("%f gtod()/usec\n", (float)count / 5000 / 1000); printf("Failed to increment %lu times\n", fail); printf("Back in time %lu times\n", back); printf("-------------\n"); count = 0; fail = 0; back = 0; } exit(0); } Note that AbsoluteTime may be declared as a struct, but you can cast to UInt64. What are you really trying to do? -Steve _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… At 11:23 AM -0700 3/27/08, Aaron Turner wrote: Huh? I posted links to the documentation and sample code, which I assumed you could read. You calculate a time in the future in AbsoluteTime units and call MPDelayUntil(), which delays until that time is past. You would be wrong. It's a high-performance, high-accuracy call from the original Mac OS Driver Services library that was moved up to Multiprocessor Sevices library. It does not rely on converting unix time units to processor time units. does exactly what what you want. Note that SubAbsoluteFromAbsolute() returns 0, not a negative number if the time has passed. int main(int argc, char *argv[]) { AbsoluteTime fiveSeconds = DurationToAbsolute(5000 * kDurationMillisecond); AbsoluteTime now = UpTime(); AbsoluteTime last = now; AbsoluteTime sleep_until = now; for (i = 0; i < 5; i ++) { sleep_until = AddAbsoluteToAbsolute(UpTime(), fiveSeconds); // what you wrote // sleep_until = AddAbsoluteToAbsolute(sleep_until, fiveSeconds); // this won't drift This email sent to site_archiver(a)lists.apple.com

1 0

Re: high accuracy timing options?
by Aaron Turner 27 Mar '08

27 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=aL0dw/QhgXGmCzpPen1Gur6MnQbPBUwfUkvA4Y8un1E=; b=iJo7pSkdeR1VpA1J4avIM0HUJMeD8FEnoyvk8DP8FSWh1KlJXs7BJdsjbc+BfaRYEfuUEN0Ds1x68dlt5KL4oiSh6LQwfhJcxbaGTk5+Lu4WoF9G4hf1UBNAwaldoCNwjEdllFAXI30PgOtox8z1ShCY/2xk6JPkk0QiTMsBa+o= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=P0OLpK2u25fFqrJxQ/1fFUBeb4+ycUrs1ZHo/y4QyIQ7y0FyVcp65vGJ1sk/Z94PNCYiRmf2Sfa6ibQEl97uwByxJA1d79L0VC84OB0j/DRh4zsGWH7AnVS4KEgdLUv8k4qGsjesAWeFOugJkmi+gmA1Me0haZ2k+/KjhAOYhMc= On Wed, Mar 26, 2008 at 5:34 AM, Andrew Gallatin <gallatin(a)cs.duke.edu> wrote: > > Rather than re-inventing the wheel, have you tried to measure the > accuracy of gettimeofday() on darwin? It will be much better > than you expect. > > MacOSX/Darwin uses a really clever method to move most of the heavy > lifting out of the kernel and into userspace for gettimeofday(). > The kernel sets up a "commpage" with an appropriate gettimeofday() > function for the hardware you're on. It then shares timestamp > information with this gettimeofday() implementation. This commpage is > then mapped into your process, so when you call gettimeoday(), you use > this clever version. > > At least in the ppc implementation, the library calls mftb, so the > timing info on ppc should be both cheap and very accurate. I don't > understand x86 asm nearly as well, so I can't say exactly what the x86 > implementation does. > > Anyway, check out the kernel sources to see what I mean: > > xnu-$VERSION/osfmk/ppc/commpage/gettimeofday.s > xnu-$VERSION/osfmk/i386/commpage/commpage_gettimeofday.s Hey Drew, So I decided to do some benchmarking: ---- code ---- int main(int argc, char *argv[]) { u_int32_t count = 0, fail = 0, back = 0, i; struct timeval sleep_until, now, last; for (i = 0; i < 5; i ++) { gettimeofday(&sleep_until, NULL); sleep_until.tv_sec += 5; gettimeofday(&last, NULL); do { count++; gettimeofday(&now, NULL); if ((now.tv_sec == last.tv_sec) && (now.tv_usec == last.tv_usec)) fail ++; else if (timercmp(&last, &now, >)) back ++; last.tv_sec = now.tv_sec; last.tv_usec = now.tv_usec; } while (timercmp(&now, &sleep_until, <)); printf("Ran gtod() %lu times in 5 seconds.\n", count); printf("%f gtod()/usec\n", (float)count / 5000 / 1000); printf("Failed to increment %lu times\n", fail); printf("Back in time %lu times\n", back); printf("-------------\n"); count = 0; fail = 0; back = 0; } exit(0); } ---- code ---- gcc -O3 -o gtod gtod.c Very interesting results: Ran gtod() 68160730 times in 5 seconds. 13.632146 gtod()/usec Failed to increment 63197136 times Back in time 0 times ------------- Ran gtod() 68386072 times in 5 seconds. 13.677215 gtod()/usec Failed to increment 63418876 times Back in time 0 times ------------- Ran gtod() 68054635 times in 5 seconds. 13.610927 gtod()/usec Failed to increment 63102535 times Back in time 0 times ------------- Ran gtod() 65948046 times in 5 seconds. 13.189610 gtod()/usec Failed to increment 61147617 times Back in time 0 times ------------- Ran gtod() 68162385 times in 5 seconds. 13.632477 gtod()/usec Failed to increment 63201624 times Back in time 0 times ------------- Interesting how gettimeofday() fails to increment around 90% of the time. I guess technically incrementing 1/10 times while requiring ~13 times to measure a usec is good enough, but it's not giving me a warm & fuzzy feeling. Running a tight gettimeofday() loop in my application seems to have good results down to 8usec between events (sending a packet), but after that things start getting skewed. I'm not sure if its a resolution issue of the timing loop or what. May just be one of those things where a RTOS is necessary to get any improvement. Anyone know if OS X/Darwin has the equivalent of Linux's RT scheduler extensions? So far I was lucky about NTP not moving the clock backwards, but I suppose that could be an issue. But since I'm not trying to land man on the moon or other dangerous tasks, I can take that risk. Anyways, I'm going to try to dig around some more and try to find the HPET timer API. Theoretically, that should be more accurate then what I'm seeing with gtod(). I saw Steve's MPDelayUntil() comment, but it makes no indication what the method is, and it seems unlikely that it would be any better then this. I'm guessing it's just a wrapper around nanosleep(). -- Aaron Turner http://synfin.net/ http://tcpreplay.synfin.net/ - Pcap editing & replay tools for Unix They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: question on exporting vcard to UTF-16 file
by Garth Cummings 27 Mar '08

27 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hi Koifans, On Mar 26, 2008, at 8:40 PM, Calvin Liu wrote: I have a question about the new address book application in leopard. An applescript like this works fine on tiger and exports a readable text file in UTF-8. I suggest trying the applescript-users mailing list: <http://lists.apple.com/mailman/listinfo/applescript-users> Best regards, --gc ____________________________________________________________________ Garth Cummings gcummings(a)apple.com Sr. Software Engineer Apple Developer Technical Support <http://developer.apple.com/technicalsupport> WWDC 2008 • June 9-13 • Moscone West • San Francisco, CA <http://developer.apple.com/wwdc/> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Address Book and AppleScript are both off-topic for this list. Neither technology is part of Darwin. This email sent to site_archiver(a)lists.apple.com

1 0

Re: kextload error
by Duane Murphy 27 Mar '08

27 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com --- At Thu, 27 Mar 2008 11:36:54 +0300, Brian Bergstrand wrote: >10.4.x PPC does not support the UUID cmd section. XCode 3 adds that by >default (even if your min target is 10.4) when building with GCC4 and >it breaks 10.4.x PPC. > >You have to manually strip the section from the PPC side: > >strip -arch ppc -no_uuid -S <KEXT_PATH> So I'm confused how the ppc code would affect an Intel machine? You are correct though, that we are using Xcode 3. >On Mar 27, 2008, at 6:54 AM, Duane Murphy wrote: > >> --- At Wed, 26 Mar 2008 21:02:25 -0500, Paul Nelson wrote: >> >>> You probably did not build both intel and ppc >> >> Nope. The exact same kext works on 10.4.11. This is on an intel iMac. >> >>>> From: Duane Murphy <duanemurphy(a)mac.com> >>>> Date: Wed, 26 Mar 2008 17:59:03 -0700 >>>> To: Darwin Dev <darwin-dev(a)lists.apple.com> >>>> Subject: kextload error >>>> >>>> We had the opportunity to run our kext on an older system (10.4.7 >>>> - I >>>> know ancient history) and discovered an unexpected load error from >> kextload: >>>> >>>> load command 2 unknown cmd field >>>> >>>> All of the dependency versions match up just fine. I found some >>>> notes >>>> that said using -fapple-kext should fix this problem. We are using >>>> that >>>> option. >>>> >>>> We're wondering what else could be causing this and how do we fix >>>> it? ...Duane _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: Getting last data from child process in Leopard
by Ingemar Ragnemalm 27 Mar '08

27 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Thunderbird 2.0.0.12 (Macintosh/20080213) Steve Checkoway wrote: /Ingemar _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… With a pty, it shouldn't buffer the pipe to a deadlock. That's the point. But maybe that is the problem, that pty's in Leopard buffer data when Tiger does not. Could be, I don't still have Tiger around to try. Then again, both Leopard and the version of Linux I was using before both buffer the printf() if I remove the fflush(). The difference is that Linux reports child quit, then it reads 16 bytes "hello!there!BYE!" whereas Leopard loses the data. This must be the loss you were describing. Maybe that is a bug, I'm not in a position to say. Jordan or Terry know far more than I. Exactly. Depending on buffering, the data may be early or late, but it should not just disappear. With too much buffering (which is what I get if I use pipes instead of ptys) the last data arrives in a big chunk at the end, but it does arrive at all. As a side note, this runs on (at least this particular flavor of) Linux as well with the small changes (included in the file) to include <pty.h> instead of <util.h> and to break out of the loop when read() returns -1 and errno is EIO. It is all built from common Unix calls so it should work. Didn't link in RHEL though, and that's the only Linux I have easily accessible. I don't have red hat, but it worked for me on both ubuntu (ppc) and fedora (x86). Did you remember to pass -lutil to gcc? You are right, I just missed -lutil. And it seems to work just fine, which means that RHEL seems to behave like Tiger. You're still using buffered IO in the signal handler. You simply cannot do that. Read the sigaction(2) man page. Well, I have had no problems with that, but that doesn't matter, it doesn't affect the real problem. I can do the whole thing without signals. So, Leopard is losing data when at least Tiger does not, and RHEL also seems to work. Under Leopard, you randomly lose data, sometimes it works but all too often it doesn't. I have found one solution that might do the trick. I spawn a pthread and let it sit and do blocking read()s, and whenever it gets something, it copies it over to a buffer that the main thread can poll. This solution needs some kind of semaphores when accessing common data, of course, but that can be done. I am testing this approach right now and it looks very promising No lost data at all so far. Another solution would be to make a pipe to a process that does these blocking read()s, and fflush() everything to the main program to get a smooth data flow. That is my conclusion so far. Thanks for the feedback! It is much appriciated! This email sent to site_archiver(a)lists.apple.com

1 0

Re: kextload error
by Brian Bergstrand 27 Mar '08

27 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com You have to manually strip the section from the PPC side: strip -arch ppc -no_uuid -S <KEXT_PATH> On Mar 27, 2008, at 6:54 AM, Duane Murphy wrote: --- At Wed, 26 Mar 2008 21:02:25 -0500, Paul Nelson wrote: Nope. The exact same kext works on 10.4.11. This is on an intel iMac. From: Duane Murphy <duanemurphy(a)mac.com> Date: Wed, 26 Mar 2008 17:59:03 -0700 To: Darwin Dev <darwin-dev(a)lists.apple.com> Subject: kextload error load command 2 unknown cmd field ...Duane Brian Bergstrand <http://www.bergstrand.org/brian/> PGP Key ID: 0xB6C7B6A2 _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… 10.4.x PPC does not support the UUID cmd section. XCode 3 adds that by default (even if your min target is 10.4) when building with GCC4 and it breaks 10.4.x PPC. You probably did not build both intel and ppc We had the opportunity to run our kext on an older system (10.4.7 - I know ancient history) and discovered an unexpected load error from kextload: All of the dependency versions match up just fine. I found some notes that said using -fapple-kext should fix this problem. We are using that option. We're wondering what else could be causing this and how do we fix it? This email sent to site_archiver(a)lists.apple.com

1 0

backtrace_symbols not giving right symbols
by James W. Walker 27 Mar '08

27 Mar '08

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com I tried using backtrace_symbols from a framework of mine, and the function names were wrong. They were names that were probably in the framework somewhere, but not code that was actually being used. This was a debug build, full debug symbols, not optimized, not stripped. I tried changing the debug info format from Dwarf to Stabs and it made no difference. What might I be missing? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: kextload error
by Duane Murphy 27 Mar '08

27 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com --- At Wed, 26 Mar 2008 21:02:25 -0500, Paul Nelson wrote: >You probably did not build both intel and ppc Nope. The exact same kext works on 10.4.11. This is on an intel iMac. >> From: Duane Murphy <duanemurphy(a)mac.com> >> Date: Wed, 26 Mar 2008 17:59:03 -0700 >> To: Darwin Dev <darwin-dev(a)lists.apple.com> >> Subject: kextload error >> >> We had the opportunity to run our kext on an older system (10.4.7 - I >> know ancient history) and discovered an unexpected load error from kextload: >> >> load command 2 unknown cmd field >> >> All of the dependency versions match up just fine. I found some notes >> that said using -fapple-kext should fix this problem. We are using that >> option. >> >> We're wondering what else could be causing this and how do we fix it? ...Duane _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

question on exporting vcard to UTF-16 file
by Calvin Liu 27 Mar '08

27 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Thunderbird 2.0.0.12 (Windows/20080213) Hi, there, I have a question about the new address book application in leopard. An applescript like this works fine on tiger and exports a readable text file in UTF-8. tell application \"Finder\" set theFolder to folder \"TemporaryItems\" of folder \"Caches\" of folder \"Library\" of home end tell tell application \"Address Book\" set theVcard to (vcard of my card) as Unicode text set theFile to (theFolder as text) & \"myAddressBookCard.vcf\" set theFileHandle to open for access theFile with write permission write (ASCII character of 254) to theFileHandle write (ASCII character of 255) to theFileHandle write theVcard to theFileHandle close access theFileHandle end tell But on leopard, it created a UTF-16 file with no line break. Why it's so different? Any parameter available to the addressbook application can force it to create the same file as that on tiger? Thanks. Koifans _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0