March 2008 - Darwin-dev - lists.apple.com

Re: high accuracy timing options?
by Andrew Gallatin 26 Mar '08

26 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Rather than re-inventing the wheel, have you tried to measure the accuracy of gettimeofday() on darwin? It will be much better than you expect. MacOSX/Darwin uses a really clever method to move most of the heavy lifting out of the kernel and into userspace for gettimeofday(). The kernel sets up a "commpage" with an appropriate gettimeofday() function for the hardware you're on. It then shares timestamp information with this gettimeofday() implementation. This commpage is then mapped into your process, so when you call gettimeoday(), you use this clever version. At least in the ppc implementation, the library calls mftb, so the timing info on ppc should be both cheap and very accurate. I don't understand x86 asm nearly as well, so I can't say exactly what the x86 implementation does. Anyway, check out the kernel sources to see what I mean: xnu-$VERSION/osfmk/ppc/commpage/gettimeofday.s xnu-$VERSION/osfmk/i386/commpage/commpage_gettimeofday.s Drew _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

high accuracy timing options?
by Aaron Turner 26 Mar '08

26 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=XgqVrPjEnJ0SGxFh8HZtLcRKpUZvqBVs015GT2PZXr4=; b=cAVuA1SChMnWCS6fV6BUvXndN0PJLd2ZBa0eUNjTXbA7X/yHc0zXv8D+ZLl81OadXyi0E7y47iPG2SjXHHsgM/MFDc3TAp+JnO9FGnJ2zOZ1RGqeeHyllMiLNv8mbhoSceHvO6qgGhdoF6zx4jX+Zg6c8zpKquJhWfARh2W1OHU= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=F9BGomEyIWF0b3b0ttXQY8Scea386eHzsM7B+TErzr+VQRHgfUAkzFAIURg8w0HhyprcoU04N6N1CmnkTPsgREdQNyFq6mC6yo0q8NvWfTx7r0oMqM198Ur1G1CR6vuunSor3YogJCWQ1xrH5ChnYJd/B9jMk4i8lHGiQWkCjJ0= I'm trying to implement a high speed (roughly 1usec accuracy) timer for OS X in C. My process runs as root and it's perfectly acceptable that I eat a large amount of CPU time. There seems to be a few ways to do this: 1. Use the RDTSC available in the Intel/PPC cpus. Doesn't seem to be reliable on my MacBook Pro though. Not sure if this is CPU scaling, multi-core issues or what. It also seems to be incrementing a lot slower then the CPU speed. 2. Use HPET in the Intel cpu's. I see how to do this under Linux in userspace with recent 2.6.x kernels, but I can't figure out how/if the Darwin kernel exposes the HPET timer. Any sample code for this would be great. 3. Write to IO port 0x80 on Intel. Should take ~1usec. OS X has inb/outb to read/write, but no ioperm (like Linux does) to allow access which is denied by default. I've heard rumors that both #2 and #3 can be implemented using IOKit, but after doing some searches on the apple website, I still have no idea how to do this. Honestly, 1usec is a bit more accurate then I need (10usec is probably good enough), but 1ms is much too slow. Ideally, the timer setup should have as little overhead as possible since this is only going to be used for short periods of time where things like nanosleep() or select() won't work. Or if Darwin/OS X has an alternative method that would be great too... even better if it's more cross platform. -- Aaron Turner http://synfin.net/ http://tcpreplay.synfin.net/ - Pcap editing & replay tools for Unix They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety. -- Benjamin Franklin _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: [Fwd: Re: Getting last data from child process in Leopard]
by Terry Lambert 26 Mar '08

26 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -- Terry On Mar 25, 2008, at 7:22 PM, Steve Checkoway <s(a)pahtak.org> wrote: Oops, didn't mean to send this off list. Ah, I misunderstood. I thought you were saying he needed to check for EAGAIN because signals were interrupting the system call since that's what you were talking about in the sentences before and you only mentioned the nonblocking afterward. Of course he needed to check for EAGAIN because of the nonblocking. (I'm still not really sure why he wants that there in the first place though.) Sure. -- Steve Checkoway -- Steve Checkoway _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/tlambert%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… EINTR is possible, but tends to be rare. This is because signals are implemented using Mach ASTs, which means that even if you send a signal, if the operation in progress can be completed prior to running up to the user/kernel boundary, then it will be. The AST only triggers the signal trampoline at that point. If it successfully completes with no blocking, then an EINTR will not be returned, and instead after the signal handler (if any) or the default action (if it is to terminate the program) have had an opportunity to return, then the system call will complete successfully instead. In the case of a non-blocking fd, the request will almost always be able to complete without hitting an operation that will trigger an EINTR return. You should probably look for both the EAGAIN and EINTR, particularly if you you plan on your code being portable to other UNIX platforms. Jordan K. Hubbard wrote: On Mar 25, 2008, at 6:39 PM, Steve Checkoway wrote: Jordan K. Hubbard wrote: The right thing to do is check for EAGAIN in your read() loop and go back and retry for the data in that case. Given that you're using O_NONBLOCK, it's even more important to make sure you handle all the async event cases. Do you mean EINTR? "If a signal is caught during the systems calls listed below [including read(2)], the call may be forced to terminate with the error EINTR, the call may return with a data transfer shorter than requested, or the call may be restarted." Nope, I do mean EAGAIN. In the context of non-blocking I/O, it's precisely what he needs to check for in this case, at least on MacOSX (other Unix variants may return EINTR instead - this is one of the grey areas of Unix behavior). If you try his example checking for EINTR on Leopard, at least, you'll still fail to catch that last "BYE" (try it). I didn't manage to get the signal to interrupt the read(), but that's probably because of the nonblocking IO. I'm not sure why you think you need unbuffered IO to avoid deadlocks or to deliver data quickly. As you probably noticed, printf() on the child side was buffering anyway. Try shortening the sleep(2) intervals. You can get the race condition to happen fairly easily - I didn't express that change in my diffs since I didn't think it was relevant to the advice I was trying to give him... I didn't manage to get read() to be interrupted even when I removed the sleep call in the parent. I also didn't get it when I changed from nonblocking to blocking, but maybe it is just restarting the calls (I'm not seeing the default restart behavior listed in the man page). That said, I don't doubt that the signal can interrupt it. I just don't think that was his problem since he said it happened every time and I can't get it to happen even once (in my oh-so-scientific sample size of about 15 runs). This email sent to tlambert(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

[Fwd: Re: Getting last data from child process in Leopard]
by Steve Checkoway 26 Mar '08

26 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Thunderbird 2.0.0.12 (X11/20080213) Oops, didn't mean to send this off list. On Mar 25, 2008, at 6:39 PM, Steve Checkoway wrote: Ah, I misunderstood. I thought you were saying he needed to check for EAGAIN because signals were interrupting the system call since that's what you were talking about in the sentences before and you only mentioned the nonblocking afterward. Of course he needed to check for EAGAIN because of the nonblocking. (I'm still not really sure why he wants that there in the first place though.) Sure. I didn't manage to get read() to be interrupted even when I removed the sleep call in the parent. I also didn't get it when I changed from nonblocking to blocking, but maybe it is just restarting the calls (I'm not seeing the default restart behavior listed in the man page). That said, I don't doubt that the signal can interrupt it. I just don't think that was his problem since he said it happened every time and I can't get it to happen even once (in my oh-so-scientific sample size of about 15 runs). -- Steve Checkoway -- Steve Checkoway _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Jordan K. Hubbard wrote: Jordan K. Hubbard wrote: The right thing to do is check for EAGAIN in your read() loop and go back and retry for the data in that case. Given that you're using O_NONBLOCK, it's even more important to make sure you handle all the async event cases. Do you mean EINTR? "If a signal is caught during the systems calls listed below [including read(2)], the call may be forced to terminate with the error EINTR, the call may return with a data transfer shorter than requested, or the call may be restarted." Nope, I do mean EAGAIN. In the context of non-blocking I/O, it's precisely what he needs to check for in this case, at least on MacOSX (other Unix variants may return EINTR instead - this is one of the grey areas of Unix behavior). If you try his example checking for EINTR on Leopard, at least, you'll still fail to catch that last "BYE" (try it). I didn't manage to get the signal to interrupt the read(), but that's probably because of the nonblocking IO. I'm not sure why you think you need unbuffered IO to avoid deadlocks or to deliver data quickly. As you probably noticed, printf() on the child side was buffering anyway. Try shortening the sleep(2) intervals. You can get the race condition to happen fairly easily - I didn't express that change in my diffs since I didn't think it was relevant to the advice I was trying to give him... This email sent to site_archiver(a)lists.apple.com

1 0

Re: Getting last data from child process in Leopard
by Jordan K. Hubbard 26 Mar '08

26 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Mar 25, 2008, at 6:39 PM, Steve Checkoway wrote: - Jordan _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Jordan K. Hubbard wrote: Your signal handler ("deadbeef") also should not be attempting to read from the pty since there are a lot of operations not considered safe from signal handlers (your results are guaranteed to be "undefined", if not outright incorrect). According to the man page for sigaction(2), read(2) is one of the base interfaces that is "either reentrant or not interruptible by signals and are async-signal safe [and therefore] applications may invoke them, without restriction, from signal-catching functions. I chose my words fairly carefully in saying "there are a lot of operations not considered safe" and I didn't call read(2) specifically unsafe since, technically, it is. However, there's a difference between the letter of the law and best practices, and best practices have always been to do as little as humanly possible from a signal handler (setting a flag, for example). Given that, I'll always recommend that people not try to be clever in their signal handlers, particularly where file I/O is concerned given that even with async safety in the calls, it's impossible to know _when_ the file operation will take place and races abound. Programmers are rarely as clever as they think they are, so it's better to advise them to stay away from the fringes of what is possible, but unwise, to do with the Unix APIs. The right thing to do is check for EAGAIN in your read() loop and go back and retry for the data in that case. Given that you're using O_NONBLOCK, it's even more important to make sure you handle all the async event cases. Do you mean EINTR? "If a signal is caught during the systems calls listed below [including read(2)], the call may be forced to terminate with the error EINTR, the call may return with a data transfer shorter than requested, or the call may be restarted." Nope, I do mean EAGAIN. In the context of non-blocking I/O, it's precisely what he needs to check for in this case, at least on MacOSX (other Unix variants may return EINTR instead - this is one of the grey areas of Unix behavior). If you try his example checking for EINTR on Leopard, at least, you'll still fail to catch that last "BYE" (try it). I didn't manage to get the signal to interrupt the read(), but that's probably because of the nonblocking IO. I'm not sure why you think you need unbuffered IO to avoid deadlocks or to deliver data quickly. As you probably noticed, printf() on the child side was buffering anyway. Try shortening the sleep(2) intervals. You can get the race condition to happen fairly easily - I didn't express that change in my diffs since I didn't think it was relevant to the advice I was trying to give him... This email sent to site_archiver(a)lists.apple.com

1 0

Re: Getting last data from child process in Leopard
by Steve Checkoway 26 Mar '08

26 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com User-agent: Thunderbird 2.0.0.12 (X11/20080213) EAGAIN comes from the nonblocking with no data ready. Ingemar, -- Steve Checkoway _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Jordan K. Hubbard wrote: Your signal handler ("deadbeef") also should not be attempting to read from the pty since there are a lot of operations not considered safe from signal handlers (your results are guaranteed to be "undefined", if not outright incorrect). According to the man page for sigaction(2), read(2) is one of the base interfaces that is "either reentrant or not interruptible by signals and are async-signal safe [and therefore] applications may invoke them, without restriction, from signal-catching functions. The right thing to do is check for EAGAIN in your read() loop and go back and retry for the data in that case. Given that you're using O_NONBLOCK, it's even more important to make sure you handle all the async event cases. Do you mean EINTR? "If a signal is caught during the systems calls listed below [including read(2)], the call may be forced to terminate with the error EINTR, the call may return with a data transfer shorter than requested, or the call may be restarted." Rather than spam the list with my version of your code, you can get it here: <http://pahtak.org/~steve/child.c>. I would have just posted a diff like Jordan did, but I couldn't handle your inconsistent spacing. It seems to work correctly for me written that way. The signal handler in my version does nothing but set a flag and the while loop ends, not when the handler fires, but when read() returns 0, i.e., end of file. I didn't manage to get the signal to interrupt the read(), but that's probably because of the nonblocking IO. I'm not sure why you think you need unbuffered IO to avoid deadlocks or to deliver data quickly. As you probably noticed, printf() on the child side was buffering anyway. As a side note, this runs on (at least this particular flavor of) Linux as well with the small changes (included in the file) to include <pty.h> instead of <util.h> and to break out of the loop when read() returns -1 and errno is EIO. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Getting last data from child process in Leopard
by Jordan K. Hubbard 25 Mar '08

25 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Mar 24, 2008, at 12:44 PM, Ingemar Ragnemalm wrote: - Jordan --- forkpty.c.orig 2008-03-25 15:35:16.000000000 -0700 +++ forkpty.c 2008-03-25 15:34:22.000000000 -0700 @@ -57,6 +57,7 @@ // Wait for result while (!done) { +try_again: i = 255; // buffer size i = read(pty, s, i); if (i>0) @@ -64,8 +65,11 @@ s[i] = 0; printf("Read %d: <%s>\n", i, s); } - else + else { printf("*"); + if (errno == EAGAIN) + goto try_again; + } sleep(1); } _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I am new to the list, so I apologize if this is common knowledge. I have searched the archives (and Google) and come up with nothing useful. I have an application which works nicely in Tiger, but has some major problems in Leopard. I spawn sub-processes with forkpty, and communicate with read and write (the standard Unix calls). I use forkpty in order to get unbuffered data, to avoid deadlocks and to deliver data promptly to the user. Under Tiger, this works very well. Leopard behaves very differently. This is because your code is very racy. Signals can interrupt system calls, including read(2), and what's happening is that your SIGCHLD is causing the read to be interrupted and miss data. Your signal handler ("deadbeef") also should not be attempting to read from the pty since there are a lot of operations not considered safe from signal handlers (your results are guaranteed to be "undefined", if not outright incorrect). The right thing to do is check for EAGAIN in your read() loop and go back and retry for the data in that case. Given that you're using O_NONBLOCK, it's even more important to make sure you handle all the async event cases. Here is a diff to your code which will make it work less by luck, as it did on Tiger, and more by design. I'm sure that this is also just sample code and not something you're trying to really do in production since looping on a non-blocking fd waiting for input is very inefficient. That's what poll(2) is for. % uname -a Darwin watusi9.apple.com 9.2.2 Darwin Kernel Version 9.2.2: Tue Mar 4 21:23:43 PST 2008; root:xnu-1228.4.31~1/RELEASE_PPC Power Macintosh This email sent to site_archiver(a)lists.apple.com

1 0

Re: Listing System V Semaphores on Tiger
by Terry Lambert 25 Mar '08

25 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Mar 25, 2008, at 6:40 AM, Ray Ruvinskiy wrote: I seem to be unable to get a list of currently existing System V semaphores on Tiger using the ipcs utility. The list is always empty, whether or not I run the command as root. On Leopard, however, ipcs works as expected. Is this a known issue, or is there something I'm missing? The kernel support for the ipcs code and the structure changes necessary in the contract between kernel and user space was considered to risky to put in Tiger, so Tiger does not support it. This email sent to site_archiver(a)lists.apple.com

1 0

Re: AbsoluteTime & IOHIKeyboard's _keyboardEventAction
by Matt Burnett 25 Mar '08

25 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:cc:message-id:from:to:in-reply-to:content-type:mime-version:subject:date:references:x-mailer; bh=ewsS92PSRaKJ5EhDOGdwHmpiU1FthC53g6EOoPThgt0=; b=nycTcJPVwHG0fd3xLBtKttLgujT5kIyGV1ScOdL75joO4S4nAXgFvvrf29eeX5s2EgCCrGdty4YcbvfdHMjU6TA8XxpgpDMOYKX15Upvc42AqZ9RSZs20LT05h8GDuO67UgyzEeiBdDWZZkpATm/4SMd4GJDGY9GQz6nTitIv60= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=cc:message-id:from:to:in-reply-to:content-type:mime-version:subject:date:references:x-mailer; b=agfuW+IbK5thkDCvWAiOeAvUg1sxJQQWFs4cr3Sy4TDA1mA9cibejdkyrW0C9TsNhc4x1P4z5sBciuydEzxTizgaZ4Psw2haGl+e2CQo5dG06l8OkCi6expt1W/nbnmE35pBYWKuys7i+wCFjmP2KAXzVNHZHm9V41jDVwnFlZA= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: darwinbuild and Xcode 3?
by William Siegrist 25 Mar '08

25 Mar '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -Bill Error: /usr/bin/xcode-select not found. I saw this from Kevin awhile back: I have no idea what that means. ---- William Siegrist Software Support Engineer Mac OS Forge http://macosforge.org/ wsiegrist(a)apple.com 408 862 7337 _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com I'm actually working on this right now. I hope to have fixes in the darwinbuild repo soon. Also, I think darwinbuild-dev(a)lists.macosforge.org might be a better list for this topic. On Mar 25, 2008, at 1:05 PM, James Reynolds wrote: Has darwinbuild been updated for Xcode 3 yet? I'm trying to build DirectoryServices and I get this error: I suggest invoking xcodebuild similarly to how darwinbuild would invoke it using the darwintrace library (or alternatively using dtruss) to record which files are accessed during the build. This will give an estimate of what's needed in the chroot. Is drawinbuild updated for 3.0 yet or are there workarounds to get it to work? James _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/wsiegrist %40apple.com This email sent to wsiegrist(a)apple.com smime.p7s

1 0