January 2008 - Darwin-dev - lists.apple.com

Re: fread/fwrite/fflush bug
by A.M. 08 Jan '08

08 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jan 8, 2008, at 10:38 AM, Scott D. Davilla wrote: Don't use buffered I/O. http://www.opengroup.org/onlinepubs/007908799/xsh/setvbuf.html Or use a shared mmap(). Cheers, M _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I too am seeing a similar problem. In my case, We have one producer thread that does an fopen for write only, and several consumer threads that do fopen for read only. The consumer threads are only able to read the number of bytes in the file that existed when the fopen was called. They never see any additional data from the producer thread. These files can be very large, they are position (x,y) with electron and x-ray events streamed in realtime from networked hardware. This is a real problem with our existing code as it's design depends on the above ability. We just moved from CFM to Mach-O (yea, yea, I know). Works fine with CW and CFM (and Borland, we are cross-platform). Any suggestions for a solution? This email sent to site_archiver(a)lists.apple.com

1 0

Re: fread/fwrite/fflush bug
by Scott D. Davilla 08 Jan '08

08 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -- _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: pthread_cond_wait() missing broadcast?
by Terry Lambert 07 Jan '08

07 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com I'm using 10.5.1 on a dual-core iMac. Or In general, calling the: pthread_cond_wait(cond, mutex); -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Jan 7, 2008, at 11:49 AM, Pierre Baillargeon wrote: Here's the question currently on my mind. Explanation about why I'm asking follow: Does pthread_cond_wait() releases the mutex before waiting on the condition in a manner that could make it lose a broadcast? Can pthread_cond_broadcast() only wake up one thread when two threads are waiting on the condition variable, leaving the other one behind? I've tried to follow the code of __pthread_cond_wait() in the Darwin source code, but I've been stumped by the fact that the mutex being passed in is not explicitly unlocked anywhere! The variable is checked to verify that it is indeed a mutex but is not otherwise used anywhere else in the code, making me wonder where and how it can possibly be released. I've gone down to wait_queue_assert_wait64_locked(). Is the unlock somehow implicit? The problem arises in the following situation: I have 3 threads and the goal is that a "central gate" blocks them until all 3 have finished their work. The code roughly is: int count = 0; // Global variable (really a class variable in the Gate implemetation) pthread_mutex_lock(mutex); if( ++count == 3 ) { count = 0; pthread_cond_broadcast(cond); } else { // Ignore spurious wake-up in this example. // Even if spuriously waken-up, it would not affect the issue. pthread_cond_wait(cond, mutex); } pthread_mutex_unlock(mutex); The state I witness is that 2 threads successfully exit but the 3rd thread is blocked in the pthread_cond_wait(). More mysteriously, count == 1. (Could be due to some cache coherency issues? Or could the processor re-order memory write? I've looked at the disassembly and the increment is fully after the mutex lock and before the cond broadcast, so it's not a compiler optimization issue.) But the only way the other two threads can possibly have escaped this function is to for the count to have reached 3, one thread being woken up by the broadcast, the other having done the broadcast. So the only situations I can see that would lead to the state I'm witnessing is either: 1. pthread_cond_wait() releases the mutex before putting teh thread on the wait queue, missing the broadcast. 2. pthread_cond_broadcast() only waking up one thread. I've seen other discussions saying that if a signal arrives on a thread sleeping on a condition variable, that thred could miss the broadcast. Is that exact? It could explain the missing wake up. Whatever mutex you were waiting on is acquired by thread before it's allowed to run, just as if you had explicitly attempted a lock. <http://www.opengroup.org/onlinepubs/009695399/functions/pthread_cond_broadc… > You are most likely going back o sleep attempting to acquire the mutex in multiple threads, and the one thread unlocks, which unblocks the remaining two, and then they race to sleep, and the other unlock either doesn't happen, or misses it's wakeup because of your order of operation. implies that you intend to do something with tthe resource protected by the mutex, which would normally prevent a race. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Posix Timers?
by Terry Lambert 07 Jan '08

07 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com % grep TIMER /usr/include/unistd.h #define _POSIX_TIMERS (-1) /* [TMR] */ So no. The proper way to feature test POSIX interfaces is: -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Jan 5, 2008, at 10:06 AM, Steve Checkoway wrote: On Jan 5, 2008, at 8:53 AM, Andre-John Mas wrote: Could someone tell me whether MacOS X supports POSIX timers? If you mean the api specified in <http://www.jaluna.com/doc/c5/html/Overview/x5408.html > (just the first hit on google), then the answer would seem to be no: $ find /usr/include -name '*.h'|xargs egrep 'clock_(settime|gettime| getres)|timer_(create|delete|settime|gettime|getoverrun)|nanosleep' /usr/include/php/ext/standard/ basic_functions.h:PHP_FUNCTION(time_nanosleep); /usr/include/php/main/php_config.h:/* Define if you have the nanosleep function. */ /usr/include/time.h:int nanosleep(const struct timespec *, struct timespec *) __DARWIN_ALIAS_C(nanosleep); Only nanosleep seems to be there. Specifically, all POSIX functionality should be feature tested using <unistd.h> macros before you assume it's there; for POSIX Timers: #include <unistd.h> ... #if defined(_POSIX_TIMERS) && (_POSIX_TIMERS - 200112L) >= 0L /* This POSIX feature is supported */ ... #else /* This POSIX feature is not supported; use an alternate implementation */ ... #endif This email sent to site_archiver(a)lists.apple.com

1 0

RE: fread/fwrite/fflush bug (Norm Green)
by David Hoerl 07 Jan '08

07 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Date: Mon, 7 Jan 2008 09:49:19 -0800 From: Norm Green <norm.green(a)gemstone.com> Subject: RE: fread/fwrite/fflush bug ... Someone pointed out the fpurge() function. Calling fpurge() on the reader before the 2nd read fixes the problem. David _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This might be one fix for your code, but how about other code not yet ported to OSX that is looking for "proper" behavior (making an assumption here). Reading the man page for fpurge, it says: "For input streams this discards any input read from the underlying object but not yet obtained via getc(3)...". I interpret this to mean the following would fail (but not your example): - writer writes say 1000 bytes - reader is told to read one byte at location 0 (but fread gets a buffer full) - writer changes all the data - all 1000 bytes - reader reads location 1, and gets old data (which it already had in the buffer) - reader keeps reading a byte at a time. When the internal buffer is exhausted THEN it would start reading the actual file again, and get a buffer of correct data. Based on the man page, one could assume that the seek backwards should cause the data in the buffer to get flushed as if fpurge was called. Apparently fread looks to see if the requested bytes are located in its buffer in all cases, and if so, uses them. This email sent to site_archiver(a)lists.apple.com

1 0

Re: fread/fwrite/fflush bug
by Stephen Hoffman 07 Jan '08

07 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Organization: HoffmanLabs LLC User-agent: Thunderbird 2.0.0.9 (Macintosh/20071031) Norm Green writes: There are platforms that can handle this, and platforms that can't. Failed opening the reader: non-translatable vms error code: 0x1828A %rms-e-flk, file currently locked by another user Failed when calling fread() on the reader: error 0 But I'd not tend to depend on sharing within a file in portable code. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I've found what I consider to be a stream IO bug. Not the answer you want, but -- since you're apparently looking for portability -- I'll send it anyway. Write sharing within a file is usually a somewhat hazardous assumption. It's not particularly portable. Here's what was received when invoking this (ported) test case on another platform: Sharing was explicitly blocked by the platform (because of the issues that can arise), and the platform (as expected) punted this request. The program source was tweaked to allow file-level sharing (including distributed locking and sharing, which the platform can provide), and received: Having worked with the code some, it doesn't directly port -- not without more time than can be assigned to the effort right now. There are various other portability concerns here, which can be provided off-list -- if you're interested. If there's a language lawyer on the list, s/he can let us know what write-sharing semantics exist in this environment. This email sent to site_archiver(a)lists.apple.com

1 0

pthread_cond_wait() missing broadcast?
by Pierre Baillargeon 07 Jan '08

07 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com I'm using 10.5.1 on a dual-core iMac. Or -- Pierre Baillargeon _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Here's the question currently on my mind. Explanation about why I'm asking follow: Does pthread_cond_wait() releases the mutex before waiting on the condition in a manner that could make it lose a broadcast? Can pthread_cond_broadcast() only wake up one thread when two threads are waiting on the condition variable, leaving the other one behind? I've tried to follow the code of __pthread_cond_wait() in the Darwin source code, but I've been stumped by the fact that the mutex being passed in is not explicitly unlocked anywhere! The variable is checked to verify that it is indeed a mutex but is not otherwise used anywhere else in the code, making me wonder where and how it can possibly be released. I've gone down to wait_queue_assert_wait64_locked(). Is the unlock somehow implicit? The problem arises in the following situation: I have 3 threads and the goal is that a "central gate" blocks them until all 3 have finished their work. The code roughly is: int count = 0; // Global variable (really a class variable in the Gate implemetation) pthread_mutex_lock(mutex); if( ++count == 3 ) { count = 0; pthread_cond_broadcast(cond); } else { // Ignore spurious wake-up in this example. // Even if spuriously waken-up, it would not affect the issue. pthread_cond_wait(cond, mutex); } pthread_mutex_unlock(mutex); The state I witness is that 2 threads successfully exit but the 3rd thread is blocked in the pthread_cond_wait(). More mysteriously, count == 1. (Could be due to some cache coherency issues? Or could the processor re-order memory write? I've looked at the disassembly and the increment is fully after the mutex lock and before the cond broadcast, so it's not a compiler optimization issue.) But the only way the other two threads can possibly have escaped this function is to for the count to have reached 3, one thread being woken up by the broadcast, the other having done the broadcast. So the only situations I can see that would lead to the state I'm witnessing is either: 1. pthread_cond_wait() releases the mutex before putting teh thread on the wait queue, missing the broadcast. 2. pthread_cond_broadcast() only waking up one thread. I've seen other discussions saying that if a signal arrives on a thread sleeping on a condition variable, that thred could miss the broadcast. Is that exact? It could explain the missing wake up. This email sent to site_archiver(a)lists.apple.com

1 0

Re: IPv6 connection issues
by Josh Graessley 07 Jan '08

07 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jan 7, 2008, at 11:30 AM, Daniel J. Luke wrote: -josh _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Jan 7, 2008, at 1:59 PM, Andre-John Mas wrote: The Apache2 server responds to IPv6, when I query it from locally, using http://[::1], using Safari. On the other hand when I try accessing the server, from my Intel machine, using the IPv6 address specifed by the en0 interface I can't connect: http://[fe80::230:65ff:fed6:b164] Is your Intel machine on the same broadcast domain as your apache2 server? The fe80:: address is a link-local (automatically configured) address. Think of it like the 169.254.0.0/16 range used for DHCP autoconfiguration. 169.254.0.0/16 is actually IPv4LL and has nothing to do with DHCP except that Macs and PCs tend to fall back on IPv4LL if they fail to get a DHCP address. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Re: IPv6 connection issues
by Andre-John Mas 07 Jan '08

07 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com > Is your Intel machine on the same broadcast domain as your > apache2 server? It should be. The server machine is 192.168.2.101 and the portable is 192.168.2.3. They are both connected directly to the same ADSL gateway/router. > You should be able to ping6 fe80::230:65ff:fed6:b164, do you > perhaps have some firewall set up? What does your routing > table (netstat -rn) look like? I don't know whether I my firewall is blocking IPv6, but the machine can't ping itself when using the non- [::1] address. The routing table is as follows: Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 192.168.2.1 UGSc 22 87 en0 127 127.0.0.1 UCS 0 0 lo0 127.0.0.1 127.0.0.1 UH 13 1324904 lo0 169.254 link#4 UCS 0 0 en0 192.168.2 link#4 UCS 3 0 en0 192.168.2.1 0:30:bd:8f:43:7c UHLW 23 2 en0 1163 192.168.2.3 link#4 UHLW 1 18669 en0 192.168.2.101 127.0.0.1 UHS 1 90 lo0 192.168.2.255 link#4 UHLWb 2 6221 en0 Internet6: Destination Gateway Flags Netif Expire ::1 link#1 UHL lo0 2001:4978:f:48::2 link#9 UHL lo0 fe80::%lo0/64 fe80::1%lo0 Uc lo0 fe80::1%lo0 link#1 UHL lo0 fe80::%en0/64 link#4 UC en0 fe80::230:65ff:fed6:b164%en0 0:30:65:d6:b1:64 UHL lo0 fe80::230:65ff:fed6:b164 link#9 UHL lo0 ff01::/32 ::1 U lo0 ff02::/32 ::1 UC lo0 ff02::/32 link#4 UC en0 Andre _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: IPv6 connection issues
by Daniel J. Luke 07 Jan '08

07 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -- Daniel J. Luke +========================================================+ | *---------------- dluke(a)geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+ _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com On Jan 7, 2008, at 1:59 PM, Andre-John Mas wrote: The Apache2 server responds to IPv6, when I query it from locally, using http://[::1], using Safari. On the other hand when I try accessing the server, from my Intel machine, using the IPv6 address specifed by the en0 interface I can't connect: http://[fe80::230:65ff:fed6:b164] Is your Intel machine on the same broadcast domain as your apache2 server? The fe80:: address is a link-local (automatically configured) address. Think of it like the 169.254.0.0/16 range used for DHCP autoconfiguration. responds, but ping to either fe80::230:65ff:fed6:b164 or fe80::230:65ff:fed6:b164%en0 (its own address) responds in: You should be able to ping6 fe80::230:65ff:fed6:b164, do you perhaps have some firewall set up? What does your routing table (netstat -rn) look like? PGP.sig

1 0