January 2008 - Darwin-dev - lists.apple.com

Re: to read larger files using POSIX functions
by Navneet Kumar 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com i'm using open() and pread() programatically. On 11-Jan-08, at 1:32 AM, Navneet Kumar wrote: Regards Navneet On 11-Jan-08, at 1:22 AM, Dave Zarzycki wrote: davez On Jan 10, 2008, at 11:38 AM, Navneet Kumar wrote: Wishes, Navneet _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/zarzycki%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… but when i read my file beyond 2 GB it shows different data every time, suggesting that it failed. because the data inside the file remains the same. There is nothing special required to read/write large files on Mac OS X. Hi, I'm using open() and pread() to read a file. But to open large files when I use open() with O_LARGEFILE, i get the error that O_LARGEFILE is undefined. I have included <fcntl.h>. Also is there any thing else i need to do to correctly use pread() for reading large files. This email sent to zarzycki(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: to read larger files using POSIX functions
by Navneet Kumar 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Regards Navneet On 11-Jan-08, at 1:22 AM, Dave Zarzycki wrote: davez On Jan 10, 2008, at 11:38 AM, Navneet Kumar wrote: Wishes, Navneet _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… but when i read my file beyond 2 GB it shows different data every time, suggesting that it failed. because the data inside the file remains the same. There is nothing special required to read/write large files on Mac OS X. Hi, I'm using open() and pread() to read a file. But to open large files when I use open() with O_LARGEFILE, i get the error that O_LARGEFILE is undefined. I have included <fcntl.h>. Also is there any thing else i need to do to correctly use pread() for reading large files. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/zarzycki %40apple.com This email sent to zarzycki(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: to read larger files using POSIX functions
by Dave Zarzycki 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com There is nothing special required to read/write large files on Mac OS X. davez On Jan 10, 2008, at 11:38 AM, Navneet Kumar wrote: Wishes, Navneet _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/zarzycki%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Hi, I'm using open() and pread() to read a file. But to open large files when I use open() with O_LARGEFILE, i get the error that O_LARGEFILE is undefined. I have included <fcntl.h>. Also is there any thing else i need to do to correctly use pread() for reading large files. This email sent to zarzycki(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

to read larger files using POSIX functions
by Navneet Kumar 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Wishes, Navneet _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Hi, I'm using open() and pread() to read a file. But to open large files when I use open() with O_LARGEFILE, i get the error that O_LARGEFILE is undefined. I have included <fcntl.h>. Also is there any thing else i need to do to correctly use pread() for reading large files. This email sent to site_archiver(a)lists.apple.com

1 0

Re: #define xxx #define yyy 123
by Greg Shenaut 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com How do you get a define to be allowed within a define.... Greg Shenaut _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… #define xxx #define yyy 123 Isn't this the kind of thing people have done very often using a tool like sed? For example, let the string __XXX__ stand for "#define yyy 123" and then filtering the source files before the compilation process begins with something like 's/^ *__XXX__/\#define yyy 123/'. I think that would be much easier that trying to make multiple cpp passes. Of course, putting the definitions inside the source file would complicate things, but it would still be doable. This email sent to site_archiver(a)lists.apple.com

1 0

Re: A Bug in pthread_cond_destroy?
by Terry Lambert 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jan 10, 2008, at 5:09 AM, Kazuho Oku <kazuhooku(a)gmail.com> wrote: Yes, I saw the quote about it being "safe to destroy a mutex on which no thread is waiting" but the fact that the sentence and paragraph before said that attempting to use such a destroyed condition would result in undefined behavior still holds. Well, my example never calls pthread_cond_wait AFTER calling pthread_cond_destroy. Maybe I was vague on my explatation, what happens is: 1) thread-A enters pthread_cond_wait 2) thread-B enters pthread_cond_signal 3) thread-A returns from pthread_cond_wait 4) thread-A calls pthread_cond_destroy 5) thread-A returns from pthread_cond_destroy (returns 0) 6) thread-B read & writes the conditional 7) thread-B returns from pthread_cond_signal which from my understanding is the exact case the spec describes as "it shall be safe to destroy an initialized condition variable upon which no threads are currently blocked." -- Terry Or if what you are trying to say is that the above sequence is valid by the pthread specification, sorry for bothering your time. In your case, the destroy _was_ done safely (the destroy didn't hang or have any other odd behavior, correct?). It was the attempt to use the destroyed condition variable that caused problems - and that is exactly what the spec says is undefined. So, I think our implementation meets the spec. --Jim On Jan 9, 2008, at 8:01 AM, Kazuho Oku wrote: 2008/1/9, Jim Magee <jmagee(a)apple.com>: Thas said, I googled and found a spec. that includes your quotes (http://www.opengroup.org/onlinepubs/007908775/xsh/pthread_cond_destroy.html ), and if you are referrring to it, my understaning is that it states It shall be safe to destroy an initialized condition variable upon which no threads are currently blocked. which is the case in my example. My understanding is that their is the case where a waiting thread get signalled and woke up and may destroy the cond before the signalling thread finishes modifying the pthread_cond_t variable. Or if there's any other version, would you please let me know? Thank you in advance. --Jim On Jan 9, 2008, at 2:25 AM, Kazuho Oku wrote: Hi, For example, in the following code, thread B would sometimes read from and/or write to freed memory. Thread A: pthread_cond_wait(cond, &mutex); while ((err = pthread_cond_destroy(cond)) != 0) { assert(err == EBUSY); usleep(1); } free(cond); Thread B: pthread_cond_signal(global_cond); This email sent to jmagee(a)apple.com -- Kazuho Oku -- Kazuho Oku _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/tlambert%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… 2008/1/10, Jim Magee <jmagee(a)apple.com>: Safe to destroy it, not safe to do anything else. Your step 6 is only legal on on an undestroyed mutex. I think you are assuming behavior that isn't specified. Specifically, the definition for pthread_cond_destroy() says: The pthread_cond_destroy() function shall destroy the given condition variable specified by cond; the object becomes, in effect, uninitialized. An implementation may cause pthread_cond_destroy() to set the object referenced by cond to an invalid value. A destroyed condition variable object can be reinitialized using pthread_cond_init(); the results of otherwise referencing the object after it has been destroyed are undefined. Thank you for your response, I did not know such a statement exists in pthread spec., maybe I was referring to an older document. So, trying to call pthread_cond_signal() from another thread after the condition is destroyed (or even "during" it being destroyed) can result is undefined behavior. You really need synchronization of your own to avoid issues there. Using the same mutex would have been good enough. Since that isn't in your model, you need something else (maybe a second condition variable that indicates whether the first has been destroyed? - just thinking out loud). If this is not the right place to report bugs of libc, please tell me the right mailing list. Thank you in advance, and sorry, if it is the case. I think I have found a bug in pthread_cond_destroy of Libc (with Mac OS X 10.4.10). Current implementation of pthread_cond_destroy does not check if sigspending==0. Thus if pthread_cond_destroy is called from a thread other than that called pthread_cond_signal, the destructor may return 0 even if pthread_cond_signal is still in work (i.e. sigspending != 0), leading to memory corruption or an infinite loop in pthread_cond_signal. Note that the problem only arises when thread B does not lock the same mutex as thread A when calling thread A (this is not a requirement in POSIX threads). Attached to this mail is a code that would reproduce this bug. It sends and receives pthread_cond_signals and increment a counter. It compiled without any compile time defines, the program prints the counter incrementing infinitely. But if compiled with -DCLEAR_MEM -DSIGNAL_WO_LOCK it would suddenly stop due to memory corruption. Or if compiled with -DAPPLE_TEST -DSIGNAL_WO_LOCK it would check the value of sispending and print an assertion failure. To reproduce the bug with the attached code, a dual processor system might be a requirement. -- Kazuho Oku <thtest.c> _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/jmagee% 40apple.com This email sent to tlambert(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: #define xxx #define yyy 123
by Terry Lambert 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -- Terry On Jan 10, 2008, at 4:09 AM, charlie <uber(a)juun.com> wrote: How do you get a define to be allowed within a define.... #define xxx #define yyy 123 More specifically, what I'm trying to do is this: #define begin_enumerations #pragma options enum=4 But the compiler gives me the following error: error: '#' is not followed by a macro parameter error: syntax error at '#' token Is there some way to force the preprocessor to make a second pass? - chuck _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/tlambert%40apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… You can't do this in the preprocessor alone. You can cause multiple passes using a Makefile with intermediate target rules, if you absolutely must, but it's highly discouraged unless you are entering an obfuscated C contest. It's interesting though.. I get the following error on the line where I use begin_enumerations: ... as if the #pragma options enum=4 actually made it down there to that line. This email sent to tlambert(a)apple.com This email sent to site_archiver(a)lists.apple.com

1 0

IOKit: Unable to catch shutdown for power management(QA 1340)
by parag vibhute 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; bh=NZPUHY1/rQccuGtYXxcZVvNl7Ep28VMTklmRBRUgYUg=; b=GddyDujeyPQze+DlqpI6k1QTvWcnDR5cqKYIUgUYRw6S7DALJxCrV5sybt2Hq3gyXuFpo8vNwwN+WNZZcUvJgBFAaJIhO3X+rC2l/cx2hO5qmOWsgHyHP155GX6zi2HkGtbhP2h8D86MzjennpykL+xLOKmNMI7nCIDQ6UDEZ7A= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=WsjFM47nowfoupU4yhHGDqtFO5ZZ8BHytriqIeP5WQQxB4eRe+pI5COz89nBryotgsmi4qq5rMprIuLfio8xI3EQKtITskmK3dVPTmP1dxd+6n1CjJTU8y8A0jUDfaZVIHz7zK/ztxU4ex4T2JiMzy5INfwc6rG71aWR5+6+EmE= _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: #define xxx #define yyy 123
by seebs＠seebs.net 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com In message <200801101209.m0AC9Yi1014695(a)rs34.luxsci.com>, "charlie" writes: >How do you get a define to be allowed within a define.... You don't. >#define xxx #define yyy 123 Can't be done, at all. >More specifically, what I'm trying to do is this: >#define begin_enumerations #pragma options enum=4 This can be done. However, you can't do it like this. You are lookping for _Pragma(options enum=4), which is new in C99 specifically because what you want was impossible in C89. >Is there some way to force the preprocessor to make a second pass? Not like this, no. -s _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: A Bug in pthread_cond_destroy?
by Kazuho Oku 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=7PaM1Ts6GShpTmphDndpA1EwWGouSf34Txbw2Shn+FE=; b=H+znNMH9Wkn1yk/YGsSh24BOkjfqxh1wYrdUXH8C+pKb65ZruHx4/Snj/71w5KVxBsl8nF6bAJg2S7pics4hUOC6tAbVsdjHa0oC9iiot85wqCcjOdT/XBUL3g5iKD5ZQgmRnnvg1Q9XyazrEl248wfkBY2DjO67vOX8eAmtgpI= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=nHx+d9rpq1uUh+u4gRKTNesNYVftDTeyY18vPorMmeb4SBHvWYUQj+6vc66SvsdTXAGAFVYhTJoQj5DMr3jNX7UGcLCTjJBWUA3gaTkmyoChy3nxTeDR05GudNzA9Bssi0gmda940CDxXGhnCA7KSQ17cPpfJvN/HQxGQyHn69Q= 2008/1/10, Jim Magee <jmagee(a)apple.com>: > Yes, I saw the quote about it being "safe to destroy a mutex on which > no thread is waiting" but the fact that the sentence and paragraph > before said that attempting to use such a destroyed condition would > result in undefined behavior still holds. Well, my example never calls pthread_cond_wait AFTER calling pthread_cond_destroy. Maybe I was vague on my explatation, what happens is: 1) thread-A enters pthread_cond_wait 2) thread-B enters pthread_cond_signal 3) thread-A returns from pthread_cond_wait 4) thread-A calls pthread_cond_destroy 5) thread-A returns from pthread_cond_destroy (returns 0) 6) thread-B read & writes the conditional 7) thread-B returns from pthread_cond_signal which from my understanding is the exact case the spec describes as "it shall be safe to destroy an initialized condition variable upon which no threads are currently blocked." Or if what you are trying to say is that the above sequence is valid by the pthread specification, sorry for bothering your time. > In your case, the destroy _was_ done safely (the destroy didn't hang > or have any other odd behavior, correct?). It was the attempt to use > the destroyed condition variable that caused problems - and that is > exactly what the spec says is undefined. So, I think our > implementation meets the spec. > > --Jim > > On Jan 9, 2008, at 8:01 AM, Kazuho Oku wrote: > > > 2008/1/9, Jim Magee <jmagee(a)apple.com>: > > > >> I think you are assuming behavior that isn't specified. > >> Specifically, the > >> definition for pthread_cond_destroy() says: > >> The pthread_cond_destroy() function shall destroy the given condition > >> variable specified by cond; the object becomes, in effect, > >> uninitialized. An > >> implementation may cause pthread_cond_destroy() to set the object > >> referenced > >> by cond to an invalid value. A destroyed condition variable object > >> can be > >> reinitialized using pthread_cond_init(); the results of otherwise > >> referencing the object after it has been destroyed are undefined. > > > > Thank you for your response, I did not know such a statement exists in > > pthread spec., maybe I was referring to an older document. > > > > Thas said, I googled and found a spec. that includes your quotes > > (http://www.opengroup.org/onlinepubs/007908775/xsh/pthread_cond_destroy.html > > ), > > and if you are referrring to it, my understaning is that it states > > > > It shall be safe to destroy an initialized condition variable upon > > which no threads are currently blocked. > > > > which is the case in my example. My understanding is that their is > > the case where a waiting thread get signalled and woke up and may > > destroy the cond before the signalling thread finishes modifying the > > pthread_cond_t variable. > > > > Or if there's any other version, would you please let me know? > > > > Thank you in advance. > > > >> So, trying to call pthread_cond_signal() from another thread after > >> the > >> condition is destroyed (or even "during" it being destroyed) can > >> result is > >> undefined behavior. You really need synchronization of your own to > >> avoid > >> issues there. Using the same mutex would have been good enough. > >> Since that > >> isn't in your model, you need something else (maybe a second > >> condition > >> variable that indicates whether the first has been destroyed? - just > >> thinking out loud). > >> > >> --Jim > >> > >> > >> On Jan 9, 2008, at 2:25 AM, Kazuho Oku wrote: > >> > >> Hi, > >> > >> If this is not the right place to report bugs of libc, please tell me > >> the right mailing list. Thank you in advance, and sorry, if it is > >> the > >> case. > >> > >> I think I have found a bug in pthread_cond_destroy of Libc (with Mac > >> OS X 10.4.10). > >> > >> Current implementation of pthread_cond_destroy does not check if > >> sigspending==0. Thus if pthread_cond_destroy is called from a thread > >> other than that called pthread_cond_signal, the destructor may return > >> 0 even if pthread_cond_signal is still in work (i.e. sigspending != > >> 0), leading to memory corruption or an infinite loop in > >> pthread_cond_signal. > >> > >> For example, in the following code, thread B would sometimes read > >> from > >> and/or write to freed memory. > >> > >> Thread A: > >> pthread_cond_wait(cond, &mutex); > >> while ((err = pthread_cond_destroy(cond)) != 0) { > >> assert(err == EBUSY); > >> usleep(1); > >> } > >> free(cond); > >> > >> Thread B: > >> pthread_cond_signal(global_cond); > >> > >> Note that the problem only arises when thread B does not lock the > >> same > >> mutex as thread A when calling thread A (this is not a requirement in > >> POSIX threads). > >> > >> Attached to this mail is a code that would reproduce this bug. It > >> sends and receives pthread_cond_signals and increment a counter. > >> It compiled without any compile time defines, the program prints the > >> counter incrementing infinitely. > >> But if compiled with -DCLEAR_MEM -DSIGNAL_WO_LOCK it would suddenly > >> stop due to memory corruption. > >> Or if compiled with -DAPPLE_TEST -DSIGNAL_WO_LOCK it would check the > >> value of sispending and print an assertion failure. > >> > >> To reproduce the bug with the attached code, a dual processor system > >> might be a requirement. > >> > >> -- > >> Kazuho Oku > >> <thtest.c> _______________________________________________ > >> Do not post admin requests to the list. They will be ignored. > >> Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) > >> Help/Unsubscribe/Update your Subscription: > >> http://lists.apple.com/mailman/options/darwin-dev/jmagee%40apple.com > >> > >> This email sent to jmagee(a)apple.com > >> > > > > > > -- > > Kazuho Oku > > -- Kazuho Oku _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0