January 2008 - Darwin-dev - lists.apple.com

#define xxx #define yyy 123
by charlie 10 Jan '08

10 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com How do you get a define to be allowed within a define.... #define xxx #define yyy 123 More specifically, what I'm trying to do is this: #define begin_enumerations #pragma options enum=4 But the compiler gives me the following error: error: '#' is not followed by a macro parameter error: syntax error at '#' token Is there some way to force the preprocessor to make a second pass? - chuck _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… It's interesting though.. I get the following error on the line where I use begin_enumerations: ... as if the #pragma options enum=4 actually made it down there to that line. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Live Streaming DSS problem
by Terry Lambert 09 Jan '08

09 Jan '08

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com Hi all, Also, you might want to read the FAQ: <http://developer.apple.com/opensource/server/streaming/faq.html> -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Jan 9, 2008, at 8:57 AM, Stylianos Chatziperis wrote: I would like to issue a very important question since the behaviour that I observe when I try live streaming with DSS looks crazy to me... :-) I have installed DSS in a LAN and it works correct with streaming of stored content. I have also set up a broadcaster (VLC)on my local PC and I do streaming. When I connect to DSS to view my broadcasting looks very good. The problem is that when another friend of mine, who is also on the LAN and has access to DSS tries his own broadcasting (using VLC as a broadcaster) and then he or I try to view his broadcasting we view mine instead of his. No matter the name of the sdp file or the different session Id in the sdp file or the path that the sdp file is stored we always view the first broadcasting in row. Could u pls help because from that behaviour I understand that DSS works with only ONE broadcasting each time and it can not host more than one concurrent broadcasts. I thank u in advance for your help. I am looking forward to your answer if there is anybody that has faced the same problem in the past. Hi, have you tried the streaming-server-users mailing list for use for questions like this? We are a bunch of Darwin developers, most of whom don't deal with DSS or QuickTime. This email sent to site_archiver(a)lists.apple.com

1 0

Live Streaming DSS problem
by Stylianos Chatziperis 09 Jan '08

09 Jan '08

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com Organization: European Dynamics SA User-agent: Mozilla Thunderbird 1.0.6 (Windows/20050716) Hi all, Yours, schatzip -- ΠΟΰ΅±α _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I would like to issue a very important question since the behaviour that I observe when I try live streaming with DSS looks crazy to me... :-) I have installed DSS in a LAN and it works correct with streaming of stored content. I have also set up a broadcaster (VLC)on my local PC and I do streaming. When I connect to DSS to view my broadcasting looks very good. The problem is that when another friend of mine, who is also on the LAN and has access to DSS tries his own broadcasting (using VLC as a broadcaster) and then he or I try to view his broadcasting we view mine instead of his. No matter the name of the sdp file or the different session Id in the sdp file or the path that the sdp file is stored we always view the first broadcasting in row. Could u pls help because from that behaviour I understand that DSS works with only ONE broadcasting each time and it can not host more than one concurrent broadcasts. I thank u in advance for your help. I am looking forward to your answer if there is anybody that has faced the same problem in the past. This email sent to site_archiver(a)lists.apple.com

1 0

kextload serialization lock busy; sleeping ?
by Stephane Sudre 09 Jan '08

09 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com What can be the origin of the following records in the system logs? _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… "localhost com.somecompany.someproduct[PID]: kextload serialization lock busy; sleeping (xx retries left)" This email sent to site_archiver(a)lists.apple.com

1 0

Re: pthread_cond_wait() missing broadcast?
by Pierre Baillargeon 09 Jan '08

09 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On 9-Jan-08, at 7:56 , Jim Magee wrote: Thanks for explanation. -- Pierre Baillargeon _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… The unlock needs to be atomic with the marking of the thread waiting on the condition. Since that latter part happens in the kernel, so must the unlock. What you are missing is the logic that converts a mutex being marked busy in user memory, to one being indicated as locked via being assigned a Mach semaphore. That Mach semaphore is then passed into the kernel in the Mach semaphore_{timed}wait_signal() call. That system call atomically marks the thread waiting on the condition's semaphore and pre-posts the mutex's semaphore - effectively unlocking the mutex for future attempts to acquire it (the first of those susequent attempts will have to make a trip to the kernel to acquire it via a semaphore_wait()). The biggest thing I was missing is that the core of pthread_cond_wait is implemented in libc, not in the kernel (in bsd/kern/ pthread_sync.c). I was thus missing a big part of the picture, which explains my confusion. This email sent to site_archiver(a)lists.apple.com

1 0

Re: A Bug in pthread_cond_destroy?
by Kazuho Oku 09 Jan '08

09 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=rISoobAVT+IwuOajVNo1XblYc0ZoQu7GfRfSWeWoaPc=; b=IHYl6C13IOWzjZMMDPGpEkc3+CNWAdKOZ9TJuCbofAWRc713lueJ9uO+TGY9wNnZCexbo276Sh1CimIaQr6DHd0ncUQFwCR9TWlFCCWbVX4FbVD25H2Cax7JtMGaIdr6odBVpEk3eKiVIPkZU6RG2P6Dyb1T8/z7WGHpXVGk3zQ= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=sAX9z2HEIzoH3qIu7eqbsqNpXBwCKjSepk6TGc0HXMGz5TBtuWAOAtUKPrZZQs+JvRiX6dFwCfLqwA5EeJcBxGB1WjdcurKS8DqbtzQp6qfy49lshN+4x/oLRjNlKcGcTv+2qyPLPqrYykuyIRyQLnMzHvQ9zV+kNz4N9rRl5hM= 2008/1/9, Jim Magee <jmagee(a)apple.com>: > I think you are assuming behavior that isn't specified. Specifically, the > definition for pthread_cond_destroy() says: > The pthread_cond_destroy() function shall destroy the given condition > variable specified by cond; the object becomes, in effect, uninitialized. An > implementation may cause pthread_cond_destroy() to set the object referenced > by cond to an invalid value. A destroyed condition variable object can be > reinitialized using pthread_cond_init(); the results of otherwise > referencing the object after it has been destroyed are undefined. Thank you for your response, I did not know such a statement exists in pthread spec., maybe I was referring to an older document. That said, I googled and found a spec. that includes your quotes (http://www.opengroup.org/onlinepubs/007908775/xsh/pthread_cond_destroy.html) and if you are referrring to it, my understaning is that it states It shall be safe to destroy an initialized condition variable upon which no threads are currently blocked. which would be the case in my example. My understanding is that there is the case in the Mac OS version of pthread where a waiting thread get signalled and woke up and may destroy the cond before the signalling thread finishes modifying the pthread_cond_t variable. Or if there's any other version, would you please let me know? Thank you in advance. > So, trying to call pthread_cond_signal() from another thread after the > condition is destroyed (or even "during" it being destroyed) can result is > undefined behavior. You really need synchronization of your own to avoid > issues there. Using the same mutex would have been good enough. Since that > isn't in your model, you need something else (maybe a second condition > variable that indicates whether the first has been destroyed? - just > thinking out loud). > > --Jim > > > On Jan 9, 2008, at 2:25 AM, Kazuho Oku wrote: > > Hi, > > If this is not the right place to report bugs of libc, please tell me > the right mailing list. Thank you in advance, and sorry, if it is the > case. > > I think I have found a bug in pthread_cond_destroy of Libc (with Mac > OS X 10.4.10). > > Current implementation of pthread_cond_destroy does not check if > sigspending==0. Thus if pthread_cond_destroy is called from a thread > other than that called pthread_cond_signal, the destructor may return > 0 even if pthread_cond_signal is still in work (i.e. sigspending != > 0), leading to memory corruption or an infinite loop in > pthread_cond_signal. > > For example, in the following code, thread B would sometimes read from > and/or write to freed memory. > > Thread A: > pthread_cond_wait(cond, &mutex); > while ((err = pthread_cond_destroy(cond)) != 0) { > assert(err == EBUSY); > usleep(1); > } > free(cond); > > Thread B: > pthread_cond_signal(global_cond); > > Note that the problem only arises when thread B does not lock the same > mutex as thread A when calling thread A (this is not a requirement in > POSIX threads). > > Attached to this mail is a code that would reproduce this bug. It > sends and receives pthread_cond_signals and increment a counter. > It compiled without any compile time defines, the program prints the > counter incrementing infinitely. > But if compiled with -DCLEAR_MEM -DSIGNAL_WO_LOCK it would suddenly > stop due to memory corruption. > Or if compiled with -DAPPLE_TEST -DSIGNAL_WO_LOCK it would check the > value of sispending and print an assertion failure. > > To reproduce the bug with the attached code, a dual processor system > might be a requirement. > > -- > Kazuho Oku > <thtest.c> _______________________________________________ > Do not post admin requests to the list. They will be ignored. > Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) > Help/Unsubscribe/Update your Subscription: > http://lists.apple.com/mailman/options/darwin-dev/jmagee%40apple.com > > This email sent to jmagee(a)apple.com > PS. I think I have initially responded to Jim only and am re-posting the mail with slight modifications. -- Kazuho Oku _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: pthread_cond_wait() missing broadcast?
by Jim Magee 09 Jan '08

09 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com On Jan 7, 2008, at 2:49 PM, Pierre Baillargeon wrote: --Jim _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I've tried to follow the code of __pthread_cond_wait() in the Darwin source code, but I've been stumped by the fact that the mutex being passed in is not explicitly unlocked anywhere! The variable is checked to verify that it is indeed a mutex but is not otherwise used anywhere else in the code, making me wonder where and how it can possibly be released. I've gone down to wait_queue_assert_wait64_locked(). Is the unlock somehow implicit? The unlock needs to be atomic with the marking of the thread waiting on the condition. Since that latter part happens in the kernel, so must the unlock. What you are missing is the logic that converts a mutex being marked busy in user memory, to one being indicated as locked via being assigned a Mach semaphore. That Mach semaphore is then passed into the kernel in the Mach semaphore_{timed}wait_signal() call. That system call atomically marks the thread waiting on the condition's semaphore and pre-posts the mutex's semaphore - effectively unlocking the mutex for future attempts to acquire it (the first of those susequent attempts will have to make a trip to the kernel to acquire it via a semaphore_wait()). This email sent to site_archiver(a)lists.apple.com

1 0

Re: A Bug in pthread_cond_destroy?
by Jim Magee 09 Jan '08

09 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

A Bug in pthread_cond_destroy?
by Kazuho Oku 09 Jan '08

09 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; bh=KkHaY3X1vTLif5gtWo7TatukUkOJ5t2oLd/je9vPRRU=; b=gcsc2sDwUk6jtFQOTec3zhyItemRISL4Z02fhoPPrtjMEiKy3364qX/EeIo91H1Tsnbll8clezGdkn+oaUb0qibHdAUog0P+Kcmgk39Jd+y7DrSl5hYVCNkrx9ckMKxKoRgUTcQlKpZVwnAPYI7UbpJdV6zbkJmAXWBgzp5KaDc= Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=mjsJbXrTB03E8UT0Y9yQmKPoXVWqdoAztWou3Qq3SJ+GehgBgYK5zzorHMLQ9stklzUu9cRpJD5pjf63DpOds7SrsqDhu7e7mM4UrHya+gq649IaiTDjZm68Rnth62LUEAqqxrpGReu9PocMa7WKvfmDhTid+QU+MhzNJFHzSwA= Hi, If this is not the right place to report bugs of libc, please tell me the right mailing list. Thank you in advance, and sorry, if it is the case. I think I have found a bug in pthread_cond_destroy of Libc (with Mac OS X 10.4.10). Current implementation of pthread_cond_destroy does not check if sigspending==0. Thus if pthread_cond_destroy is called from a thread other than that called pthread_cond_signal, the destructor may return 0 even if pthread_cond_signal is still in work (i.e. sigspending != 0), leading to memory corruption or an infinite loop in pthread_cond_signal. For example, in the following code, thread B would sometimes read from and/or write to freed memory. Thread A: pthread_cond_wait(cond, &mutex); while ((err = pthread_cond_destroy(cond)) != 0) { assert(err == EBUSY); usleep(1); } free(cond); Thread B: pthread_cond_signal(global_cond); Note that the problem only arises when thread B does not lock the same mutex as thread A when calling thread A (this is not a requirement in POSIX threads). Attached to this mail is a code that would reproduce this bug. It sends and receives pthread_cond_signals and increment a counter. It compiled without any compile time defines, the program prints the counter incrementing infinitely. But if compiled with -DCLEAR_MEM -DSIGNAL_WO_LOCK it would suddenly stop due to memory corruption. Or if compiled with -DAPPLE_TEST -DSIGNAL_WO_LOCK it would check the value of sispending and print an assertion failure. To reproduce the bug with the attached code, a dual processor system might be a requirement. -- Kazuho Oku #include <assert.h> #include <mach/semaphore.h> #include <pthread.h> #include <stdint.h> #include <stdio.h> #include <string.h> #ifdef APPLE_TEST typedef int32_t pthread_lock_t; struct _pthread_mutex; struct _pthread_cond { long sig; /* Unique signature for this structure */ pthread_lock_t lock; /* Used for internal mutex on structure */ semaphore_t sem; /* Kernel semaphore */ struct _pthread_cond *next, *prev; /* List of condition variables using mutex */ struct _pthread_mutex *busy; /* mutex associated with variable */ uint32_t waiters:16, /* Number of threads waiting */ sigspending:16; /* Number of outstanding signals */ }; #endif static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; static pthread_cond_t cond_to_main = PTHREAD_COND_INITIALIZER; static pthread_cond_t *cond_to_thread = NULL; static void *my_thread(void *_1) { pthread_cond_t t[16]; int i; #if APPLE_TEST assert(sizeof(struct _pthread_cond) == sizeof(pthread_cond_t)); #endif memset(t, 0x55, sizeof(t)); for (i = 0; ; i = (i + 1) & 15) { assert(pthread_mutex_lock(&mutex) == 0); assert(pthread_cond_init(t + i, NULL) == 0); cond_to_thread = t + i; assert(pthread_cond_signal(&cond_to_main) == 0); assert(pthread_cond_wait(t + i, &mutex) == 0); assert(pthread_cond_destroy(t + i) == 0); #ifdef APPLE_TEST assert(((struct _pthread_cond*)(t + i))->sigspending == 0); #endif assert(pthread_mutex_unlock(&mutex) == 0); #ifdef CLEAR_MEM memset(t + i, 0x55, sizeof(t[i])); #endif } return NULL; } int main(int argc, char **argv) { pthread_t tid; int i; assert(pthread_mutex_lock(&mutex) == 0); assert(pthread_create(&tid, NULL, my_thread, NULL) == 0); for (i = 0; ; i++) { pthread_cond_t *cc; while (cond_to_thread == NULL) { assert(pthread_cond_wait(&cond_to_main, &mutex) == 0); } cc = cond_to_thread; cond_to_thread = NULL; #ifdef SIGNAL_WO_LOCK assert(pthread_mutex_unlock(&mutex) == 0); #endif printf("%d\n", i); assert(pthread_cond_signal(cc) == 0); #ifdef SIGNAL_WO_LOCK assert(pthread_mutex_lock(&mutex) == 0); #endif } return 0; } _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Re: fread/fwrite/fflush bug (Norm Green)
by Terry Lambert 09 Jan '08

09 Jan '08

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com -- Terry _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… On Jan 7, 2008, at 12:51 PM, David Hoerl wrote: Date: Mon, 7 Jan 2008 09:49:19 -0800 From: Norm Green <norm.green(a)gemstone.com> Subject: RE: fread/fwrite/fflush bug ... Someone pointed out the fpurge() function. Calling fpurge() on the reader before the 2nd read fixes the problem. This might be one fix for your code, but how about other code not yet ported to OSX that is looking for "proper" behavior (making an assumption here). Reading the man page for fpurge, it says: "For input streams this discards any input read from the underlying object but not yet obtained via getc(3)...". I interpret this to mean the following would fail (but not your example): - writer writes say 1000 bytes - reader is told to read one byte at location 0 (but fread gets a buffer full) - writer changes all the data - all 1000 bytes - reader reads location 1, and gets old data (which it already had in the buffer) - reader keeps reading a byte at a time. When the internal buffer is exhausted THEN it would start reading the actual file again, and get a buffer of correct data. Based on the man page, one could assume that the seek backwards should cause the data in the buffer to get flushed as if fpurge was called. Apparently fread looks to see if the requested bytes are located in its buffer in all cases, and if so, uses them. The presumption by the man page here is it's the same program doing the reading and writing, and therefore the buffers are shared because you are dealing with the same descriptor. For other descriptors/buffers, you are effectively assuming a distributed cache coherency for the data cached by the stdio subsystem, which is not a valid assumption, under almost any circumstances. There's actually a stdio implementation that was done in university research that uses mmap()'ed pages rather than local buffers, but it pretty much never went anywhere outside of research because of the performance penalties for anything but small files. This email sent to site_archiver(a)lists.apple.com

1 0