September 2004 - Darwin-dev

AFS / CodaFS on Panther?
by jmzorko＠mac.com 16 Sep '04

16 Sep '04

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Hello, all ... Regards, John Falling You - exploring the beauty of voice and sound New album, "Touch", available now http://www.magnatune.com/artists/falling_you _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… I'd like to experiment with filesystems of a more distributed nature i.e. more than just NFS-type mounts, more like systems that run daemons on clients that support callbacks, distributed locking mechanisms, etc. Since i'm a MacNut, i'd like to do all of this on my Mac(s). Am I tripping alone, or are there others who have done these things? This email sent to site_archiver(a)lists.apple.com

1 0

Re: Using MIG RPC between 2 threads inside a process
by Godfrey van der Linden 16 Sep '04

16 Sep '04

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com Oops, sorry I forgot to signal the condition in enqueue should read Godfrey On Sep 16, , at 11:50, Stéphane Letz wrote: @interface fr_grame_letz_Queue { pthread_mutex_t queueLock; pthread_condition_t queueSleeper; queue_head_t queueHead; } void enqueue: (queue_entry_t) record; queue_entry_t dequeueAndBlock: (BOOL) block; @end @implementation fr_grame_letz_Queue queue_entry_t dequeueAndBlock: (BOOL) block { queue_entry_t ret = 0; pthread_mutex_lock(&queueLock) while (queue_empty(&queueHead)) { if (!block) goto abortDequeue; pthread_cond_wait(&queueSleeper, &queueLock); } ret = dequeue(&queueHead); abortDequeue: pthread_mutex_unlock(&queueLock); return 0; } @end Thanks Stephane Letz _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… void enqueue: (queue_entry_t) record { // Note no allocation to hold us up - it is assumed that the client has done the allocation itself pthread_mutex_lock(&queueLock); enqueue(&queueHead, record); pthread_cond_signal(&queueSleeper); pthread_mutex_unlock(&queueLock); } Also a small optimisation would be to track if you have any sleepers behind the dequeue and only do a signal if sleepers > 0. Basically the solution is to use an internal queue protected by the mutex and when there is no work to do you use a condition to block the thread. (This is exactly what we do in the kernel a whole lot ;-) It is most common to write the queue mechanism in a new class, if you are doing OO. These queues are extraordinarily easy to write. This general model is not only efficient but if you do it write the locks on the queues are the only locks you need in your entire design. I recommend that you don't allocate any storage in the queue insert structure, rather your queue is only appending the argument data structure on the end of a linked list. I usually use the <kern/queue.h> macros for this, you can copy and paste the macros into your new class from /System/Library/Frameworks/Kernel.framework/Headers/kern/queue.h Below is a rough cut at the implementation of the ObjC class for queues based upon pthread locking and <kern/queue.h>. void enqueue: (queue_entry_t) record { // Note no allocation to hold us up - it is assumed that the client has done the allocation itself pthread_mutex_lock(&queueLock); enqueue(&queueHead, record); pthread_mutex_unlock(&queueLock); } I don't understand here : who is going to "signal" to condition variable that there is something to read in the queue? Is it part of the " enqueue(&queueHead, record);" operation? The above code is the guts of a queuer but you still have to write the intialisation and teardown code. This basic model is very, very common in the kernel and is remarkably efficient. It works best when you combine the queue with the concept of 'ownership'. That is your 'client' thread allocates the 'record', initialises it and enqueues it. It agrees that it is not allowed to deal with the record until it gets returned to it on it's own queue, the only function that is allowed to operate on it is an 'abort' function if that is meaninngful. Another problem for me is that the thread which has things to enqueue is a time-constraint thread. I guess locking the pthread_mutex can cause priority inversion problems that I can not afford. This is why using this inter-thread *asynchonous* RPC mechanism, i was thinking that I could also avoid the priority inversion problem. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Using MIG RPC between 2 threads inside a process
by Stéphane Letz 16 Sep '04

16 Sep '04

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com @interface fr_grame_letz_Queue { pthread_mutex_t queueLock; pthread_condition_t queueSleeper; queue_head_t queueHead; } void enqueue: (queue_entry_t) record; queue_entry_t dequeueAndBlock: (BOOL) block; @end @implementation fr_grame_letz_Queue queue_entry_t dequeueAndBlock: (BOOL) block { queue_entry_t ret = 0; pthread_mutex_lock(&queueLock) while (queue_empty(&queueHead)) { if (!block) goto abortDequeue; pthread_cond_wait(&queueSleeper, &queueLock); } ret = dequeue(&queueHead); abortDequeue: pthread_mutex_unlock(&queueLock); return 0; } @end Thanks Stephane Letz _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Basically the solution is to use an internal queue protected by the mutex and when there is no work to do you use a condition to block the thread. (This is exactly what we do in the kernel a whole lot ;-) It is most common to write the queue mechanism in a new class, if you are doing OO. These queues are extraordinarily easy to write. This general model is not only efficient but if you do it write the locks on the queues are the only locks you need in your entire design. I recommend that you don't allocate any storage in the queue insert structure, rather your queue is only appending the argument data structure on the end of a linked list. I usually use the <kern/queue.h> macros for this, you can copy and paste the macros into your new class from /System/Library/Frameworks/Kernel.framework/Headers/kern/queue.h Below is a rough cut at the implementation of the ObjC class for queues based upon pthread locking and <kern/queue.h>. void enqueue: (queue_entry_t) record { // Note no allocation to hold us up - it is assumed that the client has done the allocation itself pthread_mutex_lock(&queueLock); enqueue(&queueHead, record); pthread_mutex_unlock(&queueLock); } I don't understand here : who is going to "signal" to condition variable that there is something to read in the queue? Is it part of the " enqueue(&queueHead, record);" operation? The above code is the guts of a queuer but you still have to write the intialisation and teardown code. This basic model is very, very common in the kernel and is remarkably efficient. It works best when you combine the queue with the concept of 'ownership'. That is your 'client' thread allocates the 'record', initialises it and enqueues it. It agrees that it is not allowed to deal with the record until it gets returned to it on it's own queue, the only function that is allowed to operate on it is an 'abort' function if that is meaninngful. Another problem for me is that the thread which has things to enqueue is a time-constraint thread. I guess locking the pthread_mutex can cause priority inversion problems that I can not afford. This is why using this inter-thread *asynchonous* RPC mechanism, i was thinking that I could also avoid the priority inversion problem. This email sent to site_archiver(a)lists.apple.com

1 0

Re: Using MIG RPC between 2 threads inside a process
by Godfrey van der Linden 16 Sep '04

16 Sep '04

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com @interface fr_grame_letz_Queue { pthread_mutex_t queueLock; pthread_condition_t queueSleeper; queue_head_t queueHead; } void enqueue: (queue_entry_t) record; queue_entry_t dequeueAndBlock: (BOOL) block; @end @implementation fr_grame_letz_Queue queue_entry_t dequeueAndBlock: (BOOL) block { queue_entry_t ret = 0; pthread_mutex_lock(&queueLock) while (queue_empty(&queueHead)) { if (!block) goto abortDequeue; pthread_cond_wait(&queueSleeper, &queueLock); } ret = dequeue(&queueHead); abortDequeue: pthread_mutex_unlock(&queueLock); return 0; } @end HTH Godfrey _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Basically the solution is to use an internal queue protected by the mutex and when there is no work to do you use a condition to block the thread. (This is exactly what we do in the kernel a whole lot ;-) It is most common to write the queue mechanism in a new class, if you are doing OO. These queues are extraordinarily easy to write. This general model is not only efficient but if you do it write the locks on the queues are the only locks you need in your entire design. I recommend that you don't allocate any storage in the queue insert structure, rather your queue is only appending the argument data structure on the end of a linked list. I usually use the <kern/queue.h> macros for this, you can copy and paste the macros into your new class from /System/Library/Frameworks/Kernel.framework/Headers/kern/queue.h Below is a rough cut at the implementation of the ObjC class for queues based upon pthread locking and <kern/queue.h>. void enqueue: (queue_entry_t) record { // Note no allocation to hold us up - it is assumed that the client has done the allocation itself pthread_mutex_lock(&queueLock); enqueue(&queueHead, record); pthread_mutex_unlock(&queueLock); } The above code is the guts of a queuer but you still have to write the intialisation and teardown code. This basic model is very, very common in the kernel and is remarkably efficient. It works best when you combine the queue with the concept of 'ownership'. That is your 'client' thread allocates the 'record', initialises it and enqueues it. It agrees that it is not allowed to deal with the record until it gets returned to it on it's own queue, the only function that is allowed to operate on it is an 'abort' function if that is meaninngful. This email sent to site_archiver(a)lists.apple.com

1 0

Marie Jung is out of the office.
by marie.jung＠kodak.com 16 Sep '04

16 Sep '04

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com I will be out of the office starting 23/04/2004 and will not return until 03/10/2004. I will respond to your message when I return. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com

1 0

Mach RPC and SecToken
by Ryan McGann 16 Sep '04

16 Sep '04

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Thanks, Ryan rmcgann(a)mac.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Sorry if this has been asked before, but the new list archives don't seem to be searchable yet... Does anybody out there know what SecToken in MIG definition files are used for, and where I can find out more about them? I have some MIG definitions as part of some sample code that use SecToken and AuditToken parameters, but then the source does nothing with them (except the MIG generated source). I'm curious as to how they would be used in "the real world". There are three kinds of people in the world: Those that can count, and those that can't. This email sent to site_archiver(a)lists.apple.com

1 0

Using MIG RPC between 2 threads inside a process
by Dario 16 Sep '04

16 Sep '04

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com #include <pthread.h> pthread_mutex_t lock; pthread_cond_t cond; pthread_cond_init( &cond, NULL ); pthread_mutex_init( &lock, NULL ); server: create client thread pthread_mutex_lock( &lock ); while(true) { pthread_cond_wait( &cond, &lock ); do something what you need } client: while( true ) { pthread_mutex_lock( &lock ); Put some data in a global varibale or so. pthread_cond_signal( &cond ); } OK... Thanks _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Moreover how can I tranfers messages between the 2 involved threads using condition variables? I though that it would need to use an intermediate queue of fifo? Using the Mac IPC allows directly to do that... Do it in the process' memory? You have a pthread mutex, it can protect a private queue in addition to the condition variable. Do you mean using "pthread_mutex" and "pthread_cond_signal/wait" API? How can this be done? Or can be resolved by using a queue structure with synchronized access and verification (still with mutexes), I have an example if you need it. with this code as you can see you can send only one job and the client will block until the end of the previous on the server thread In fact, the condition variable is only interesting when the queue runs dry. This is always more efficient than trying to queue the memory in the kernel. Besides, you are limited to a queue depth of 16 (default 5) when you use Mach message queues. Not really much of a queue to store things in, when you think about it. Stephane This email sent to site_archiver(a)lists.apple.com

1 0

Re: Using MIG RPC between 2 threads inside a process
by Dario 16 Sep '04

16 Sep '04

site_archiver(a)lists.apple.com Delivered-To: darwin-dev(a)lists.apple.com Il giorno 16/set/04, alle 09:56, Stéphane Letz ha scritto: #include <pthread.h> pthread_mutex_t lock; pthread_cond_t cond; pthread_cond_init( &cond, NULL ); pthread_mutex_init( &lock, NULL ); server: create client thread pthread_mutex_lock( &lock ); while(true) { pthread_cond_wait( &cond, &lock ); do something what you need } client: while( true ) { pthread_mutex_lock( &lock ); Put some data in a global varibale or so. pthread_cond_signal( &cond ); } OK... Thanks Stephane This email sent to yojimbo(a)katamail.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Moreover how can I tranfers messages between the 2 involved threads using condition variables? I though that it would need to use an intermediate queue of fifo? Using the Mac IPC allows directly to do that... Do it in the process' memory? You have a pthread mutex, it can protect a private queue in addition to the condition variable. Do you mean using "pthread_mutex" and "pthread_cond_signal/wait" API? How can this be done? or can be resolved by using a queue structure with synchronized access and verification, I have an example if you need it. with this code as you can see you can send only one job and the client will block until the end of the previous In fact, the condition variable is only interesting when the queue runs dry. This is always more efficient than trying to queue the memory in the kernel. Besides, you are limited to a queue depth of 16 (default 5) when you use Mach message queues. Not really much of a queue to store things in, when you think about it. _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/ yojimbo%40katamail.com This email sent to site_archiver(a)lists.apple.com

1 0

Re: Using MIG RPC between 2 threads inside a process
by Stéphane Letz 16 Sep '04

16 Sep '04

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com OK... Thanks Stephane _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… Moreover how can I tranfers messages between the 2 involved threads using condition variables? I though that it would need to use an intermediate queue of fifo? Using the Mac IPC allows directly to do that... Do it in the process' memory? You have a pthread mutex, it can protect a private queue in addition to the condition variable. Do you mean using "pthread_mutex" and "pthread_cond_signal/wait" API? How can this be done? In fact, the condition variable is only interesting when the queue runs dry. This is always more efficient than trying to queue the memory in the kernel. Besides, you are limited to a queue depth of 16 (default 5) when you use Mach message queues. Not really much of a queue to store things in, when you think about it. This email sent to site_archiver(a)lists.apple.com

1 0

Re: RAID-1 possible for multi-filesystem disk?
by Garth Cummings 15 Sep '04

15 Sep '04

site_archiver(a)lists.apple.com Delivered-To: Darwin-dev(a)lists.apple.com _______________________________________________ Do not post admin requests to the list. They will be ignored. Darwin-dev mailing list (Darwin-dev(a)lists.apple.com) Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/darwin-dev/site_archiver%40lists.app… This email sent to site_archiver(a)lists.apple.com smime.p7s

1 0