Mailing Lists: Apple Mailing Lists
Image of Mac OS face in stamp
 
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: kqueue/kevent project

On Monday, March 11, 2002, at 06:04 PM, Stefan Arentz wrote:
On Mon, Mar 11, 2002 at 05:06:55PM -0500, Jim Magee wrote:
On Monday, March 11, 2002, at 04:45 PM, Stefan Arentz wrote:
On Mon, Mar 11, 2002 at 09:16:16AM -0500, Jim Magee wrote:
One "problem" the kqueues implementation presents: multiple threads
sharing a kqueue.

Are you refering to kernel threads or user threads?

I'm referring to multiple threads (pthreads) in an application waiting
on a single kqueue. The same problem arises, however, if each thread
has its own kqueue, but the same event source is added to multiple of
them. In either case, more than thread will be awakened if the source
fires.

The documentation does not say anything about multiple threads sharing a
kqueue. I think it was never intended to be used like that. Don't you
typically create multiple queues (multiple kqueues can exist in a single
process) with different event sources and then use worker threads to
serialize the incoming events?

There are many cases, in Mac OS X/Darwin today, where the same event source is part of multiple thread's queues. This is made OK by the fact that the Mach API does not only the "select" part of the operation, but also the "receive" part, atomically. In fact, it is quite typical to have a single queue (portset) with many threads waiting on it simultaneously. My view was that the same could be handled by having a single kqueue that all the threads wait on, but mark the events as EV_ONESHOT. When the event comes in, one thread is awakened, the event is removed from the kqueue, and the thread handles the event. It then goes out to user space, reads the data associated with the event (or handles the other event types, like EOF), and then adds the event back into the queue. If there is more data there, the next thread wakes up, etc...

This is much more efficient, typically, than having a single thread receive all the events and then dole the work out to other worker threads.

When multiple threads listen on the same queue then an out of order problem
can occur. Say that two threads are listening to events from the same
socket descriptor. It is possible that thread A fires and receives the
EV_READ event that is also marked EOF (socket closed) *before* thread B
is able to process the previous EV_READ.

That's why my assumption was that all multi-threaded kqueue event sources would have to be added as EV_ONESHOT sources. The first thread gets things back to a known state, transitions any user-level state that needs to reflect what just happened, and then adds the source back into the queue. I had just taken this as a "given," and my assumption for the main reason EV_ONESHOT was added. I have reason to assume that, other than intuition. I just did.

BTW, I'm reading http://stuff.sateh.com/Mach/kernel_principles.pdf now. Is
this document still relevant? It's almost 10 years old now :-/

It's not bad. But don't forget to read the Apple/Darwin documentation on the same subjects:

http://developer.apple.com/techpubs/macosx/Darwin/kernel.html

--Jim
_______________________________________________
darwin-development mailing list | email@hidden
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development
Do not post admin requests to the list. They will be ignored.
References: 
 >Re: kqueue/kevent project (From: Stefan Arentz <email@hidden>)



Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2007 Apple Inc. All rights reserved.