Mailing Lists: Apple Mailing Lists
Image of Mac OS face in stamp
 
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: kqueue/kevent project

On Monday, March 11, 2002, at 07:49 PM, Stefan Arentz wrote:
On Mon, Mar 11, 2002 at 05:06:55PM -0500, Jim Magee wrote:
I'm referring to multiple threads (pthreads) in an application waiting
on a single kqueue. The same problem arises, however, if each thread
has its own kqueue, but the same event source is added to multiple of
them. In either case, more than thread will be awakened if the source
fires.

Ok, and this is a problem because waking up multiple threads at the
same time is expensive and gives a high load on the system? Sorry
this is one of the areas where I miss the practical experience :)

Well, you hit the nail on the head in your previous posting. There is no good way to maintain consistency if multiple threads _try_ to handle events from the same source in parallel. Well, maybe if it is a datagram socket, they could. But otherwise, it gets real dicey. That was why I assumed EV_ONESHOT was the way to go for multi-threaded kqueue use. It allows the first thread time to "get the house in order" (i.e. read the data off the socket, etc...) before adding the source back in for other threads.The load on the system is important, but only slightly less so than allowing proper behavior.

However, the reason I originally suggested multiple threads waiting at the same time is for system performance concerns. We need to avoid the overhead of waking one thread and then that first thread immediately waking another thread and going back to sleep on the kqueue. This "worker thread approach" means you need to schedule twice, at a minimum, to handle an event (never mind having to keep an extra thread's resources ready [stack in memory, etc...]). For high volume services (web server, Darwin streaming server, file server), this extra overhead can be quite substantial - especially since it isn't required with the Darwin-proprietary waitevent() mechanism they use today. We could simply take their code that re-enables a source within a queue, and replace it with code that inserts an EV_ONESHOT source back into the queue. The performance delta between those two operation should be more reasonable to swallow.

But that's all the "one kqueue, multiple threads" issue (and also covers the issues surrounding fork).

What about the case of two or more threads with distinct kqueues, each containing the same event source. You would probably have to assume they don't even know about each other's existence?!? Right? So it seems like a "wake one thread from each kqueue" is the proper thing at the kernel level. If they stumble across each other at user-space, so be it. That's what they get for being so confused. At least that's the decision I would assume the kqueue guys arrived at.

However, the Mach equivalent does not work that way. It assures only one handler thread per event, regardless of how many queues the event source is a member of, sees the event. This, in turn, is actually relied upon by Core Foundation's CFRunLoop() model (and all the higher level event loops [Cocoa and Carbon] built on top of that). So, you can guess, it's a pretty fundamental assumption in the way we do things now. What I'm looking for are the places our expected semantics are the same, and where they are different, so I know the requirements on the common mechanisms that would sit below the knode level.

--Jim
_______________________________________________
darwin-development mailing list | email@hidden
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development
Do not post admin requests to the list. They will be ignored.
References: 
 >Re: kqueue/kevent project (From: Stefan Arentz <email@hidden>)



Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2007 Apple Inc. All rights reserved.