Mailing Lists: Apple Mailing Lists
Image of Mac OS face in stamp
 
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: kqueue/kevent project

On Mon, Mar 11, 2002 at 08:50:05PM -0500, Jim Magee wrote:
> On Monday, March 11, 2002, at 07:49 PM, Stefan Arentz wrote:
> > On Mon, Mar 11, 2002 at 05:06:55PM -0500, Jim Magee wrote:
> >> I'm referring to multiple threads (pthreads) in an application waiting
> >> on a single kqueue. The same problem arises, however, if each thread
> >> has its own kqueue, but the same event source is added to multiple of
> >> them. In either case, more than thread will be awakened if the source
> >> fires.
> >
> > Ok, and this is a problem because waking up multiple threads at the
> > same time is expensive and gives a high load on the system? Sorry
> > this is one of the areas where I miss the practical experience :)
>
> Well, you hit the nail on the head in your previous posting. There is
> no good way to maintain consistency if multiple threads _try_ to handle
> events from the same source in parallel. Well, maybe if it is a
> datagram socket, they could. But otherwise, it gets real dicey. That
> was why I assumed EV_ONESHOT was the way to go for multi-threaded kqueue
> use. It allows the first thread time to "get the house in order" (i.e.
> read the data off the socket, etc...) before adding the source back in
> for other threads.The load on the system is important, but only slightly
> less so than allowing proper behavior.

I understand, but this is a client design decision. The kqueue API
should know about threads and simply support all models (single process,
forked processes, pthreads). But it is up to the code in userland to
choose whether EV_ONESHOT is appropriate.

If CF needs the event notification to work like that than there
are probably better ways to solve that out of userland. We could add
an extra flag to kqueue like EV_AUTOREADD that can be used in
combination with EV_ONESHOT. This way save an extra syscall to add
the event to the kqueue again for the next thread.

> For high volume services
> (web server, Darwin streaming server, file server), this extra overhead
> can be quite substantial - especially since it isn't required with the
> Darwin-proprietary waitevent() mechanism they use today. We could
> simply take their code that re-enables a source within a queue, and
> replace it with code that inserts an EV_ONESHOT source back into the
> queue. The performance delta between those two operation should be more
> reasonable to swallow.

I've looked at the Streaming Server and it's strategy is pretty simple,
it's one thread ... It's basically a waitevent() loop that takes events
and dispatches them to tasks that run within the same thread.

> But that's all the "one kqueue, multiple threads" issue (and also covers
> the issues surrounding fork).
>
> What about the case of two or more threads with distinct kqueues, each
> containing the same event source. You would probably have to assume
> they don't even know about each other's existence?!? Right? So it seems
> like a "wake one thread from each kqueue" is the proper thing at the
> kernel level. If they stumble across each other at user-space, so be
> it. That's what they get for being so confused. At least that's the
> decision I would assume the kqueue guys arrived at.

Yeah the kernel can't clean up design mistakes made at a higher level. That
is the programmer's responsibility. The manpage should probably contain
hints about proper usage for getting the best performance out of the API.

> However, the Mach equivalent does not work that way. It assures only
> one handler thread per event, regardless of how many queues the event
> source is a member of, sees the event. This, in turn, is actually
> relied upon by Core Foundation's CFRunLoop() model (and all the higher
> level event loops [Cocoa and Carbon] built on top of that). So, you can
> guess, it's a pretty fundamental assumption in the way we do things
> now. What I'm looking for are the places our expected semantics are the
> same, and where they are different, so I know the requirements on the
> common mechanisms that would sit below the knode level.

Hmm. I'm wondering if it really makes sense to combine the Mach and BSD
APIs in one. I think I can come with a clean solution on which both
waitevent, select and kevent() can be efficiently build and run but I
do not yet have a picture in my mind that also includes CFRunLoop().

Is the source code for the CFRunLoop available in Darwin?

More reading to do :-) More ideas tomorrow.

Stefan
_______________________________________________
darwin-development mailing list | email@hidden
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development
Do not post admin requests to the list. They will be ignored.
References: 
 >Re: kqueue/kevent project (From: Stefan Arentz <email@hidden>)
 >Re: kqueue/kevent project (From: Jim Magee <email@hidden>)



Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2007 Apple Inc. All rights reserved.