Re: User-space threads and NSAutoreleasePool
Re: User-space threads and NSAutoreleasePool
- Subject: Re: User-space threads and NSAutoreleasePool
- From: BJ Homer <email@hidden>
- Date: Thu, 18 Mar 2010 00:41:51 -0600
On Wed, Mar 17, 2010 at 11:47 PM, Greg Guerin <email@hidden> wrote:
>
> Two main questions come to mind:
>
> Q1. What are you trying to accomplish?
> Q2. Why do you think this would work?
>
> More on Q1: You said you need user-space threads, but you gave no reason
> or rationale. If it's because you need 500 threads, then why do you need
> 500 threads? Exactly what are you trying to accomplish?
>
Q1: What am I trying to accomplish?
In this particular case, I'm working on an application that sends very large
HTTP PUT requests over an HTTP connection in a pipelined fashion. For
performance reasons, the server prefers to delay acknowledgement of these
files until it can process them in large groups. (Before it can send an ack,
it must update a database. Doing 400-500 single updates is far slower than
doing one transaction updating 400-500 records.) Hence, we pipeline the HTTP
requests, starting transfer of the second before the first one is finished.
There are a large number of servers that don't handle pipelining, but we'll
only we talking to one particular server, and we know it does.
NSURLConnection does not (according to various mailing list messages)
implement pipelining, allegedly due to the lack of server support. There's
some suggestion that CFHTTPStream does support pipelining, but there's
little to no documentation about it, and I don't know if it will handle 500
at once.
If you can use user-space threads, this becomes simple; you send the first
file, and then when you're waiting for the ACK, you swap out and let another
file start sending. When the ack is received, the first user-thread is
rescheduled. We've developed an extremely high-performance cross-platform
library that handles all the scheduling directly. Then we've built another
library around that that handles all the particulars of our server protocol.
At the moment, I'm converting our OS X client to use this library. That's my
immediate motivation; if I can use an existing library, I'll save a large
amount of time, get a big performance boost, and have less code to test. To
a large extent, it is already working.
> More on Q2: The ucontext(3) functions appear to me to be more intended for
> signal-handling, specifically for alt-signal-stack handling. The nature of
> signals is that they eventually return (nested), they don't work in a
> round-robin or any other scheduling. That's not a question, just prep. The
> question is this: Are you sure the Objective-C runtime is compatible with
> ucontext user-threads? There are lots of things you can't do in
> signal-handlers, not even handlers written in plain C (ref: man sigaction,
> the list of safe functions). If you can't call malloc() in a signal-handler
> (and I'm pretty sure you can't), what do you expect to happen with your
> Objective-C user-threads, since object allocation is tied to malloc()?
>
Q2: Why do I think this would work? Am I sure the Obj-C runtime is
compatible with ucontext user-threads?
No. There is, in general, relatively little information on use of ucontext
user-threads on OS X. Mostly, it exists in man pages. So far the only issue
I've run into is the autorelease pool issue mentioned previously, which
isn't (I suspect) actually a runtime issue at all. There may be other
issues, but that's part of the reason I'm asking on the list. I can work
around the autorelease pool issue if I have to (by limiting my use of
Objective-C objects and autorelease pools in such a way that I can guarantee
correctness), but if there are other known issues, I'd love to hear about
them.
It's true that ucontext stuff is often used for signal handling, but that's
not the only use case. My code does not relate to signal handling at all. As
mentioned above, all this ucontext stuff happening in a cross-platform
library. It's already running on Linux, where I suspect the
malloc-in-signal-handlers restrictions equally apply. I've also had it
running (to some extent) as well. In neither Linux nor my initial attempts
on OS X have I seen anything to indicate that object allocation was not
working. In fact, I have a fair amout of evidence that it is likely working
just fine. If object allocation was failing or otherwise unstable, my code
would be crashing all over the place, not just having autorelease pool
issues.
-BJ
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden