If a procedural APi is built on top of XgridFoundation, the
implementation of that APi will still need to do the "fragile"
task of managing an NSRunLoop by hand. So it seems like this
approach wouldn't result in anything more robust that what we
already have (in xgf for example).
But obviously, a procedural API that is built on top of the BEEP
layer would probably need to come from within Apple (unless
someone wants to reverse engineer the BEEP layer). That would
mean the benefits of points i), ii) and iv) are lost.
As you note, we don't have a public BEEP API, or a C API. However,
if we did either of those, we'd almost certainly use CFNetwork,
which means using CFRunLoop -- which probably isn't any better for
you than NSRunLoop. :-)
It sounds like the "hard" problem is mapping a call-back API onto a
RunLoop API. While tricky, hopefully it would be easier to do that
in C than its been for you to do in Python.
I wonder -- do any of the DRMAA folks have experience with that
problem?
-- Ernie P.
Some more random thoughts on all this:
* I am all for additional layers on top of XgridFoundation if needed
(I am working on one too!), and DRMAA would be best addressed this
way, which could be done now. I don't think it is reasonable to
expect Apple to support 2 official Xgrid APIs in Leopard, nor can
they just drop the whole current APIs, so we have to live with that
one for the next few years!
* Adding layers can make the code less robust, of course, but I don't
see why NSRunLoop in particular makes things more fragile (I may very
well be missing the details). There has to be communications between
the client and the controller, so at some levels, things have to be
asynchronous.
* Adding layers can hurt performance, also depending how well
asynchronous calls are handled. But I don't think performance is an
issue on this kind of problems, the events to deal with are
relatively rare (unless you have hundreds of job results to deal with
every second).
* It is best to wrap XgridFoundation than the CLI; wrapping the CLI
creates issues of its own, as you have to deal with a separate process
Now, how to map a call-back API onto a RunLoop API? This is a good
question...
Before somebody that knows what he is talking about answers, I will
try to come up with something. The issue here is that the DRMAA
specifies functions that have to return immediately something
(drmaa_run_job(), drmaa_run_bulk_jobs()) or block until something
happens to some job(s) (drmaa_wait and synchronize - not sure about
the difference between these 2). In fact, the "run_job" functions
that returns "immediately" something have to return a job id; if this
is going to be the job id from Xgrid, it will have to wait for the
XGActionMonitor to return the job id, which means the function has in
fact to block until the id is back. So, in all cases, the drmaa
function has to block and wait for a asynchronous callback from
XgridFoundation, that will require to start an NSRunLoop.
So maybe, what could be done is to have the XgridFoundation stuff
running in a unique separate thread with its own NSRunLoop. This
thread would be started and stopped by the DRMAA "init" call. Then
the calls to the DRMAA wrappers would use a lock, call some function
on the XgridFoundation thread, and then block on the lock until it is
released by the XgridFoundation thread. The XgridFoundation thread
would be shared by all the threads calling DRMAA, so the
XgridFoundation thread would have to keep track of the different
locks for the different queries. This all means there has to be some
communications between threads, but this is probably unavoidable is
the DRMAA implementation is to be thread-safe (as recommanded).
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Xgrid-users mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/xgrid-users/email@hidden
