Re: EOF / multithreading app design question
Re: EOF / multithreading app design question
- Subject: Re: EOF / multithreading app design question
- From: Kieran Kelleher <email@hidden>
- Date: Tue, 01 Dec 2009 23:31:10 -0500
My 2 cents:
Use java.util.concurrent classess ......
Create a task (processing) class that implement Callable and return a
result. Create a ThreadPoolExecutor of a size that keeps the CPU
pegged at 100% during typical processing (for 8-core, try 8 or
multiples of that).
In your Callable tasks, create a new EC for each task ....... a task
does not necessarily have to have business logic in it ..... the task
can simply call a business logic method in an EO.
As you receive tasks in the queue, submit them to the ThreadPool which
returns a Future that can be used to get the result when computation
is complete. An appropriate thread pool would have a fixed size and a
FIFO input Queue....... sth like this:
new ThreadPoolExecutor(8, 8, 0L, TimeUnit.SECONDS, new
LinkedBlockingQueue())
Depending on the OSC intensity, you may need a different OSC per
thread. Easy way it to create one, and dispose of it at end of the
task, but you would need ERXJDBCAdaptor to ensure you don't have an
ever-growing set of idle connections to your database. A better way
might be to have some other mechanism that lazily creates one OSC per
thread using a ThreadLocal, ERXThreadStorage or sth like that.
Finally, don't pass EOs in and out of the task classes, use
EOGlobalIDs and hydrate the seed EO (or EOs) using ec.faultForGlobalID
in your task classes.
HTH, Kieran
On Dec 1, 2009, at 9:48 PM, Andrew Lindesay wrote:
Hi Ken;
I do this sort of thing; feeding in workload via messaging queues
and process it.
If you had a number of instances consuming from the queue rather
than focussing on threading within one instance, I wonder if you
really need to do anything special around EOF concurrency within a
single JVM? In my applications, the batch processing occurs on a
different EOObjectStoreCoordinator to the regular user-processing
and so there is no contention between the two. This seems to work
well for me.
...a single context for all this stuff takes up a few hundred
megabytes already.
Can you not break down the workload (presumably by message) and
create a new EC each time you have to do some work?
cheers.
___
Andrew Lindesay
www.lindesay.co.nz
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden