RE: Threaded / offline processor (EOF Memory Management)
RE: Threaded / offline processor (EOF Memory Management)
- Subject: RE: Threaded / offline processor (EOF Memory Management)
- From: <email@hidden>
- Date: Tue, 26 Apr 2005 09:29:33 +0200
- Thread-topic: Threaded / offline processor (EOF Memory Management)
Hi!
We are using an implementation of plan three. Roughly:
We have defined Wokflows : an EO with a State. Workflows are taken through a series of States by Processes which are defined with a starting State.
The WorkflowManager is a process which runs permanently and checks for Workflows to match the initial State of a Process. If found a JVM is started with the mission to run the appropriate Process. The Process runs as long as there is work to do then dies off.
Actually each Process is implemented as two threads: a producer and a consumer. The producer retrieves the EOs to process. State may not be the only criteria. The consumer is a chain of reusable worker classes. The WorkflowManager intantiates the producer of each process it is in charge of.
Users can initiate processing by creating Workflow instances from the UI. The WorkflowManager is controlled by direct actions called from crontab using curl. This may instruct it not to spawn certain CPU hungry processes at certain times of the day etc.
The advantages of this solutions are:
- Reduced memory footprint. We have many processes. Only those having actual work are running
- Independance of processes. E.g. one cannot take the other down. They may run with different 'nice' factors,...
- One business process may be split into sevral Processes with the end State of one being the start State of the next
- WorkflowManager may be configured to spawn multiple instances of bottleneck Processes
- Each Process has its own database connection, cache, cache policy, log file, ...
- WorkflowManager is a direct action based WO application. The processes are Java/EOF applications.
Pierre
-----Original Message-----
From: webobjects-dev-bounces+pierre.bernard=email@hidden
[mailto:webobjects-dev-bounces+pierre.bernard=email@hidden]On
Behalf Of David Aspinall
Sent: Monday, April 25, 2005 6:24 PM
To: WOdev List
Subject: Threaded / offline processor (EOF Memory Management)
I have the need for a Cron style offline processor to run arbitrary
jobs. Some of the jobs will be simple (invoke a database stored
procedure), others will be more complex (synchronize account
information from 3rd party financial institutions using a XML web
service), and some will be in the middle (generate daily/weekly/monthly
reports)
We want the end users to be able to schedule and configure the 'jobs'
themselves, and the jobs must process even if they miss their
'run-time' because of system maintenance or error. So the basic job
definition and schedule is database driven. On the processor side I
have taken several approaches trying to find a workable one:
Plan One - Thread Queues
- one thread check the database schedule and enqueues jobs in a thread
safe queue.
- two or more threads wait for jobs in the queue and process as needed.
- each job allocates its own EOEditingContext with a new
EOObjectCoordinator (so each job should have a separate EOF stack)
- each job locks the EC, processes, then commits/rollsback/aborts,
then invalidates and 'disposes' of itself.
Plan two - Direct actions
- job requests come in as WS/direct action requests
- each request follows the standard action request logic (with
multithreaded request handling OFF)
- uses the Direct Action session().defaultEditingContext()
- because the request had no sessionId, it would generate a new
session for each request.
Plan Three - FULL JVM subprocesses
- each job runs as a separate process (not thread)
- OMG i feel sooooo heavy.
Plan Nine - drop EOF and go JDBC/JavaBeans/OpenSymphony Quartz
- ouch, there goes my business logic
I have built and tested Plans One and Two. Plan One goes like all get
out, chewing up through jobs non-stop. Unfortunately it will only go
about 3 hours before running out of memory and failing. And it fails
badly depending where the Out Of Memory Exception takes place. If it
is caught and discarded (especially in the MSSQL server JDBC adaptor) I
can get database deadlock, thread deadlock .... bad bad!
Plan 2 ran consistently with a steady 20 - 30Mb JVM footprint for over
3 hours. So I bumped up the request cycle (using cron and curl), and
then it started to go a little crazy also. I think it lived close to
10 hours (I had to go home and get married the next day, so I didn't
see the crash)
Both processes were doing the same jobs, a simple HTML template using
EOF to fill with data, then FOP into PDF. Both use the same data from
the database over and over and over ...
Does anyone have any experience in this area? Any suggestions for
getting better memory management out of EOF? I need this process to
last consistently and not fail badly.
Looking for help
David
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
**********************************************************************
This email and any files transmitted with it are intended solely for
the use of the individual or entity to whom they are addressed.
If you have received this email in error please notify the sender
of this message. (email@hidden)
This email message has been checked for the presence of computer
viruses; however this protection does not ensure this message is
virus free.
Banque centrale du Luxembourg; Tel ++352-4774-1; http://www.bcl.lu
**********************************************************************
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden