Re: long running task
Re: long running task
- Subject: Re: long running task
- From: Mike Schrag <email@hidden>
- Date: Sun, 15 May 2005 09:36:55 -0400
There's a fairly extensive discussion from about maybe a month or two
ago from when I ran into a similar problem, and it's most likely an
OutOfMemoryError that has caused all kinds of terrible problems (they
always do).
The #1 most likely candidate is that you have the default UndoManager
on your editing context. This will SCREW you. The default undo
manager has an UNLIMITED UNDO HISTORY. You want to either remove the
undo manager (which can cause some problems with deleting EO's), or
set it to something like a history level of 10 or something.
The #2 most likely candidate is simply that EOF really does not
appear to be ideal for large import processes. From everything I
could tell, even without an undo manager, your EditingContext WILL
hold onto data far longer than you want it to -- to the point of
running out of memory. They really should be WeakReferences inside
there, but they don't appear to be now. There are several solutions
to this that came up in the previous discussion:
1) don't use EOF to do your large import: pros: you don't have to
worry about it, cons: this completely sucks
2) periodically make new EditingContexts: pros: you get EOF, cons:
you have to commit in the middle of your import transaction and you
have to manage localInstanceOfObjects properly
I went the route of #2. Now keep in mind that if you're talking
about directories and files, the natural implementation of that is
recursive. If it's recursive and you want to rotate EditingContexts,
it's particularly tricky to get right. My first impl of this was to
have a wrapper object AROUND editing context that allowed me to
rotate the context underneath. The problem with this is that you
cannot just then pass around EO's into your recursively called
methods, because they need to rotate into the new editing context as
the editing context rotates. I ended up keeping a Map of objects
that would otherwise pass around, and instead request them from the
wrapper by key, so the wrapper can localInstanceOfObject them for
me. Ultimately this is just really ugly, so my fix was to unroll the
recursive method into a list (so I have a queue of directories and
files and the directories post their children into the queue). What
this means is that you can rotate the editing context in a LOOP
(albeit with some slightly tricky synchronization) instead of a
recursive method and the handling of the EC makes a lot more sense.
Emotionally prepare yourself for some hoop-jumping.
Chuck, who knows FAR more than I do, advocated dropping below
EditingContext and just inserting directly. I just couldn't bring
myself to do it, but only because I felt dirty doing it, not because
of any technical reason.
You should look back through the archives for the full discussion --
there was a lot of good information that flew around.
ms
On May 15, 2005, at 3:05 AM, Wolfram Stebel wrote:
Hi all,
i have a test data generator, that ran about 5-6 hours tonight,
creating
10000 mandants with a lot of dependent data, 40000 directories,
20000 files
on my xserve with mysql.
the application was running in deployment mode.
after some time (an hour or so, after creating about 2500 mandants)
safari
showed:
No instance available
The app was still running and generating data for hours.
After trying to restart woapp in javamonitor the following:
MyApp cannot be started because port 2000 is still in use
The log is attached.
i still cant start an instance now.
i will restart the server because its an test environment.
But what has happened?
How to manage the problem in production environment?
Thanks for your support!
Wolfram
<webobjectslog.rtf>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
40mdimension.com
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden