Re: long running task
Re: long running task
- Subject: Re: long running task
- From: Pierre Frisch <email@hidden>
- Date: Mon, 16 May 2005 07:06:38 -0700
Hi Mike,
Did you try you problems with 5.2.3? The model for holding on to
Objects has been changed the EOs have a reference to the EC and the
EC has a Weak Reference to the EO. This allows for much better memory
management. Any object that is fetch is not retained, only inserted,
modified and deleted objects are retained so you want to clear up you
EC regularly.
Pierre
On 15-May-05, at 6:36 AM, Mike Schrag wrote:
There's a fairly extensive discussion from about maybe a month or
two ago from when I ran into a similar problem, and it's most
likely an OutOfMemoryError that has caused all kinds of terrible
problems (they always do).
The #1 most likely candidate is that you have the default
UndoManager on your editing context. This will SCREW you. The
default undo manager has an UNLIMITED UNDO HISTORY. You want to
either remove the undo manager (which can cause some problems with
deleting EO's), or set it to something like a history level of 10
or something.
The #2 most likely candidate is simply that EOF really does not
appear to be ideal for large import processes. From everything I
could tell, even without an undo manager, your EditingContext WILL
hold onto data far longer than you want it to -- to the point of
running out of memory. They really should be WeakReferences inside
there, but they don't appear to be now. There are several
solutions to this that came up in the previous discussion:
1) don't use EOF to do your large import: pros: you don't have to
worry about it, cons: this completely sucks
2) periodically make new EditingContexts: pros: you get EOF, cons:
you have to commit in the middle of your import transaction and you
have to manage localInstanceOfObjects properly
I went the route of #2. Now keep in mind that if you're talking
about directories and files, the natural implementation of that is
recursive. If it's recursive and you want to rotate
EditingContexts, it's particularly tricky to get right. My first
impl of this was to have a wrapper object AROUND editing context
that allowed me to rotate the context underneath. The problem with
this is that you cannot just then pass around EO's into your
recursively called methods, because they need to rotate into the
new editing context as the editing context rotates. I ended up
keeping a Map of objects that would otherwise pass around, and
instead request them from the wrapper by key, so the wrapper can
localInstanceOfObject them for me. Ultimately this is just really
ugly, so my fix was to unroll the recursive method into a list (so
I have a queue of directories and files and the directories post
their children into the queue). What this means is that you can
rotate the editing context in a LOOP (albeit with some slightly
tricky synchronization) instead of a recursive method and the
handling of the EC makes a lot more sense. Emotionally prepare
yourself for some hoop-jumping.
Chuck, who knows FAR more than I do, advocated dropping below
EditingContext and just inserting directly. I just couldn't bring
myself to do it, but only because I felt dirty doing it, not
because of any technical reason.
You should look back through the archives for the full discussion
-- there was a lot of good information that flew around.
ms
On May 15, 2005, at 3:05 AM, Wolfram Stebel wrote:
Hi all,
i have a test data generator, that ran about 5-6 hours tonight,
creating
10000 mandants with a lot of dependent data, 40000 directories,
20000 files
on my xserve with mysql.
the application was running in deployment mode.
after some time (an hour or so, after creating about 2500
mandants) safari
showed:
No instance available
The app was still running and generating data for hours.
After trying to restart woapp in javamonitor the following:
MyApp cannot be started because port 2000 is still in use
The log is attached.
i still cant start an instance now.
i will restart the server because its an test environment.
But what has happened?
How to manage the problem in production environment?
Thanks for your support!
Wolfram
<webobjectslog.rtf>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
40mdimension.com
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
40spearway.com
This email sent to email@hidden
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden