Re: Memory Management
Re: Memory Management
- Subject: Re: Memory Management
- From: Chuck Hill <email@hidden>
- Date: Mon, 2 Mar 2009 11:58:47 -0800
On Mar 1, 2009, at 7:14 PM, Andrew Lindesay wrote:
Hello Jeff;
I posted on this as well? Did you get this post and have you tried
it?
Load the list of EO's into memory to be processed as raw-rows with
the PK in the raw rows. Break that lists up into batches of, for
example, 100 items. For each bunch of 100, create a new EC, lock it
and then fetch those EO's into memory and use pre-fetching to
optimise the database activity. Undertake your algorithm on the
EO's and then unlock the EC and move onto the next block of 100.
A big part of slowing-down with saturated object stores is that as
there are more and more EO's in memory it takes longer and longer
for fetches to be processed because as the system ensures
uniqueness, it will have to keep checking to see if it already has
each new row of data in memory already.
That does not sound right to me. It is a hashed lookup so it should
be O(1). Unless the hashing function is bad and there are a lot of
collisions, the lookup time should not vary greatly with the number of
snapshots on hand.
Chuck
Then I think there must be a LOT of references that build up. When
I changed my code to reset the ec periodically (after each pool
processed) it made a HUGE difference. Before that, my development
machine would eventually come to a grinding halt. After the change
it ran through everything with no noticeable slowdown. e.g.
...
Would unlock and dispose and creating a new EC in each loop be
better than ec.reset()?
___
Andrew Lindesay
www.lindesay.co.nz
--
Chuck Hill Senior Consultant / VP Development
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems.
http://www.global-village.net/products/practical_webobjects
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden