Re: Lots of EOs slow down the performance
Re: Lots of EOs slow down the performance
- Subject: Re: Lots of EOs slow down the performance
- From: Andrew Lindesay <email@hidden>
- Date: Fri, 7 Nov 2008 10:55:28 +1300
Hello Yung;
Sorry I missed the start of your thread, but...
Do you have any trick or pattern to recycle EC? Call System.gc()?
Best not to use forced GC if you can help it because it has some side-
effects like blocking all the threads for a wee bit. See comments
below.
- create the CVS file on disk with an output stream that doesn't
keep the
whole thing in memory
Or maybe also helpful (to a limited degree) is to compress the
outbound data into a ByteArrayOutputStream from a GzipOutputStream in-
memory. This will compress the outbound data.
Memory: it appears that EC isn't suited for insert a lot EO into
database because the memory footstep of EC work like this: hold
I used EOF to insert large quantities of EO's in batch runs without
too much trouble, but as Guido has said, you need to perhaps use EOF
in a specific manner which is more suited to a bulk situation. I
would try this; load any pre-requisite reference material into GID's
and then loop through ~50 lots of your input file inside one fresh,
locked and EC. Do the save-changes there and then make a fresh EC on
the next loop. Doing 50 at a time will reduce the overall volume of
transactions which can help for some database products. On each loop,
use ec.faultForGlobalId(..) to turn the GID's into EO's inside the 50-
loop. Do you follow?
I know there are some fans of MySQL on this list and I have clients
using MySQL, but I have also had situations where MySQL has "had
memory problems" which went away when I migrated to another database
product. Perhaps get jconsole looking at your instance and keep an
eye on memory consumption as the process runs. Maybe run a smaller
batch, then use jconsole to force GC the instance and then you can see
if it has relinquished the memory after the job?
cheers.
___
Andrew Lindesay
www.lindesay.co.nz
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden