Re: Report generation: How to avoid "java.lang.OutOfMemoryError: Java heap space"?
Re: Report generation: How to avoid "java.lang.OutOfMemoryError: Java heap space"?
- Subject: Re: Report generation: How to avoid "java.lang.OutOfMemoryError: Java heap space"?
- From: Simon McLean <email@hidden>
- Date: Thu, 27 Sep 2007 22:46:48 +0100
HI -
We are in a similar situation right now, trying to figure out the
best way of getting huge reports out of EOF whilst maintaining the
richness provided by Enterprise Objects.
Here's the first thing we figured: forget components. As you appear
to have found out they simply don't scale when you get into thousands
of rows.
At the moment we are playing with fetching just the pk's of the
objects required for the report (using raw rows), then cycling
through the array instantiating each object one by one (using a
temporary EC for each object - creating it to fetch the object, doing
it's stuff, then disposing of it) and appending a string of the
required attributes of each object to an XML document which is
ultimately returned by the method. We are then going to use a
reporting tool (Jasper, Pentaho, BIRT ?) to generate the reports
based on the XML document we have got from EOF.
Simon
On 27 Sep 2007, at 22:07, Fabian Peters wrote:
Hi all,
I'm trying to set up a simple reporting component, output of which
is to be used in a statistical analysis. It's basically a
WORepetition wrapped around about 40 WOStrings, supposed to render
a long list, CSV-formatted in this case. The WOStrings are bound to
various keypathes of "item" EO. Some of the keypathes touch on
relationships, so there are other objects getting fetched as needed
(about 7 per "item" EO on average).
What I did so far was to wrap the main repetition in a second
repetition that in turn is bound to a "batches" array, which simply
contains an index for the number of batches (e.g. "(1, 2, 3, 4, 5,
6)")
<batchingRepetition>
<mainRepetition>list contents</mainRepetition>
</batchingRepetition>
I then fetch only the objects for the current batch, using a fresh
EC for each batch (disposing the old one). I've used an
ERXBatchingDisplayGroup and then an
ERXEOControlUtilities.objectsInRange() for the batching, both with
the same results memory-wise. An example of the apps memory usage
is at the end of this mail. From what I see, the disposal of the
"old" EC (before the "lower range limit" log statement) does not
free any memory?
Increasing Java heap space is not really a solution, since this
will have to scale. Using SQL or JDBC also is not an option (or at
least one that I really want to avoid), since I'm depending on code
in the EOs.
Since I'm a bit lost right now, I hope someone can push me in the
right direction...
cheers,
Fabian
Memory usage ("registered objects" shows the EC's registered
objects count before disposal):
INFO 10,01 MB used/2,78 MB free com.myapp.WeightAnalysisPage:102 -
list batches array: (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
INFO 10,16 MB used/2,63 MB free com.myapp.WeightAnalysisPage:75 -
registered objects: 0
INFO 10,25 MB used/2,54 MB free com.myapp.WeightAnalysisPage:80 -
lower range limit is 0
INFO 10,31 MB used/2,48 MB free com.myapp.WeightAnalysisPage:82 -
upper range limit is 100
INFO 20,79 MB used/5,97 MB free com.myapp.WeightAnalysisPage:75 -
registered objects: 726
INFO 20,87 MB used/5,9 MB free com.myapp.WeightAnalysisPage:80 -
lower range limit is 100
INFO 20,94 MB used/5,83 MB free com.myapp.WeightAnalysisPage:82 -
upper range limit is 200
INFO 29,59 MB used/11,16 MB free com.myapp.WeightAnalysisPage:75 -
registered objects: 678
INFO 29,7 MB used/11,04 MB free com.myapp.WeightAnalysisPage:80 -
lower range limit is 200
INFO 29,76 MB used/10,99 MB free com.myapp.WeightAnalysisPage:82 -
upper range limit is 300
INFO 37,45 MB used/24,54 MB free com.myapp.WeightAnalysisPage:75 -
registered objects: 716
INFO 37,54 MB used/24,46 MB free com.myapp.WeightAnalysisPage:80 -
lower range limit is 300
INFO 37,62 MB used/24,38 MB free com.myapp.WeightAnalysisPage:82 -
upper range limit is 400
INFO 49,25 MB used/12,75 MB free com.myapp.WeightAnalysisPage:75 -
registered objects: 677
INFO 49,35 MB used/12,65 MB free com.myapp.WeightAnalysisPage:80 -
lower range limit is 400
INFO 49,43 MB used/12,57 MB free com.myapp.WeightAnalysisPage:82 -
upper range limit is 500
INFO 53,78 MB used/9,78 MB free com.myapp.WeightAnalysisPage:75 -
registered objects: 680
INFO 53,92 MB used/9,64 MB free com.myapp.WeightAnalysisPage:80 -
lower range limit is 500
INFO 54 MB used/9,56 MB free com.myapp.WeightAnalysisPage:82 -
upper range limit is 600
INFO 57,8 MB used/5,76 MB free com.myapp.WeightAnalysisPage:75 -
registered objects: 729
INFO 57,9 MB used/5,66 MB free com.myapp.WeightAnalysisPage:80 -
lower range limit is 600
INFO 57,91 MB used/5,65 MB free com.myapp.WeightAnalysisPage:82 -
upper range limit is 700
INFO 63,12 MB used/450,52 KB free com.myapp.WeightAnalysisPage:75 -
registered objects: 678
INFO 63,14 MB used/435,39 KB free com.myapp.WeightAnalysisPage:80 -
lower range limit is 700
INFO 63,21 MB used/364 KB free com.myapp.WeightAnalysisPage:82 -
upper range limit is 800
WARN 63,28 MB used/291,5 KB free NSLog :41 -
<er.extensions.ERXDirectActionRequestHandler>: Exception while
handling action named "pages" on action class
"null" :com.webobjects.foundation.NSForwardException
[java.lang.OutOfMemoryError] Java heap space
[2007-09-27 20:51:34 GMT] <WorkerThread0>
com.webobjects.foundation.NSForwardException for
java.lang.OutOfMemoryError: Java heap space
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
40mac.com
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden