Re: Out-of-memory with large dataset
Re: Out-of-memory with large dataset
- Subject: Re: Out-of-memory with large dataset
- From: Jan Willem Luiten <email@hidden>
- Date: Wed, 11 May 2005 22:58:08 +0200
Hi again,
I was possibly caught off-guard by the out-of-memory exception thrown
by eomodeler when using PostgreSQL. I was used to getting the dialog
"fetched 1000 rows" when using Openbase. I'm now looking into
ERXBatchingDisplayGroup to solve my little academic problem.
Thanks all for responding so quickly.
Kind regards,
Jan Willem.
On May 11, 2005, at 10:44 PM, Ken Anderson wrote:
Jan,
His statement is definitely true. Trust us, you really don't want
to put 600,000 rows into a display group. Can you possibly share
with the list what you're trying to do so we could possibly point
you in a better direction?
Ken
On May 11, 2005, at 12:57 PM, Jan Willem Luiten wrote:
Hello,
Your statement might not be true. I just tried about the same
thing with eomodeler. I now have this table also available in an
Openbase database. While in EOModeler I click the browse data
button. After fetching 1000 rows, EOModeler comes up with a dialog
telling me it fetched 1000 rows, giving me the choice to load
another 1000, load all the rest of the rows, or be satisfied with
the first 1000 rows.
If I do the same using PostgreSQL, I get the out of memory error.
If I remember correctly your assumption about WO loading all
tuples is incorrect. It probably is the coordination between WO
and the adaptor that causes problems in case of PostgreSQL.
Any other suggestions?
Kind regards,
Jan Willem Luiten.
On May 11, 2005, at 5:44 PM, David LeBer wrote:
On May 11, 2005, at 11:36 AM, Jan Willem Luiten wrote:
Hello,
I'm using WO 5.2 in combination with PostgreSQL 8. I have a
table containing well over 600.000 tuples. With a WODisplayGroup
on this table I get an out of memory error. Anyone on the list
knows wether this would happen with other DB's as well or is
this a problem between EOF and PostgreSQL?
Any hints on how to solve this are more than welcome.
Kind regards,
Uh, yeah.
WODisplayGroup will fetch *everything* into memory (unless you
qualify the fetch). Probably not what you want to do with 600,000
rows.
Solutions:
- Don't allow your WODisplayGroup to load automatically. Make the
user qualify the fetch first.
- Roll your own batching/sorting/displaying controller that batch
fetches.
- Look at ERXBatchingDisplayGroup from ProjectWONDER (caveat: I
seem to remember there being issues with PostgreSQL and this
component, search the list)
;david
--
David LeBer
Codebase Software Systems
site: http://www.codebase.ca
blog: http://david.codebase.ca
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
40anderhome.com
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden