Re: Dynamically changing the number of columns included in a D2W query/listing
Re: Dynamically changing the number of columns included in a D2W query/listing
- Subject: Re: Dynamically changing the number of columns included in a D2W query/listing
- From: Ramsey Gurley <email@hidden>
- Date: Wed, 22 Dec 2010 11:49:23 -0500
On Dec 22, 2010, at 11:09 AM, Daniel Beatty wrote:
Thank you for responding Ramsey,
On Dec 22, 2010, at 6:44 AM, Ramsey Gurley wrote:
On Dec 22, 2010, at 1:08 AM, Daniel Beatty wrote:
Greetings Gentlemen, Chuck, and Davids,
I am exploring how to limit the number of columns queried and used
in for a listing in D2W. My experiment uses ERModernLook, and has
a the SDSS DR1 data set. Several of the tables have an annoyingly
large number of columns (641 to be precise) and a large number of
rows (770k of them). If the JVM is tweaked to 3GB of memory,
ERModernLook can return a list from a blank query (taking nearly
2.5GB memory footprint). I have a few more tables with just
about as a bad of a scheme and size arrangement. It would be
nice to use this data set to measure just how much abuse D2W can
handle.
It sounds like you're mainly interested in just thrashing the app
to see what it can do under stress. That database table sounds
sufficient for the job. Are you putting all 770K rows on a single
page? (^_^) I think it would be interesting to see it using a
"real world" setup with a batching display group also. That would
probably reduce the memory usage a lot, but that's still a ton of
columns to render.
For the starting app, I literally reverse engineered the original
database from inside a framework and am driving the program from an
ERModernLook about as straight out of the box (by your example from
WOWODC 2010), and ran it. The next step I took was to increase the
memory available to the JVM to 3GB.
One thing to be proud of, ERModernLook has a semi-batching display
group, but still loads the entire table into memory.
Try
100 : (entity.name = 'BestTSField' and task = 'list') =>
useBatchingDisplayGroup = "true" [BooleanAssignment]
and see if that changes anything. Note from the
ERXBatchingDisplayGroup doc (since the javadocs are down today for
relocation)...
Extends WODisplayGroup in order to provide real batching. This is done
by adding database specific code to the select statement from the
EOFetchSpecification from the WODisplayGroup'sEODataSource which must
be an EODatabaseDataSource. If used with other datasources, it reverts
to the default behaviour.
I am starting with the one of the large tables. Obviously, the
small tables load without flaw. The rule set I have for the large
table (so far) includes the following:
"60 : (entity.name = 'BestTSField' and task = 'list') =>
batchSize = 5 [com.webobjects.directtoweb.Assignment]",
"60 : (entity.name = 'BestTSField' and task = 'list') =>
fetchLimit = 5 [com.webobjects.directtoweb.Assignment]",
"60 : (entity.name = 'BestTSField' and task = 'list') =>
displayPropertyKeys = (run,rerun,camcol,field)
[com.webobjects.directtoweb.Assignment]",
"60 : entity.name = 'BestTSField' => queryDataSourceDelegate =
edu.ttu.cs.dcglab.dr1.model.ListBestTSFieldQueryDataSourceDelegate
[ERDDelayedObjectCreationAssignment]",
I currently have the last rule disabled since
"edu.ttu.cs.dcglab.dr1.model.ListBestTSFieldQueryDataSourceDelegate"
is not implemented. While these are set the way they are, it does
not seem to have an effect on retrieval of this large table. It
still basically loads the entire table into memory.
Since this is a "real world" database that even Microsoft Research
tried improve, it seems to be a worthy challenge to make
ERModernLook tackle it. I can podcast the next effort and place it
on scorpius.hpcc.ttu.edu. That machine is a Mac Pro running OSX
Server 10.6.5 (developer seed) with about 6GB or RAM. I still need
to talk the powers that be to open the ports podcast producer use to
allow outside access, but I think that they can honor a community of
interest.
A couple things would be handy for this experiment.
1. How do I use ERDQueryDataSourceDelegateInterface? It appears
that the interface requires implementation of the queryDataSource
and qualifierFromSender methods.
Just the queryDataSource method really. In the documentation I put
on the wiki, I included a qualifierFromSender() method since
customizing the qualifier on a query page seemed to be a common
request.
2. Is there a way to specify different query data sources for
different tasks for the same entity (list versus inspect)?
I think the datasource is passed from the query page to the list
page. I may be mistaken, I'd have to look to be sure. Every
inspect page I've seen just uses the object(). No datasource
involved.
So the task would inherently be "query", it would not matter if the
result were list or inspect.
3. How do I get more run time logging from the WO application
itself? I thought Shark could do something with the JVM. Can it
measure the memory foot print over time? If so, how?
I'm not sure, but I've heard Pascal mention he is using Nagios to
monitor memory usage.
Well lets hope that Pascal has some ideas.
Thank you,
Dan Beatty, M.S. CS (B.S. EECS)
Ph.D. Student
Texas Tech University
email@hidden
http://web.me.com/danielbeatty/My_Home_Page/Welcome.html
(806)438-6620
Ramsey
Thank you,
Dan Beatty, M.S. CS (B.S. EECS)
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden