Re: Dynamically changing the number of columns included in a D2W query/listing
Re: Dynamically changing the number of columns included in a D2W query/listing
- Subject: Re: Dynamically changing the number of columns included in a D2W query/listing
- From: Daniel Beatty <email@hidden>
- Date: Wed, 22 Dec 2010 08:09:46 -0800
Thank you for responding Ramsey,
On Dec 22, 2010, at 6:44 AM, Ramsey Gurley wrote:
>
> On Dec 22, 2010, at 1:08 AM, Daniel Beatty wrote:
>
>> Greetings Gentlemen, Chuck, and Davids,
>> I am exploring how to limit the number of columns queried and used in for a listing in D2W. My experiment uses ERModernLook, and has a the SDSS DR1 data set. Several of the tables have an annoyingly large number of columns (641 to be precise) and a large number of rows (770k of them). If the JVM is tweaked to 3GB of memory, ERModernLook can return a list from a blank query (taking nearly 2.5GB memory footprint). I have a few more tables with just about as a bad of a scheme and size arrangement. It would be nice to use this data set to measure just how much abuse D2W can handle.
>
> It sounds like you're mainly interested in just thrashing the app to see what it can do under stress. That database table sounds sufficient for the job. Are you putting all 770K rows on a single page? (^_^) I think it would be interesting to see it using a "real world" setup with a batching display group also. That would probably reduce the memory usage a lot, but that's still a ton of columns to render.
For the starting app, I literally reverse engineered the original database from inside a framework and am driving the program from an ERModernLook about as straight out of the box (by your example from WOWODC 2010), and ran it. The next step I took was to increase the memory available to the JVM to 3GB.
One thing to be proud of, ERModernLook has a semi-batching display group, but still loads the entire table into memory.
I am starting with the one of the large tables. Obviously, the small tables load without flaw. The rule set I have for the large table (so far) includes the following:
"60 : (entity.name = 'BestTSField' and task = 'list') => batchSize = 5 [com.webobjects.directtoweb.Assignment]",
"60 : (entity.name = 'BestTSField' and task = 'list') => fetchLimit = 5 [com.webobjects.directtoweb.Assignment]",
"60 : (entity.name = 'BestTSField' and task = 'list') => displayPropertyKeys = (run,rerun,camcol,field) [com.webobjects.directtoweb.Assignment]",
"60 : entity.name = 'BestTSField' => queryDataSourceDelegate = edu.ttu.cs.dcglab.dr1.model.ListBestTSFieldQueryDataSourceDelegate [ERDDelayedObjectCreationAssignment]",
I currently have the last rule disabled since "edu.ttu.cs.dcglab.dr1.model.ListBestTSFieldQueryDataSourceDelegate" is not implemented. While these are set the way they are, it does not seem to have an effect on retrieval of this large table. It still basically loads the entire table into memory.
Since this is a "real world" database that even Microsoft Research tried improve, it seems to be a worthy challenge to make ERModernLook tackle it. I can podcast the next effort and place it on scorpius.hpcc.ttu.edu. That machine is a Mac Pro running OSX Server 10.6.5 (developer seed) with about 6GB or RAM. I still need to talk the powers that be to open the ports podcast producer use to allow outside access, but I think that they can honor a community of interest.
>
>> A couple things would be handy for this experiment.
>>
>> 1. How do I use ERDQueryDataSourceDelegateInterface? It appears that the interface requires implementation of the queryDataSource and qualifierFromSender methods.
>
>
> Just the queryDataSource method really. In the documentation I put on the wiki, I included a qualifierFromSender() method since customizing the qualifier on a query page seemed to be a common request.
>
>
>> 2. Is there a way to specify different query data sources for different tasks for the same entity (list versus inspect)?
>
> I think the datasource is passed from the query page to the list page. I may be mistaken, I'd have to look to be sure. Every inspect page I've seen just uses the object(). No datasource involved.
So the task would inherently be "query", it would not matter if the result were list or inspect.
>
>> 3. How do I get more run time logging from the WO application itself? I thought Shark could do something with the JVM. Can it measure the memory foot print over time? If so, how?
>
>
> I'm not sure, but I've heard Pascal mention he is using Nagios to monitor memory usage.
Well lets hope that Pascal has some ideas.
Thank you,
Dan Beatty, M.S. CS (B.S. EECS)
Ph.D. Student
Texas Tech University
email@hidden
http://web.me.com/danielbeatty/My_Home_Page/Welcome.html
(806)438-6620
>
> Ramsey
>
>
>>
>> Thank you,
>>
>> Dan Beatty, M.S. CS (B.S. EECS)
>
>
>
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden