Re: EO Caching (2)
Re: EO Caching (2)
- Subject: Re: EO Caching (2)
- From: Chuck Hill <email@hidden>
- Date: Tue, 11 Dec 2007 10:19:12 -0800
On Dec 11, 2007, at 7:11 AM, Paul Hertz wrote:
At 4:37 PM -0800 12/10/07, Chuck Hill wrote:
On Dec 10, 2007, at 3:58 PM, Paul Hertz wrote:
/-- snip --/
As I understand it (more foggy than I'd like), the record from
the database will be discarded if the global ID that gets
generated for the fetched row corresponds to an object in the
editing context that is not a fault.
Yes. Unless the fetch is marked as refreshing refetched objects,
in which case the object store and all editing contexts are updated.
In that case, the object from the editing context is returned.
That should save some computational time (yes?),
Yes, but can also lead to inconsistencies. The objects returned
match the criteria as determined by the database. If another
process has updated the database, you may have objects in memory
that would have matched the criteria but which are not included in
the results as the in-memory values differ from the database.
but clearly not as much as if one knows the object is already in
memory and uses it without calling objectsWithFetchSpecification.
Especially not the case if the WO server and the database server
are separate machines, and time is taken up by network
communications.
That needs to also account for the time to evaluate the query
against the objects in memory vs letting the database do it and
moving the data around. It many situations, it ought to be faster.
So refetching all rows may be faster than letting the comparisons run?
I can think of situations where that may be true. I don't think it
is true in all situations.
I am, however, curious about what you are trying to fix. I have
never had to resort to this sort of thing.
This is a different problem from the one that started this thread,
though I think it touches the saem issues. I've added a (2) to the
subject line.
I'm revisiting code Jonathon Rochkind and I wrote years ago, when
we were first learning WebObjects. The page in question generates
reports on data from user-created surveys.
Reporting with EOF is a different situation than most pages.
Manipulating lots of data requires some planning.
Some of the content creators have gone overboard and put over 100
questions in a survey.
I am familiar with that phenomena.
For the surveys with many questions and hundreds of persons
replying, the report page takes so long to generate that WO reports
a timeout to the HTTP server and the client sees a timeout error
message instead of the report.
One quick solution to this is to use the WOLongResponsePage. This
is, more or less, what it was designed to handle.
A further complication: the timeout is only generated on the
deployed code--my development machine, in debugging mode, doesn't
time out.
You are probably using Direct Connect for development (and I will
just skip my rant on that topic :-). Direct connect does not go
through the woadaptor so there is no timeout. Lots of people get bit
by this "But it works on my machine!" problem.
I think the major culprits in the code are 1) repeated calls to
ec.objectsWithFetchSpecification (a separate fetch for each
question, repeated rather than cached),
That would be a problem. And this will be compounded if that fetch
is called by a binding. This is a good place to look at doing some
caching or pre-fetching.
and 2) faults fired one at a time on the fetched rows.
Batch faulting may or may not help here. I would strongly recommend
that you start using Project Wonder, even if only in the "library of
useful code" mode where you don't extend any of its classes.
ERXRecursiveBatchFetching will make quick work of optimizing these
fetches. HIGHLY recommended.
Perhaps 3) statistical calculations performed on the answers to
each survey question slow things down significantly, too.
I don't have much help for that, other than to suggest looking for an
open source Java library that has optimized calculations for these.
An additional problem is an EOAndQualifier that correctly filters
rows in a fetch but not in memory (filteredArrayWithQualifier).
Until I figure that out, I have to fetch answers for each question
separately--but at least I can eliminate repeated fetches for the
same question.
That is a curious one. I have usually seen that in the opposite
direction, it works in memory but not in a fetch.
Faults only need be fired on multi-choice questions with multiple
answers, which are accessed through a flattend relationship.
Flattened relationships (apparently) do not get pulled in through
batch fetching--the relationship has to operate on a concrete field
of the entity one is fetching. So, batch fetching may not be a
solution (modifyng the databse may be).
Again, ERXRecursiveBatchFetching can do some amazing stuff in a
single line of code.
Performing the fetches and statistical operations, etc., in a
threaded task in a WOLongResponse page, rather than having bindings
in the html/wod files trigger calculation, should provide a
reasonable solution to preventing timeouts, if not to speeding
things up. Even if the page still takes a long time to build and
load, the client will see some sort of progress indicator.
Yes. Not a perfect solution, but far better than an error page and
no results.
Finally, our WO server and our DB server are separate machines.
Possibly there are deeper-seated problems with the synchronization
of calls and database locking, but I'm going after the low-hanging
fruit first.
I would suspect lack of indexes needed to fully optimize the queries
you are issuing before ever considering anything that low level.
Chuck
--
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems.
http://www.global-village.net/products/practical_webobjects
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden