Re: Managing EOF caching
Re: Managing EOF caching
- Subject: Re: Managing EOF caching
- From: "Jerry W. Walker" <email@hidden>
- Date: Fri, 20 Jan 2006 23:23:40 -0500
Hi, Michael,
I was aware of these two, but thanks for sharing them. Heh, the
greatest barrier to communication is the belief that it's already
occurred. :-)
Ken Anderson asked why I had extended this thread so far into the
area of database issues. I hadn't believed that I had, but I have a
general resistance to premature optimization. In this case, I was
resisting a reasonable suggestion based on my own ignorance of
database internals. I want to explore Ken's approach more deeply, it
sounds like a great suggestion.
In any case, I will keep calling these issues into question and will
continue to appreciate efforts like Mike Schraggs to put some fact
behind the speculation. The only problem with such facts is, once the
point is made and everyone who reads it starts adopting the
heuristic, the world of technology changes some more and it's no
longer true, or, at least, it opens again to question.
A few years ago, Bill Bumgarner posted on one of the lists several
very good reasons why one should never store images for a WO website
in a database. Many of his reasons still hold, but both database and
disk storage technologies have advanced far enough that it's no
longer such a cut and dried argument. At this point, I think that
argument has moved from very one sided to at least contentious.
Perhaps in a couple more years, it will move all the way to the other
side. My point being that we need to keep the optimization heuristics
open to question and keep testing them once in a while. Or take the
old approach, "make it work, make it work right, make it fast" since
some significant optimizations will remain both application specific
and anti-intuitive and because all optimizations come at some cost.
In any case, it's been an interesting discussion.
Regards,
Jerry
On Jan 20, 2006, at 5:54 PM, Michael DeMan wrote:
Hi Jerry,
A couple of other items off the top of my head in order to manage
your memory utilization as you start trying to take advantage of
the EOF caching...
1. assuming you are not using undo().
defaultEditingContext().undoManager().setLevelsOfUndo(1);
2. On your snapshots, the method you want to drop that to less than
an hour for refresh is:
defaultEditingContext().setFetchTimestamp()
It used to be that EOF still did not flush out the snapshots
correctly but I believe that was fixed a couple of years ago.
Apple badly needs to get a Tech Bulletin on EOF memory and
performance optimization. There are probably a few hundred people
at most with deep WO/EOF experience on this stuff and they all have
their bag of tricks and nobody knows each other's tricks that well
and all the tricks have one caveat or another. WO works great for
small apps, but the minute you get a decent sized application the
tuning becomes an issue.
On Jan 20, 2006, at 10:20 AM, Jerry W. Walker wrote:
Hi, Mike,
Thanks for taking the time to do this.
If I understand your test correctly, in each of the two cases,
you've looped through the 10,000 EOs updating a single one of your
12 attributes. You then did a saveChanges to update the database,
resulting in 10,000 individual SQL updates hitting the database,
correct?
So, the result was 10,000 round trips to the DB, one for each row
updated?
If all of that is correct, and if it's a trivial change to test,
would it be possible to test the difference between a SINGLE
locked column vs all columns locked?
You've taken the time to do something helpful here and I don't
want to reward you by suggesting you need to do a lot more work,
but the test I'm suggesting would put the results more in line
with the original contention, that locking a single attribute
saves a significant amount over locking all the attributes (in
your case, 12).
If you would rather not take the tests any further, would you mind
sending me a tarball of your current test project? I would be
pleased to do it.
If all of the above is correct, your results say that the
difference for a single update is approximately:
27.753 sec / 10,000 = 2.7 ms / update
vs.
7.111 / 10,000 = .7 ms / update
So we've achieved a 2 ms savings per update between no checking
and checking 12 columns.
If that's the case, my untested (ignorant?) opinion that the
difference is insignificant (and is probably less than 10 ms)
stands corrected. Much as I hate to be wrong, my yearning for
facts over both my ignorant opinions and other WO "old wive's
tales" leads me to offer you unequivocal thanks for your efforts.
Regards,
Jerry
On Jan 20, 2006, at 12:54 PM, Mike Schrag wrote:
I haven't done any timings, but I find it hard to believe that
this technique will effect any significant savings for most
databases. As Art indicates, the only effect of reducing the
number of lock columns is to correspondingly reduce the number
of columns in the WHERE clause for updates and deletes.
I've always wondered this, too, so I threw together a quick
test. I created two identical entities which have 3 ints, 3
booleans, 3 strings, and 3 dates as properties, but one locks all
attrs and one locks none. I then inserted 10,000 of each into
the db, followed by an update of the same attribute for each with
the same change in value.
This was run on the latest version of FrontBase on an Intel iMac
with WO 5.3:
Main.main: 10000 with all columns locked = 25753ms
Main.main: 10000 with no columns locked = 7111ms
So in this test, it is 3.6x faster to not lock any columns in
this example, with this database, one this hardware. Obviously
db benchmarks are never this simple, so take this with a grain of
salt.
ms
--
__ Jerry W. Walker,
WebObjects Developer/Instructor for High Performance Industrial
Strength Internet Enabled Systems
email@hidden
203 278-4085 office
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
40geminisolutions.com
This email sent to email@hidden
--
__ Jerry W. Walker,
WebObjects Developer/Instructor for High Performance Industrial
Strength Internet Enabled Systems
email@hidden
203 278-4085 office
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden