Re: Core Data performance advice... creating relationships.
Re: Core Data performance advice... creating relationships.
- Subject: Re: Core Data performance advice... creating relationships.
- From: Phil <email@hidden>
- Date: Tue, 15 Jan 2008 15:57:08 -0500
On Jan 15, 2008, at 11:22 AM, I. Savant wrote:
I appreciate your followup messages. I had tried a schema of similar
complexity (granted, it was a larger data set) and had fairly
unfavorable results under Tiger (re: memory usage and performance.)
It sounds like it might be worth another look under Leopard.
With a statement like that, you've *got* to provide more detail. :-D
What was your schema? What was the size of the data set (approximate
number of each entity's instances, if you know it)? What did you find
to be slow/memory-intensive, specifically? How did you implement that
which you found to be slow/memory-intensive?
Inquiring minds want to know! I'm not saying "I don't believe it,
prove it!" ... merely curious. I too have come across a number of
inefficiencies but most of them had fairly simple work-arounds once
the problem area was identified.
No problem providing more detail...
When I was first looking at Core Data a couple of years ago, I
attempted to migrate a moderate sized data set (~300 Meg in a vanilla
SQLite database over about two dozen tables) for an application that
was very data processing oriented (i.e. very little GUI, mostly an ETL
job) to Core Data and after getting everything loaded, setting up
relationships and predicate templates that were as selective as
possible, it still appeared to be performing full table scans most, if
not all, of the time because Core Data would apparently not see the
value of index creation of critically important columns from a
performance standpoint (I confirmed via the command line SQLite
interface but had read and was told that CD would take care of such
things automatically. I'm sure it did but have no idea what criteria
it used to determine since it didn't seem to take any of the hints I
was giving it... was it using: relationships in the model? predicate
templates? database stats? phase of the moon?) and I couldn't see a
(supported) way to force it to create them.
For example, one of the key tables involved was ~10 Million rows of
daily stock quote data (i.e. ticker, date_id, open, high, low, close,
etc.) which had relationships to the security (ticker, companyName,
etc.) and date (date_id, date, yearMonth, etc) tables. For the life
of me, I could never figure out how to get Core Data to decide to
create the appropriate indexes so that when I was after all daily
quotes where companyName='Apple Computer' and
yearMonth='2007-11' (i.e. highly selective criteria) it would always
want to fetch back 10 million managed objects even though I really
only cared, at most, about a few thousand. It seemed like the more
selective my query got, the worse Core Data would perform. I assumed
that this was because it was deciding that it needed to fetch
everything back and apply the filtering criteria itself which makes
sense if that is what it was actually doing. This assumption
appeared to be supported by watching memory usage for my app memory
usage to shoot up to 500+ Meg at times even though I was never working
with more than a fraction of the data set at a time and being quite
liberal in my use of autorelease pools. (This was mostly speculation
on my part because Core Data was fairly opaque re: what was going on
with the data store so I was never really certain what was going on.)
This was all quite some time ago and I didn't bother revisiting it for
anything nearly that large since as there were other things about Core
Data that turned me off to using it for anything that was primarily
database-oriented in nature. As some of these issues appear to be
addressed in Leopard, it looks like it's worth taking at least another
look and possibly expanding the cases where I should consider it as a
solution.
Thanks,
Phil
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden