Re: CoreData huge memory usage - is this right ?
Re: CoreData huge memory usage - is this right ?
- Subject: Re: CoreData huge memory usage - is this right ?
- From: Frank Reiff <email@hidden>
- Date: Wed, 19 Dec 2007 10:34:27 +0100
Dear Ben,
Thanks for the comprehensive answer. It's nice to get a full
explanation from somebody who is in the know.
As a former full-time framework developer, I fully understand why you
are a bit defensive about the documentation (It's all in there!!):
there's always somebody complaining who has obviously not read it
properly and how can anybody expect to do anything this complicated
without taking the trouble of reading the documentation? Been there,
done that, bought the T-shirt.
On the other hand, when you have something as complicated as Core
Data, the documentation is almost as important as the framework
itself. Apple's documentation in general is high on specifics and low
on the "big picture" stuff. (I just wasted a day trying to get an
NSBrowser to work with bindings and then I'm off trying to figure out
how to resize an NSSplitView). Each method is documented just fine,
but how they interact and what role they play in the overall design of
the functionality is usually at best touched upon in the class
documentation.
There is a "big picture" documentation for Core Data and it does a
better job than much of the AppKit (NSBrowser, etc.) documentation,
but to be honest it's still far from great. As this import/ memory
leak question seems to come up a lot, it might be worth putting your
explanation into the Core Data FAQ and perhaps even making some sample
code available for it.
A great example of good documentation for a framework just like this
is Hibernate's manual (http://www.hibernate.org/), which does a great
job of demonstrating functionality by example and discussing the
design decisions in some detail.
The classical object graph retaining problem that you describe IMHO is
a fundamental weakness of the Core Data framework. I'm sure you've got
sound technical reasons for putting so much emphasis on inverse
relationships even where they don't make sense, but in practice they
are awkward to use. It's the age old question of whether your object-
relational mapping tool should be more "relational" or more "object"-
oriented. In my personal opinion, it's too "relational" when it
refuses to support ordered collection directly and it's too "object"-
oriented when it forces (ok, strongly recommends that) you to define
inverse relationships.
That's just my 5 euro-cents of feedback. I'm a happy Core Data user
and the new enhancements for Leopard rock.
Keep up the good work.
Best regards,
Frank
On 19 Dec 2007, at 02:46, Ben Trumbull wrote:
Martin,
We recommend an import use its own MOC, and simply set the undo
manager on the MOC to nil. Easier, better, less filling.
The staleness interval has nothing to do with imports. It affects
the, well, staleness of cached data for faulting.
It's insufficient to drain the autorelease pool every 1000
iterations. You need to actually save the MOC, then drain the
pool. Until you save, the MOC needs to retain all the pending
changes you've made to the inserted objects. Losing changes just
because somebody popped an autorelease pool would be bad. This is
documented in several places, including the programming guide and -
setRetainsRegisteredObject:
There is an additional issue that complicates matters greatly under
the retain/release (not GC) memory model. Managed objects with
relationships nearly always create unreclaimable retain cycles.
Example:
Department <->> People
Department has a relationship to many people, and each person has an
inverse relationship to the department. Since these objects retain
each other, the retain game is over. This is a fundamental
limitation to retain counting.
To help alleviate this, Core Data allows you to turn a managed
object back into a fault. You can turn a specific object back into
a fault with -refreshObject:mergeChanges:NO (which shouldn't be
called on a managed object with unsaved changes pending). Or you
can use the sledge hammer, and just -reset the whole context. A big
but not quite that big hammer would be to call -registeredObjects on
the MOC you just saved, and then for (NSManagedObject* mo in
registeredObjects) { [moc refreshObject:mo mergeChanges:NO] } This
is a bit like reset, but you can still re-use the managed objects
themselves in the future.
As for why can't Core Data break the retain cycles for the objects
you no longer want (i.e. like -reset, but not everything), how would
we know ? We can't tell the difference between your UI/array
controller/whatever retaining an object, and an inverse relationship
retaining the object. It's trivial enough for you to grab the -
registeredObjects set, make a mutable copy, remove the objects you
want to preserve, and refresh the rest. And if you can't tell the
difference between the objects you want to preserve and those you
want to nuke, how could we ?
If you have concrete ideas for an API that would make this easier,
please file an enhancement request at bugreport.apple.com
The only meaningful difference between -reset and -release on the
MOC is whether or not you ever need to use that MOC object again.
And, now under GC, -reset is more immediate.
It may occur to some to ask why turning the object back into a fault
helps. Faults represent futures for unrealized/unmaterialized
pieces of a graph of connected objects. They don't retain anything
as they have no data, just an identity. A managed object that's in
use has data, and retains that data much in the same way any Cocoa
object's standard setter methods might work. The active pieces of
the graph are wired down, and the periphery are faults. Like a
cloud. If you turn enough of the center pieces of the cloud back
into faults, you'll have multiple distinct clouds instead (which is
usually better)
MOC are not particularly expensive to create, so if you cache your
PSC, you can use different MOCs for different working sets or
distinct operations.
I can import millions of records in a stable 3MB of memory without
calling -reset.
- Ben
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden