Re: basic Core Data scaling question
Re: basic Core Data scaling question
- Subject: Re: basic Core Data scaling question
- From: Michael B Johnson <email@hidden>
- Date: Tue, 2 Sep 2008 20:16:04 -0700
Thanks Ben. This was all great information.
The doc link was the key - after I posted, and kept sampling my app, I
figured out a number of these myself, but the doc made me more
confident that I wasn't missing something in Core Data.
On a tangential note, how can I get Instruments to give me the same
display that Activity Monitor does when I poke "Sample Process" and
select "percent of thread"?
That's what I ended using to debug this.
I've gotten my speed down to 30 seconds to read 170,000 objects from
a .csv file and display them in the autogenerated UI from IB, so I'm
happy for now. Now on to the real issues...
On Sep 2, 2008, at 2:11 PM, Ben Trumbull wrote:
Michael,
There are three common issues.
First has to do with keeping a stable heap and general memory
management. You can find suggestions about how to import a lot of
data with Core Data in the Core Data Programming Guide <http://developer.apple.com/documentation/Cocoa/Conceptual/CoreData/Articles/cdImporting.html#//apple_ref/doc/uid/TP40003174
>
The second is faulting. There is a Core Data template in
Instruments that can give you an idea if you're doing large
quantities of fine grained faulting.
The third is using a vanilla -setImages: or -setValue:forKey:
operation on a large to-many relationship. When setting to-many
relationships, you really really want to use either the -
mutableSetValueForKey:, or an accessor like -addImageObject or -
addImages. You want to leverage KVO's to-many variations on -
willChange/-didChange like
- (void)willChangeValueForKey:(NSString *)inKey withSetMutation:
(NSKeyValueSetMutationKind)inMutationKind usingObjects:(NSSet
*)inObjects
The reason for this is that the vanilla -willChange KVO operation,
typically for something like -setImages or -setValue:forKey:
basically removes ALL the old objects, and then adds ALL the new
objects. If you're just adding 1 new image at a time, then you've
just dirtied all the other objects in the relationship. That's a
lot of KVO notifications, and a lot of dirtied objects that need to
be saved. This mistake would be resaving N objects N times.
If you're on Leopard, the dynamic accessors for -addImageObject:
will automatically do the right thing. See the programming guide's
section on Managed Object Accessor Methods. If you'd like a custom
to-many accessor method, see the recommended implementation & KVO
support by selecting the relationship in the modeling tool, going to
the Design Menu -> Data Model -> Copy Obj-C 2.0 Implementations to
Clipboard
and pasting into an empty text file.
If the Image data itself runs more than 8-16K, you should consider
putting it directly on the file system, and keeping an URL in the
database instead. SQLite *can* handle binary data in the gigabytes,
so this is a guideline not a rule. However, can != should. The
file system is very very good at what it does. Being able to mmap
images and other multimedia, or use a No Cache read, is often a
valuable option.
Depending on what you're doing, putting the thumbnails in the
database can be a reasonable compromise between theoretical peak
performance and practical engineering limitations (e.g. your time).
- Ben
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden