Re: basic Core Data scaling question
Re: basic Core Data scaling question
- Subject: Re: basic Core Data scaling question
- From: Quincey Morris <email@hidden>
- Date: Mon, 1 Sep 2008 20:30:44 -0700
On Sep 1, 2008, at 18:52, Michael B Johnson wrote:
The problem comes when I start adding the Images to the managed
store. I time how long it takes to add 100 at a time. The first
100 go in 0.022 seconds, but by the time I've inserted 4,200 of
them, it's taking 1 second/100, at 20,000 it's taking 6sec/100, and
by the time I'm up to 90,000, it's taking over 20sec/100. It
literally takes hours to chew through.
After much spelunking, I've found that it's when I set the project
relationship on the Image that is taking up all the time. If I
bring in all the Projects, Artists, Albums and Images without wiring
up the Images to the Project (although I do do it for the Albums),
the whole thing runs in about 30 seconds.
First of all, make sure you set the managed object context's undo
manager to nil while you import all the data.
Second, find out if it's a virtual memory problem or something within
Core Data that's slowing things down. Activity Monitor is fine for
this, because you don't need exactness. Watch your app's real and
virtual memory allocation while it imports, and see if it goes up
without limit. If so (which is my guess), it's likely only a memory
footprint problem.
Third, read the section of the Core Data programming guide about
importing data efficiently. Keep in mind that if you just keep adding
objects to a managed object context, they (and their properties) are
all kept in memory PLUS whatever caches Core Data keeps internally.
Generally, the approach for importing large amounts of data is
approximately this:
-- import some data, let's say 50-500MB, depending on how much RAM you
expect to have to play with
-- do a [NSManagedObjectContext save:error] to get everything that's
dirty written to disk
-- fault out everything you no longer need with
[NSManagedObjectContext refreshObject:mergeChanges:] (or release the
managed object context and create a new one)
-- drain the autorelease pool (if doing old-fashioned memory
management) or force garbage collection
Repeat for each successive batch of data.
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden