Re: Validating unique objects in CoreData
Re: Validating unique objects in CoreData
- Subject: Re: Validating unique objects in CoreData
- From: Roland King <email@hidden>
- Date: Sat, 13 Feb 2010 11:06:32 +0800
That's not a horrible solution, except for the feeling that core data ought to let you do what you want without having to implement your own UUID cache. I'm still a bit surprised that a lookup for an object by one attribute is taking so long, over just 30,000 objects. You do have the uuid attribute marked as indexed right?
I found http://cocoawithlove.com/2008/03/testing-core-data-with-very-big.html whilst hunting around for some examples of core data with big data sets. This guy was working on sets of 1 million objects and doing fetches with indexed properties was taking about 2 seconds, vs non-indexed, 600 seconds. There are some comments at the bottom from an apple engineer too.
On 13-Feb-2010, at 2:06 AM, malcom wrote:
> I've tried to make an alternative method. I'll try to describe it.
> I would to use a second an auxiliary index where to save my uuid<->objectURI; results are better, so I try to make a summary of the problem and the solution. If anyone have a better idea I'll be happy to talk about it :)
> I've about 30,000 messages taken from network and I need to save all into a Core Data Store in form of a tree.
> Each message contains an unique identification string and no more than one message can be saved on database with the same id.
> CoreData at this time does not support uniqueness of attributes and I can't use objectID property to ensure this kind of thing.
> A first solution is, in pseudo code:
> - Execute a query to see if uuid string is presents in storage
> - If it's not present I can make a new NSManagedObject with that uuid and put it into storage, otherwise I'll ignore it (it's already on db)
> - Execute another query to find the direct parent of this new message, if found I'll link both messages, if not it's a root message
>
> This solution has a big problem. With 30k messages I need of 30k query to check if the new message exist on coredata, another 30k to check for parent (plus, I think, another 30k to insert the new object). Over 60k+ queries takes lots of time (a minute or more here).
>
> My second solution is to create a second auxiliary NSMutableDictionary where i'll save message uuid as key and NSManagedObjectID's URI rapresentation (the only I can save to NSData) as value for dictionary entry.
> The result in pseudo code is:
> - Use objectForKey:uuid to my auxiliary dictionary to see if the message exist in coredata
> - If yes I'll ignore it. If not i'll put it into the store
> - Use objectForKey:parentuuid to my auxiliary dictionary to see if the parent of the message is present on coredata. If yes i'll use NSPersistentCoordinator's managedObjectIDForURIRepresentation: to get the NSManagedObject (the parent of the message) and link both parent and child
>
> With this solution the entire process takes about 5 seconds to finish (the result dictionary it's around 2mb).
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden