Re: [CoreData] Background Insertion
Re: [CoreData] Background Insertion
- Subject: Re: [CoreData] Background Insertion
- From: Aurélien Hugelé <email@hidden>
- Date: Thu, 7 Jun 2007 09:33:59 +0200
Very interesting thread,
I'll share my experience here, since I encountered this problem a
while back and dig a little on that subject.
1/ When a MOC is saved, it sends a private notification that is
listened by objects controllers, then send the documented
NSManagedObjectContextDidSaveNotification notification.
The private notification is highly optimized towards
NSArrayControllers since it precise which objects are new (inserted),
which objects are updated and which ones are deleted.
NSArrayControllers use a mergeSort algorithm to insert the newly
inserted object sorted accordingly to its sort descriptors (if any).
This is extremly important because above 10 000 objects, mergeSort()
is definitely faster than quickSort(). Suppose the array controller
contains 50K objects. When the private notification is posted, the
array controller will be *extremly* fast to insert the newly inserted
object at the right "index". It avoids a slower linear quickSort().
2/ I faked myself this private notification in order to use those
much needed optimizations (I had more than 100K objects...) I hate to
do that but it is the only solution I found to keep performance
acceptable, I use this since 10.4.0 and never had a problem since. I
just hate to do that.
3/ What happens if you do not fake this notification? Answer: You are
forced to use "prepareContent" or fetch yourself. It is simply
evil... What does prepareContent simply do ?
- It does register the array controller to the private notifications
(and probably does much more, but it is important to understand that
this is needed ONCE, not at eachof your dataSaved: call !!!!)
- it triggers a DELAYED fetch using the array controller fetch
predicate. This is what you see here: (see Marcus post)
#6 0x93c7b6d0 in -[NSArrayController(NSManagedController)
_performFetchWithRequest:merge:error:]
#7 0x93c7ebe8 in -[NSObjectController(NSManagedController)
fetchWithRequest:merge:error:]
#8 0x93c7ec74 in -[NSObjectController(NSManagedController)
_executeFetch:didCommitSuccessfully:actionSender:]
#9 0x90a461f4 in objc_msgSendv
#10 0x92bdec94 in -[NSInvocation invoke]
#11 0x92bdf244 in -[NSInvocation invokeWithTarget:]
#12 0x93c590e4 in _NSSendCommitEditingSelector
#13 0x93c7c600 in -[NSController
_controllerEditor:didCommit:contextInfo:]
#14 0x90a461f4 in objc_msgSendv
#15 0x92bdec94 in -[NSInvocation invoke]
#16 0x92bdf244 in -[NSInvocation invokeWithTarget:]
#17 0x92bedc04 in __NSFireDelayedPerform
Using invocation and performSelector:afterDelay:
So what is the problem here ?
Answer: performance:
Contrary to what apple has (correctly) done with the private
notification, and optimized insertion, using prepareContent triggers
a FULL fetch (this can be looooooooong), then replace the entire
content of the array controller with the fetch result. This is
extremely heavy compared to apple private mechanism.
Imagine you had a filter predicate (not a fetch predicate), the
entire array controller content is filtered again, when apple private
mechanism uses the already filtered content. Suppose you are sorting
the array controller content, not by using a sort descriptors (that
is passed to the fetch request so that it is executed by core data
and often by SQLite internally if you are using SQL store), but by
having subclassed NSArrayController arrangeObjects: method. Then all
the previous sorting you've done are not usefull and you have to
resort the entire content again!
I've listed several methods that triggers a delayed fetch:
prepareContent
fetch:(id)sender
maybe other ones I don't remember.
I won't encourage using the private notification because it is bad
(tm), but if you have performance problems because your background
thread massively insert objects you want to make appear in a
(filtered and sorted) array controller bound UI, then you know that
you can...
Aurélien,
Objective Decision Team
On 5 juin 07, at 23:55, Marcus S. Zarra wrote:
I wonder if it would be possible to see what notification is being
sent? If it is a true notification, then you could perhaps
simulate it yourself....
Might be worth exploring just to see if it can be cleaned up a bit.
Marcus S. Zarra
Zarra Studios LLC
Simply Elegant Software for OS X
www.zarrastudios.com
On Jun 5, 2007, at 3:51 PM, Andrew Kimpton wrote:
On Jun 5, 2007, at 3:56 PM, Andrew Kimpton wrote:
On Jun 5, 2007, at 3:15 PM, Marcus S. Zarra wrote:
This is very odd. I did get it to work by changing your
dataSaved: method to:
[listController prepareContent]; //MSZ Added
As to why this is necessary, I am stumped. It should be picking
it up automatically. This solves it but I am not sure if you
want to do something like this. Although having your delegate
broadcast a "data updated" notification might solve the issue.
When I do background processing, I close all of the windows that
display the data and bring up a progress dialog which is modal.
So I am probably not seeing this behavior because the controller
reload when the windows get presented. Now I will have to go
back into my code and see if I can duplicate this behavior and
join you in a radar filing...
Cool.... now I get valid objects - and the exact same symptoms
described above. At least I think it's the same I'm not able to
so easily verify that sending prepareContent fixes the problem -
I have a number of controllers and I don't generally have them
registered with any window controller that's also receiving the
completed save notification. I'll try that next...
I found that updating just one of my controllers seems to be
enough to cause 'everything' to come to life. Sending a
prepareContent message seems to trigger a notification that my
regular observeValueForKeyPath receivers pick up.
So.....
Use objectForID to fault the newly inserted items in the main thread
Use prepareContent on a controller to trigger updates for everyone
(?)
Problem (mostly - if not cleanly) solved - it still feels like the
prepareContent shouldn't be needed.
Andrew 8-)
Aurélien,
Objective Decision Team
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden