Mailing Lists: Apple Mailing Lists
Image of Mac OS face in stamp
 
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

re: CoreData and NSOperation

Alex,

Conceptually, you should treat NSOperations as if they were on a separate thread. The OS may take certain liberties with implementation details based on system load and other factors, but basically NSOperations might as well be described as a light weight mechanism for creating threaded tasks.

Importing tasks are often easily parallelizable by simply importing 1/Nth of the data on a thread/operation. Here's an excerpt of some code I've been working with recently. It's GC and non-GC compatible, and has 3 implementations for comparison: NSOperation, NSThread, and boring serial code. As you can see, the NSOperation version is basically the same in terms of thread handling, but NSOperationQueue provides some convenient out-of-box handling for finding out when the tasks are complete. The NSThread code has whacky NSConditions and memory barriers.

The key to making this pattern useful is that each element in the work queue ('keyQueues' below) is sufficiently large to be worth the overhead of queuing up. In this sample code, each key is a file path, so this is importing from a directory of files, importing 'maxCores' files simultaneously.

This division of labor doesn't work if the data in each 1/N sets has relationships to data in other import groups.

static OSSpinLock _queueLock;
static NSOperationQueue* _operationQueue;
static NSDate *_startDate;

#define USE_NSOPERATIONS 1
// #define USE_NSTHREADS 1

- (IBAction)createEntities:(id)sender
{
	_startDate = [[NSDate date] retain];

_operationQueue = [[NSOperationQueue alloc] init];
NSUInteger j = 0;
NSUInteger maxCores = [[NSProcessInfo processInfo] activeProcessorCount];
NSMutableArray* keyQueues = [[NSMutableArray alloc] init];
for(NSString* key in s_importFileSet) {
[keyQueues addObject:key];
}
#if USE_NSOPERATIONS
for (j = 0; j < maxCores; j++) {
NSOperation* op = [[NSInvocationOperation alloc] initWithTarget:self selector:@selector(processFilesForKeys:) object:keyQueues];
[_operationQueue addOperation:op];
[op release];
}
#elif USE_NSTHREADS
_condition = [[NSCondition alloc] init];
_notFinished = maxCores;
OSMemoryBarrier();
for (j = 0; j < maxCores; j++) {
[NSThread detachNewThreadSelector:@selector(processFilesForKeys:) toTarget:self withObject:keyQueues];
}
#else
for (j = 0; j < maxCores; j++) {
[self processFilesForKeys:keyQueues[j]];
}
#endif
[NSThread detachNewThreadSelector:@selector(finishImportOperation:) toTarget:self withObject:keyQueues];
}

- (void)finishImportOperation:(id)keys {
NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];
#if USE_NSOPERATIONS
[_operationQueue waitUntilAllOperationsAreFinished];
#elif USE_NSTHREADS
[_condition lock];
while (_notFinished > 0) {
[_condition wait];
}
[_condition unlock];
#else
#endif
[keys release];
[_operationQueue release];
_operationQueue = nil;
NSLog(@"Total create time %f", [[NSDate date] timeIntervalSinceDate:_startDate] );
[_startDate release];
[pool drain];
}

- (void)processFilesForKeys:(NSMutableArray*)importKeys {
NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];
DataDumpImporter* importer = [[DataDumpImporter alloc] init];

NSPersistentStoreCoordinator* mainPSC = [[appDelegate managedObjectContext] persistentStoreCoordinator];
NSPersistentStoreCoordinator* psc = [[NSPersistentStoreCoordinator alloc] initWithManagedObjectModel:[mainPSC managedObjectModel]];
[psc addPersistentStoreWithType:NSSQLiteStoreType configuration:nil URL:[[[mainPSC persistentStores] lastObject] URL] options:[NSDictionary dictionaryWithObject:[NSDictionary dictionaryWithObject:@"0" forKey:@"synchronous"] forKey:NSSQLitePragmasOption] error:nil];
// we disable synchronous because if an import fails, we can delete the file and re-import.
// if you can't just delete the file, don't do this

	NSManagedObjectContext *moc = [[NSManagedObjectContext alloc] init];
	[moc setPersistentStoreCoordinator:psc];
    [psc release];
	[importer setImportPath:[self importPath]];
	[importer setMoc:moc];

	[moc setUndoManager:nil];

while (1) {
NSString* key = nil;
OSSpinLockLock(&_queueLock);
key = [importKeys lastObject];
if (key) {
[importKeys removeLastObject];
}
OSSpinLockUnlock(&_queueLock);
if (!key) {
break;
}
@try {
DataDumpImporterParams *params = [s_entityImporterParams objectForKey:key];
[importer importFile:[params filename] usingEntity:key andFlags:[params flags]];
} @catch (id e) {
NSLog(@"e = %@", e);
}
}
[importer release];
[moc release];
[pool drain];
OSAtomicDecrement32Barrier(&_notFinished);
[_condition lock];
[_condition signal];
[_condition unlock];
}

--

-Ben
_______________________________________________

Cocoa-dev mailing list (email@hidden)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/email@hidden

This email sent to email@hidden


Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2007 Apple Inc. All rights reserved.