Re: Problem enumerating a directory
Re: Problem enumerating a directory
- Subject: Re: Problem enumerating a directory
- From: Andy Lee <email@hidden>
- Date: Mon, 30 Jul 2012 08:28:17 -0400
On Jul 27, 2012, at 12:44 PM, Mark Allan <email@hidden> wrote:
> I'm trying to list a directory recursively to build up a snapshot of the contents and store them in a core data DB, but keep running into issues with the directory list. The core data part is working fine as far as I can tell!
Just for grins, what if you take out the Core Data part and just do an NSLog?
NSLog(@"[%@] [%@]", thePath, theSubPath]);
I'm wondering if there's any chance an exception is being thrown in the Core Data part and you're accidentally ignoring it. It might help to include a counter in the NSLog, in case the enumeration is dying after some particular number or multiple of iterations.
Another thought: try running with NSZombie turned on. Maybe you have a memory error that you "get away with" for random amounts of time. Long shot, but I wonder in particular if thePath is getting clobbered somewhere, which *might* explain why it suddenly starts processing the root directory. (This is why I suggest NSLogging both thePath and theSubPath, separately, in case one of them is getting clobbered.)
> I then gave up on that approach and opted for the easier but slower cocoa solution (NSDirectoryEnumerator) but for some reason it gives up with neither an error nor a warning about half way through the tree. Could it be that modifications to the file system during the enumeration are causing it to fail?
I notice you mention "/Users/mark" as a value you've tried hardcoding. Your home directory seems a pretty big dataset to test with, and it will certainly be modified during the test. What if you try a big but less-huge subdirectory that you know won't change during your test?
A thought about why it seems to fail at random places -- I suspect NSDirectoryEnumerator isn't guaranteed to iterate in any particular order, so that *might* make the bug seem random. Another guess: are you using a thread (perhaps via NSOperation) to enumerate the directory? I'm guessing that's likely since you don't want to block your UI during this long operation. What if you do an enumeration, just for grins, on the main thread? Maybe have a button whose action is "testEnumeration:", and have it do nothing but enumerate and print NSLog statements?
If you're using an NSOperation, is there any place in your code where you cancel the operation, and might be doing so accidentally? Or maybe the NSOperation is under-retained and getting dealloc'ed prematurely?
When these "can't be happening" bugs occur, I find it sometimes helps to make it work *somewhere* and work forward from there to figure out what's not working in the place you really want. Throw away as many assumptions as possible, *especially* the very basic ones. Should it work with your home directory? Sure it should, but try with a smaller, unmutating directory anyway. Should it work in NSOperation? Sure it should, but try it on the main thread anyway. Should it work with the Core Data saves? Sure it should (well, if it's in a thread I'm not sure), but take that away too.
(Now after all this, I bet you find the bug just as I hit Send... :))
--Andy
>
>
> -(void) startSnapshotForPath:(NSString *) thePath {
> int theCounter = 0;
> NSDirectoryEnumerator *dirEnumerator = [[NSFileManager defaultManager] enumeratorAtPath: thePath];
> thePool = [[NSAutoreleasePool alloc] init];
>
> for (NSString *theSubPath in dirEnumerator) {
> [self addEntityWithPath:[thePath stringByAppendingPathComponent:theSubPath]];
> theCounter++;
> if (theCounter >= 1000) {
> theCounter = 0;
> [[self managedObjectContext] save:NULL];
> [[self managedObjectContext] reset];
> [thePool drain];
> thePool = [[NSAutoreleasePool alloc] init];
> }
> }
>
> [[self managedObjectContext] save:NULL];
> [[self managedObjectContext] reset];
> [thePool drain];
> }
>
> I've also tried Uli Kusterer's UKDirectoryEnumerator but that doesn't appear to be recursive! I suspect (although I haven't tried) that requesting the type of the path (i.e. file/directory) and creating a new UKDirectoryEnumerator for each subdirectory would be massively expensive.
>
> Does anyone have any suggestions for where I can go from here please? How can I find out why NSDirectoryEnumerator is failing half-way through the process, and how can I stop it doing so? Failing that, does anyone have a better suggestion for how I can build the snapshot please?
>
> Many thanks
> Mark
> _______________________________________________
>
> Cocoa-dev mailing list (email@hidden)
>
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
>
> Help/Unsubscribe/Update your Subscription:
>
> This email sent to email@hidden
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden