I am working on some code to
iterate over the directory tree of an entire volume. I have run into
two different issues (although they may be different symptoms of the same
problem). Note that this is on OS X 10.4.10. I have not tested it
on Leopard yet.
Issue 1:
My code recursively walks the tree, making calls to FSGetCatalogInfoBulk. I noticed that the counts of files/folders was changing from test
to test and I thought it was probably due to temp files getting created.
I tested this assumption by starting a tree scan, then using Safari while the
scan was running. This forced cache files to get created in the
/Users/username/Library/Caches/Safari/... folders. This is where the odd
behavior started. Even though the contents of the /Users/username/Library/Caches/Safari/...
folders was changing, the iteration of the root of the volume (MacintoshHD) was
getting messed up.
When iterating the contents of the root of MacintoshHD (the
startup volume), the maximumObjects requested in the call to FSGetCatalogInfoBulk was 22. There
were 51 items in the root folder. The first time through the loop, 22
items were retrieved with the FSGetCatalogInfoBulk call.
The second time through the loop, 22 items were also retrieved. However,
the first 2 items retrieved where the same as the last 2 items from the
previous batch. The last time through the loop, 7 items were
retrieved. The total items retrieved was 51, but since 2 items were
retrieved twice, the final 2 items were skipped. My understanding is that
this can happen if the contents of the container being iterated over change.
However, the root folder contents are not changing. The contents of a
folder several levels down are changing. I don’t see why this
should affect the root folder iteration.
I could get this behavior 50-60% of the time by using Safari while
a tree scan was being performed. This wasn't ideal, so I set up a script
to simulate the cache files getting created. I created a Test subfolder
in /Users/username/Library/Caches/Safari/ and used Applescript plus
Automator to copy a new file into that folder every 5 seconds. Initially,
this gave me the same behavior as above. I'm not sure why, but the
behavior then changed for some reason.
Issue 2:
I got the behavior I'm about to describe in my real app, but to
isolate the problem, I created a small test app to reproduce the
problem. This app recursively walks the directory tree starting at the
/Users/username/Library/Caches/ folder. This occurs inside a loop to get
it to walk that small branch 100's of times. While this app runs, the script
to copy files to the /Users/username/Library/Caches/Safari/Test folder also
gets run. It takes a different number of times through the loop each time,
but I see the same behavior every time: the iteration of the
/Users/username/Library/Caches/ folder gets messed up. The first 22 items
are found correctly. For the next batch, FSGetCatalogInfoBulk says that there are 15 items. However, there are only 14
items left in that directory. I am retrieving arrays of FSRef,
FSCatalogInfo and file name objects through the FSGetCatalogInfoBulk call. When the
iteration gets messed up, it is always the same. The 13th
element of the FSRef array has an invalid FSRef object and the 13th element in
the file names array is "TemporaryItems" every time. Again, the
folder two levels down from this one is being updated, but the iteration of the
Caches folder gets messed up.
I made a different version of this app that uses scandir instead
of FSGetCatalogInfoBulk. I get the same behavior.
This seems to have something to do with the Caches folder. I
made a copy of the Caches folder and put it somewhere else on disk. I
updated my script to copy files to my Test subfolder of that Caches folder copy
instead. I updated the test app to walk the tree starting at that copy of
the Caches folder. I did not see the same behavior.
Is there something special about the Caches folder? Should I
avoid it when walking the tree? Could that also explain the behavior of
issue 1?
I can post my test code if necessary. I will need to clean
it up a little first.
Thanks in advance for any help!