I figured out why issue 2 was happening, so please ignore
that. I still haven’t figured out what is causing issue 1, so any
help with that would be very much appreciated. Thanks!
The strange behavior I was seeing in Issue 2 below was due to
the way I was simulating Safari creating cache files. I created an Automator
script to copy a file into the /Users/username/Library/Caches/Safari/Test
folder. Since I am running this on Tiger, there is no looping mechanism
available in Automator. I had to save my Automator script as an
application, then used Applescript to run that app inside a loop. Launching that
app from the Finder creates a folder called TemporaryItems in the /Users/username/Library/Caches/
folder. When the app exits, that folder is deleted. Since my
Automator script app only runs for a split second, I did not see this folder
get created.
From: Trainor, Chris
Sent: Tuesday, February 12, 2008 10:36 AM
To: 'email@hidden'
Subject: Strange behavior iterating the directory tree of a volume
I am working on some code to iterate
over the directory tree of an entire volume. I have run into two
different issues (although they may be different symptoms of the same
problem). Note that this is on OS X 10.4.10. I have not tested it
on Leopard yet.
Issue 1:
My code recursively walks the tree, making calls to FSGetCatalogInfoBulk. I noticed that the counts of files/folders was changing from test
to test and I thought it was probably due to temp files getting created.
I tested this assumption by starting a tree scan, then using Safari while the
scan was running. This forced cache files to get created in the
/Users/username/Library/Caches/Safari/... folders. This is where the odd
behavior started. Even though the contents of
the /Users/username/Library/Caches/Safari/... folders was changing, the
iteration of the root of the volume (MacintoshHD) was getting messed up.
When iterating the contents of the root of MacintoshHD (the
startup volume), the maximumObjects requested in the call to FSGetCatalogInfoBulk was 22. There
were 51 items in the root folder. The first time through the loop, 22
items were retrieved with the FSGetCatalogInfoBulk call.
The second time through the loop, 22 items were also retrieved. However,
the first 2 items retrieved where the same as the last 2 items from the
previous batch. The last time through the loop, 7 items were
retrieved. The total items retrieved was 51, but since 2 items were
retrieved twice, the final 2 items were skipped. My understanding is that
this can happen if the contents of the container being iterated over
change. However, the root folder contents are not changing. The
contents of a folder several levels down are changing. I don’t see
why this should affect the root folder iteration.
I could get this behavior 50-60% of the time by using Safari while
a tree scan was being performed. This wasn't ideal, so I set up a script
to simulate the cache files getting created. I created a Test subfolder
in /Users/username/Library/Caches/Safari/ and used Applescript plus
Automator to copy a new file into that folder every 5 seconds. Initially,
this gave me the same behavior as above. I'm not sure why, but the
behavior then changed for some reason.
Issue 2:
I got the behavior I'm about to describe in my real app, but to
isolate the problem, I created a small test app to reproduce the
problem. This app recursively walks the directory tree starting at the
/Users/username/Library/Caches/ folder. This occurs inside a loop to get
it to walk that small branch 100's of times. While this app runs, the
script to copy files to the /Users/username/Library/Caches/Safari/Test folder
also gets run. It takes a different number of times through the loop each
time, but I see the same behavior every time: the iteration of the /Users/username/Library/Caches/
folder gets messed up. The first 22 items are found correctly. For
the next batch, FSGetCatalogInfoBulk says that there are 15 items. However, there are only 14
items left in that directory. I am retrieving arrays of FSRef,
FSCatalogInfo and file name objects through the FSGetCatalogInfoBulk call. When the
iteration gets messed up, it is always the same. The 13th
element of the FSRef array has an invalid FSRef object and the 13th element in
the file names array is "TemporaryItems" every time. Again, the
folder two levels down from this one is being updated, but the iteration of the
Caches folder gets messed up.
I made a different version of this app that uses scandir instead
of FSGetCatalogInfoBulk. I get the same behavior.
This seems to have something to do with the Caches folder. I
made a copy of the Caches folder and put it somewhere else on disk. I
updated my script to copy files to my Test subfolder of that Caches folder copy
instead. I updated the test app to walk the tree starting at that copy of
the Caches folder. I did not see the same behavior.
Is there something special about the Caches folder? Should I
avoid it when walking the tree? Could that also explain the behavior of
issue 1?
I can post my test code if necessary. I will need to clean
it up a little first.
Thanks in advance for any help!