Re: Identify image file count in directory
Re: Identify image file count in directory
- Subject: Re: Identify image file count in directory
- From: Jens Alfke <email@hidden>
- Date: Sat, 14 Nov 2015 11:51:02 -0800
> On Nov 14, 2015, at 10:15 AM, Jonathan Taylor <email@hidden> wrote:
>
> Thanks for your reply. Do you know for sure that this should be significantly faster, or were you just guessing? You didn't say which particular API you had in mind, but some reading has led me to the fts APIs. These appear to be about 20% faster than NSFileManager (I specified FTS_NAMEONLY). Better than a kick in the teeth, but still seems to me like it might be possible to do it faster.
If you don’t need to traverse subdirectories, drop down a level to readdir. This appears to be the lowest-level public API for directory traversal.
> I measured about 2.5 seconds for a directory on a "Mac OS Extended (Journaled)" external USB3 hard disk containing 110,000 files with names around 15 characters long. I don't know much about how filesystems are structured, so this information may not be all in one place, but that should represent less than 1MB/s of data read in order to get a list of all the filenames in the directory.
Most filesystems get noticeably slower when working with directories containing large numbers of items. HFS+ does better than most, but I wouldn’t be surprised if a lot of the slowness you’re seeing is just due to filesystem overhead. If you can, try dividing up the files among multiple subdirectories and see if that helps. (For example, you could create a directory “A” for all the files whose names begin with “A”, etc.)
I don’t know exactly how HFS+ stores directory entries, but there’s significantly more metadata for a file than just its name, so the filesystem is definitely reading more than 15 bytes per file from disk. (For instance, it’s pretty likely that 256 bytes are reserved for the filename whether or not all of them are used. Then there are the created/modified/used dates, owner/group, and probably 8 bytes to store the sector # where the file itself begins.)
—Jens
_______________________________________________
Cocoa-dev mailing list (email@hidden)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden