Mailing Lists: Apple Mailing Lists
 Image of Mac OS face in stamp
Re: Min HFS+ size 512 bytes vs standard 4kb size
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Min HFS+ size 512 bytes vs standard 4kb size

In article
<email@hidden>, Shawn
Erickson <email@hidden> wrote:

> On Saturday, May 31, 2003, at 02:22 PM, Dieder Bylsma wrote:
> > What I remember was that in the days of hard drives being somewhat
> > more expensive and somewhat less expansive than today, that it was
> > suggested that we could maximize our HFS+ drives' usage by formatting
> > the drives with a allocation block size of 512 bytes instead of the
> > standard OS X install and Apple HFS+ default of 4kb.

> I don't think this has some much to do with I/O optimization as with
> file system optimizations / table sizes. VFS will attempt to buffer and
> coalesce I/O requests into larger requests, ones the better match that
> capabilities of the I/O "channel" being used. This is independent of
> the HFS+ allocation block size.

Not entirely independent. Parts of the disk I/O subsystem use a signed
32-bit block number. For volumes over 1 terabyte, we have to tell the
disk drivers to consider a block as being bigger than 512 bytes; we
currently use 4K. All I/O needs to operate in multiples of this device
block size. We won't mount a volume whose allocation block size is
smaller than that artificial device block size (since one device block
could belong to more than one file at a time).

Also, the buffer cache and VM will end up trying to do I/O in whole
pages (so, some multiple of 4K). Smaller allocation block sizes can
result in a page being discontiguous on disk, or the physical end of
file not ending on a page boundary. These cases require special
handling, and can cause a large I/O to be broken into smaller I/Os
because of alignment.

> Basically the larger the allocation size the fewer block entries HFS+
> has to track (the faster it can de/allocate fie space);

True. The performance impact due to the size of the allocation bitmap
is much more noticable in Mac OS 9 than in Mac OS X.

By the way, larger bitmaps tend to impact allocation more than
deallocation because allocation may need to scan more of the bitmap
until it finds a hole.

> however, the
> larger allocation size implies the potentially for wasted space for
> files that are less then the allocation size.

Strictly speaking, that's true. But in practice, the absolute
difference in wasted space is almost always insignificant.

When we were originally trying to come up with default allocation block
sizes for HFS Plus (back in Mac OS 8.1), we studied the distribution of
file sizes on lots of existing drives. We gathered the logical file
sizes and then computed the physical sizes resulting from a variety of
allocation block sizes. It turns out that the "knee" of the curve was
about 4KB. Smaller allocation block sizes made almost no difference.

With the way Mac OS X uses files, the distribution of file sizes has
changed. I'll bet Mac OS X tends to have more very tiny files (though
I don't have specific numbers to verify). But I'll bet the difference
in wasted space is still in the noise (512 vs. 4KB allocation blocks).

One other effect is that the smaller the allocation block size, the
more likely a single file, or the volume's free space, will be
fragmented. Mac OS X's allocation policy is getting smarter, and is
better at keeping files contiguous. But as your disk starts to fill
up, it will eventually need to make files discontiguous (or perhaps
have to relocate existing files to make contiguous free space, which we
don't do).

> I personally do not know
> if 4KB is special cased in HFS+ to some end or not (I don't think it
> is).

The minimum node size for the Catalog B-tree is 4KB. The default in
Mac OS X is 8KB. The default node size for the Extents B-tree in Mac
OS X is 4KB. That means I/O to the B-trees is going to be in multiples
of 4KB. If your allocation block size is smaller than that, you may
end up with individual nodes being fragmented. In Mac OS X 10.0, a
volume with fragmented nodes will fail to mount. That was fixed in
later versions, but it is still a performance problem.

My recommendation is to just use the default allocation block size
unless you know in advance that you're going have so many small files
that they won't all fit with the default allocation block size.

-Mark
_______________________________________________
darwin-development mailing list | email@hidden
Help/Unsubscribe/Archives: http://www.lists.apple.com/mailman/listinfo/darwin-development
Do not post admin requests to the list. They will be ignored.


Visit the Apple Store online or at retail locations.
1-800-MY-APPLE

Contact Apple | Terms of Use | Privacy Policy

Copyright © 2011 Apple Inc. All rights reserved.