Tip: Your next XRaid may be an SSD
Tip: Your next XRaid may be an SSD
- Subject: Tip: Your next XRaid may be an SSD
- From: Pierce T.Wetter III <email@hidden>
- Date: Mon, 15 Dec 2008 13:52:46 -0700
So in an effort to get more performance out of FrontBase I've been
experimenting with different values of the RDD cache on FrontBase. I
thought I would pass along the results, as I figure there are probably
more people on this list using FrontBase then anywhere else.
Background Data Points:
1. Databases do mostly Random access.
2. FrontBase uses 4K block sizes internally.
3. Databases are primarily I/O limited. See #1. So writing a
database is really about managing I/O.
4. In FB, Varchars > 32 chars are stored in a spelling table, <=
32 are stored in the row. Chars switch at 64 chars.
5. When you execute a COMMIT, FrontBase won't return until it has
confirmed that it has written the data to disk.
6. Our database is currently about 40GB, so 99% of the RAID goes
on unused.
So I started with our "oldschema" which was badly designed in a
number of ways:
1. Varchars were often specified to be quite long, yet only a few
bytes were used.
2. Varchars were used for fixed-length strings.
3. No column matching was used on any strings, so things like
"ClassCode" would be independently stored for each row in certain ables.
One of the great things about databases is that they are basically
filters. SQL goes in one end and data gets written out the back end.
So for my benchmark, I took a backup of our database, and the
1,000,000 lines of logged SQL and fed it to FrontBase. FrontBase sent
me their ClientSimulator tool which can take a raw SQL log and split
it into a number of parallel executing streams, pretty much simulating
how it executed in production. I then measured how long it took to go
through those million lines of SQL on a single HDD:
Old Schema with RDD=512: 200 lines/second.
So that was my baseline. So the first thing I did was try increasing
the RDD cache:
Old Schema with RDD=1512: 224 lines/second
So a 10% gain for more memory. Next I tweaked the schema to handle
chars and varchars better, and to set column match appropriately:
New Schema with RDD=1512: 240 lines/second
To go above 2GB of RAM, I would need a 64-bit database. FrontBase is
working on such a beast, so I got a prerelease from them for testing.
New Schema, FB5, RDD=3500: 288 lines/second
New Schema, FB5, RDD=5500: 359 lines/second
New Schema, FB5, RDD=11000: 407 lines/second
So RAM really helps, while the schema tweaks helped some, during
this process I got a factor of 2x. I was then gearing up to work with
Apple to test their new Promise-based RAIDs at their developer center
when I started hearing good things about the Intel X25-E SSD drives.
So I got two to try out, set them up as a RAID-Concatenated and ran
my test:
New Schema, FB5, RDD=(various), 8198 lines/second.
That's a factor of 40x. Not only that, but the CPU ran at upwards of
160% during the test compared to the 4-16% I was getting typically. So
it was to some extent CPU bound. Changing the RDD size didn't make
much difference, and may actually have slowed things down.
Why this is so much faster:
First off, since I didn't have a "spare" XRaid lying around, I'm
comparing a single HDD to an SSD. In theory, the XRaid is going to be
faster. So I also did some basic I/O experiments with the SSDs and the
RAID. While I experimented with some exotic tests like iozone and
bonnie++, it turns out that the easiest was just good old XBench. So
here are the XBench stats:
Single HDD XRaid SSD
Sequential 4K Write 17.3 MB/s 192 MB/s 214 MB/s
Sequential 4K Read 41 MB/s 126 MB/s 197 MB/s
Random 4K Write 1 MB/s 28 MB/s 93 MB/s
Random 4K Read .6 MB/s 9.6 MB/s 16 MB/s
So the XRaid is 12x faster then a single drive on sequential writes.
Some of that is because the RAID isn't really writing, its really just
putting it into the RAIDs cache.
It's only about 3x faster on reads, as reads have to wait for the
disk to catch up.
For Random, which is what the database is mostly doing the single
hard disk get's creamed because it has to physically move the disk
head. The RAID does better, presumably because it may only have to
seek one drive, or perhaps because its caching. Again, reads are
slower then writes, because you can't fake that with cache really. The
RAID is actually much faster then the single drive supposedly, but its
hard to verify that because the RAID hardware might be caching the
test file the XRAID is using.
But if we look at the SSD numbers, they exceed all of the XRaid
numbers, and kick its ass on Random accesses.
So I'm seriously looking into switching from our XRaid to a set of
SSD drives in production. However, there are some challenges with this:
1. Intel only makes the SSD in 1.8 and 2.5" form factors. Which
means most rack mount raid enclosures can't hold the drives. IcyDock
makes an adaptor for $20. But there will be LOTS of wasted space.
Which is an issue later on.
2. Our existing XRaid is IDE, so I can't use the SSDs in there. :-(
3. Apple charges you for even empty drive sleds. :-(
4. SSDs are really, really fast. So the interface to them really
matters, especially in comparison to existing RAID interfaces:
FiberChannel: 400MB/s
SataII/eSata: 300MB/s
SSD: 200MB/s.
So that means typical raid solutions won't be able to keep up
with any sort of raid of SSDs. What you pretty much need is an eSata
connection to each drive.
5. If its not in a RAID, you don't have any fall back for disk
failure. That means either setting up a RAID configuration, or a
mirror. Yet, the SSDs are so fast, its not clear that doing so might
not CUT performance for anything but a mirror with a hardware RAID
card. RAIDs usually increase the block size, so mirrors might be better.
6. I can't tell if the MultiLane Infiniband connectors end up
cutting performance or not.
7. ZFS looks really, really cool for SSDs. RAID-Z might be a better
alternative then RAID. But in testing, I lost about 1/2 the
performance in Leopard when I went to ZFS (without the Z, I only have
2 drives).
So basically, I'm now trying to figure out the options vis-a-vis 3rd
party RAID/SATA II solutions. So far:
There's a company Addonics that makes a cool storage rack: http://www.addonics.com/products/raid_system/rack_overview.asp
For third-party RAID cards the RR4320 tops out at 1000MB/s, or about
4SSDs in parallel. Most others top out at 500MB/s. So its seems that
RAID 0+1 makes more sense then RAID 5/6.
Pierce
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (email@hidden)
Help/Unsubscribe/Update your Subscription:
This email sent to email@hidden