I was given the task of purchasing a NAS/SAN solution for a secondary (dev) oracle database (11g). Pricing it out on the normal branded websites yielded roughly $12,000 to $30,000 depending on the options. A quick look at newegg gave me a lot more for less coin.

H55 Intel board, i3 2.93GHz dual core, 2GB DDR3 1333 dual channel, 6x 2TB 7200RPM SATA HDs in hot swapable cages, 4U case, Redundant 500Watt PSU, 8x RAID controller (SAS/SATA), Intel 10/100/1000. $2250. We could technically buy 5 of these for the lowest price of a branded system. But would it be fast enough?

Linux has this command for finding disk speeds:
/sbin/hdparm -t /dev/yourHDhere
And on my test system, here are the results:

Timing buffered disk reads: 202 MB in 3.01 seconds = 67.16 MB/sec

You can find out the path of the HD you want by:
df -h

That’s on a pretty decent VMWare box. On a real box, with much lower specs, the read times were abysmal:

Timing buffered disk reads: 10 MB in 3.32 seconds = 3.01 MB/sec

I guess IDE HD’s on P4 1.8GHz systems are not the best…
To find out your processor information:
cat /proc/cpuinfo

So how about on my NAS/SAN solution? Unfortunately the command /sbin/hdparm will not work as the OS on this machine is FreeBSD. However, there is a similar command for BSD:
diskinfo -t /dev/yourHDhere

So you can see that the average for software RAID5 is 190834KB/sec (186.36MB/sec). That’s pretty stellar.

Next I’ll be trying RAID5 on the hardware card, followed by RAID10 on that same card. I’ll post the results as soon as I can.

***EDIT***

RAID5 on an Areca ARC-1222
466.74MB/sec using the exact same hardware setup. Next will be trying RAID1+0.

RAID10 is actually slower. It’s showing 319.70MB/sec. I’m going to try with the following commands from /mnt/raidcontainer:
dd if=/dev/zero of=testfile bs=1M count=5000
and
dd if=/dev/zero of=testfile bs=8K count=5000
to see what is up.

254MB/sec on 1MB writes over 5000 attempts. I’m trying to find out how to not cache disk copies on the smaller file – it gave me over 1000MB/sec… 🙂

***EDIT again***
OK, so FreeBSD doesn’t allow for the oflag on dd, so we have to use conv=sparse instead (and I changed to 50000):
dd if=/dev/zero of=testfile bs=8K count=50000 conv=sparse
Which gives me 265MB/sec using 8K files. Not a bad idea of the range. I think that I must go back to HW RAID5 for a complete list of read and writes.

***EDIT 4***
I built another FreeNAS system (0.7.2.5543 x64) with the following specs:
H55 Intel board, i3 3.06GHz dual core, 4GB DDR3 1333 dual channel, 8x 2TB 7200RPM SATA HDs in hot swapable cages, 4U case, Redundant 658Watt PSU, 8x RAID controller (areca arc-1222) with battery backup cache, Intel 10/100/1000, dummy gig switch with jumbo frame support. $2800.
dd if=/dev/zero of=testfile bs=8K count=500000 conv=sparse

340MB/sec! Although the older rig averaged 44MB/sec per drive, this one is at 42.5MB/sec per drive. After this I think the primary issue will be the raid controller as a bottleneck.