{"id":584,"date":"2010-09-22T10:16:50","date_gmt":"2010-09-22T15:16:50","guid":{"rendered":"http:\/\/it.thelibrarie.com\/weblog\/?p=584"},"modified":"2010-12-16T11:34:50","modified_gmt":"2010-12-16T16:34:50","slug":"testing-raid-and-hd-speeds-under-nix","status":"publish","type":"post","link":"https:\/\/it.thelibrarie.com\/weblog\/2010\/09\/testing-raid-and-hd-speeds-under-nix\/","title":{"rendered":"Testing RAID and HD Speeds under *nix"},"content":{"rendered":"<p>I was given the task of purchasing a NAS\/SAN solution for a secondary (dev) oracle database (11g).  Pricing it out on the normal branded websites yielded roughly $12,000 to $30,000 depending on the options.  A quick look at newegg gave me a lot more for less coin.<\/p>\n<p>H55 Intel board, i3 2.93GHz dual core, 2GB DDR3 1333 dual channel, 6x 2TB 7200RPM SATA HDs in hot swapable cages, 4U case, Redundant 500Watt PSU, 8x RAID controller (SAS\/SATA), Intel 10\/100\/1000.  $2250.  We could technically buy 5 of these for the lowest price of a branded system.  But would it be fast enough?<\/p>\n<p>Linux has this command for finding disk speeds:<br \/>\n<code>\/sbin\/hdparm -t \/dev\/yourHDhere<\/code><br \/>\nAnd on my test system, here are the results:<\/p>\n<blockquote><p>Timing buffered disk reads:  202 MB in  3.01 seconds =  67.16 MB\/sec<\/p><\/blockquote>\n<p>You can find out the path of the HD you want by:<br \/>\n<code>df -h<\/code><\/p>\n<p>That&#8217;s on a pretty decent VMWare box.  On a real box, with much lower specs, the read times were abysmal:<\/p>\n<blockquote><p>Timing buffered disk reads:   10 MB in  3.32 seconds =   3.01 MB\/sec<\/p><\/blockquote>\n<p>I guess IDE HD&#8217;s on P4 1.8GHz systems are not the best&#8230;<br \/>\nTo find out your processor information:<br \/>\n<code>cat \/proc\/cpuinfo<\/code><\/p>\n<p>So how about on my NAS\/SAN solution?  Unfortunately the command \/sbin\/hdparm will not work as the OS on this machine is FreeBSD.  However, there is a similar command for BSD:<br \/>\n<code>diskinfo -t \/dev\/yourHDhere<\/code><br \/>\n<a href=\"http:\/\/it.thelibrarie.com\/weblog\/wp-content\/uploads\/2010\/09\/softwareraid5.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/it.thelibrarie.com\/weblog\/wp-content\/uploads\/2010\/09\/softwareraid5.png\" alt=\"\" title=\"softwareraid5\" width=\"619\" height=\"326\" class=\"alignnone size-full wp-image-585\" srcset=\"https:\/\/it.thelibrarie.com\/weblog\/wp-content\/uploads\/2010\/09\/softwareraid5.png 619w, https:\/\/it.thelibrarie.com\/weblog\/wp-content\/uploads\/2010\/09\/softwareraid5-300x157.png 300w\" sizes=\"auto, (max-width: 619px) 100vw, 619px\" \/><\/a><br \/>\nSo you can see that the average for software RAID5 is 190834KB\/sec (186.36MB\/sec).  That&#8217;s pretty stellar.<\/p>\n<p>Next I&#8217;ll be trying RAID5 on the hardware card, followed by RAID10 on that same card.  I&#8217;ll post the results as soon as I can.<\/p>\n<p>***EDIT***<\/p>\n<p>RAID5 on an Areca ARC-1222<br \/>\n466.74MB\/sec using the exact same hardware setup.  Next will be trying RAID1+0.<\/p>\n<p>RAID10 is actually slower.  It&#8217;s showing 319.70MB\/sec.  I&#8217;m going to try with the following commands from \/mnt\/raidcontainer:<br \/>\n<code>dd if=\/dev\/zero of=testfile bs=1M count=5000<\/code><br \/>\nand<br \/>\n<code>dd if=\/dev\/zero of=testfile bs=8K count=5000<\/code><br \/>\nto see what is up.<\/p>\n<p>254MB\/sec on 1MB writes over 5000 attempts.  I&#8217;m trying to find out how to not cache disk copies on the smaller file &#8211; it gave me over 1000MB\/sec&#8230; \ud83d\ude42<\/p>\n<p>***EDIT again***<br \/>\nOK, so FreeBSD doesn&#8217;t allow for the oflag on dd, so we have to use conv=sparse instead (and I changed to 50000):<br \/>\n<code>dd if=\/dev\/zero of=testfile bs=8K count=50000 conv=sparse<\/code><br \/>\nWhich gives me 265MB\/sec using 8K files.  Not a bad idea of the range.  I think that I must go back to HW RAID5 for a complete list of read and writes.<\/p>\n<p>***EDIT 4***<br \/>\nI built another FreeNAS system (0.7.2.5543 x64) with the following specs:<br \/>\nH55 Intel board, i3 3.06GHz dual core, 4GB DDR3 1333 dual channel, 8x 2TB 7200RPM SATA HDs in hot swapable cages, 4U case, Redundant 658Watt PSU, 8x RAID controller (areca arc-1222) with battery backup cache, Intel 10\/100\/1000, dummy gig switch with jumbo frame support.  $2800.<br \/>\n<code>dd if=\/dev\/zero of=testfile bs=8K count=500000 conv=sparse<\/code><br \/>\n<a href=\"http:\/\/it.thelibrarie.com\/weblog\/wp-content\/uploads\/2010\/09\/freenas3.png\" target=\"_blank\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/it.thelibrarie.com\/weblog\/wp-content\/uploads\/2010\/09\/freenas3.png\" alt=\"\" title=\"freenas3\" width=\"645\" height=\"95\" class=\"alignnone size-full wp-image-648\" srcset=\"https:\/\/it.thelibrarie.com\/weblog\/wp-content\/uploads\/2010\/09\/freenas3.png 645w, https:\/\/it.thelibrarie.com\/weblog\/wp-content\/uploads\/2010\/09\/freenas3-300x44.png 300w\" sizes=\"auto, (max-width: 645px) 100vw, 645px\" \/><\/a><br \/>\n340MB\/sec!  Although the older rig averaged 44MB\/sec per drive, this one is at 42.5MB\/sec per drive.  After this I think the primary issue will be the raid controller as a bottleneck.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I was given the task of purchasing a NAS\/SAN solution for a secondary (dev) oracle database (11g). Pricing it out on the normal branded websites yielded roughly $12,000 to $30,000 depending on the options. A quick look at newegg gave me a lot more for less coin. H55 Intel board, i3 2.93GHz dual core, 2GB &hellip; <a href=\"https:\/\/it.thelibrarie.com\/weblog\/2010\/09\/testing-raid-and-hd-speeds-under-nix\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Testing RAID and HD Speeds under *nix<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[],"class_list":["post-584","post","type-post","status-publish","format-standard","hentry","category-linux"],"_links":{"self":[{"href":"https:\/\/it.thelibrarie.com\/weblog\/wp-json\/wp\/v2\/posts\/584","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/it.thelibrarie.com\/weblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/it.thelibrarie.com\/weblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/it.thelibrarie.com\/weblog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/it.thelibrarie.com\/weblog\/wp-json\/wp\/v2\/comments?post=584"}],"version-history":[{"count":8,"href":"https:\/\/it.thelibrarie.com\/weblog\/wp-json\/wp\/v2\/posts\/584\/revisions"}],"predecessor-version":[{"id":650,"href":"https:\/\/it.thelibrarie.com\/weblog\/wp-json\/wp\/v2\/posts\/584\/revisions\/650"}],"wp:attachment":[{"href":"https:\/\/it.thelibrarie.com\/weblog\/wp-json\/wp\/v2\/media?parent=584"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/it.thelibrarie.com\/weblog\/wp-json\/wp\/v2\/categories?post=584"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/it.thelibrarie.com\/weblog\/wp-json\/wp\/v2\/tags?post=584"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}