Disaster Recovery
Disaster Recovery
Testing Methodology
While a RAID Hard Drive Enclosure should be easy to use and have decent performance there is one area where it absolutely has to be 100% reliable and that is with its ability to recover from a dead hard drive (aka Disaster Recovery). Unfortunately, in the real world things are rarely as black and white as this and nothing should be taken for granted; especially when precious records are on the line. Even more basic is that while yes an enclosure should always recover from a bad drive, but what about its rebuild times? The amount of time it takes for an Enclosure to rebuild its array is very critical, as this is a time where it is most vulnerable to catastrophic loss of data. This test can therefore be considered a three part test.
The first part is PASS/FAIL and deals with did the RAID array work, is the array’s data still available and not corrupt when a hard drive “dies”.
The second part deals with how the array deals with replacement of the missing hard drive and is also a PASS/FAIL in that does the array start rebuilding or will it crash / hang? Does it require you to rebuild the array manually by first destroying all data on the array and starting from scratch (which is in our opinion unacceptable performance).
The third part (should it pass the first two parts) deals with how long it took for the array to get out of degraded mode and is also a PASS/FAIL. Regardless of performance we will inform you how long it takes but if its rebuild performance is less than 50 gigabytes an hour it is not fast enough to be considered effective.
To this end for each mode that the Enclosure offers that has disaster recovery, we start by first formatting and then filling them to the maximum capacity available in a given mode. After this was completed and we then verified that the data was indeed stored on the Enclosure before simulating a catastrophic failure. This failure was accomplished by removing one of the hard drives, while the unit is running (if it supports hot swapping, if it does not we power off the unit and remove the hard drive and then turn the unit back on). We then checked to see if the data was still available. If it is we then replace the “dead” hard drive and observe what the unit does with it. As soon as the hard drive is reinserted we start the timer and keep it running until the Enclosure informs us that it has finished its rebuild. All tests were run twice. All tests were not performed “one after the other”; rather they took place over a period of a couple of days and were widely spaced out so as to help reduce any possible chances of overloading the array and introducing any non repeatable errors. If for any reason, the array fails a section of the test. The test will be reran, if it fails a second time it will be considered a true FAIL, we will however inform you of any errors we get during testing even if the error did not reoccur.
JBOD Mode
In JBOD mode the enclosure hard drives will show up as 2 separate drives under USB but only one drive will be seen and be usable when using eSATA. This is because you need a port multiplier (sold separately) for you computer to be able to see and access more than one array per eSATA/SATA cable.
When USB was used and one of the hard drives was removed the data on that hard drive was gone, but all data on the other hard drive was still available. Conversely, with eSATA it all depends on which hard drive you remove, if you remove the one your computer can not see it does not appear to matter, as all data is still available from the single JBOD drive. If you yank the hard drive that the computer does see, the other JBOD array instantly appears. This can be confusing as it gives no indication that it is a different array.
While using USB, if the “dead” hard drive was reinstalled in the MobileSTOR the hard drive was instantly recognized and all data was still intact. If however, we replaced the “dead” hard drive with a freshly formatted drive (or just reformatted the dead one) all data was lost.
While using eSATA, if the “dead” hard drive was reinserted that array would instantly take over for the second JBOD array as if it had never be missing! This to us will just compound your confusion if suddenly your “new” data disappears and your old presumed missing data suddenly reappears!
Please Note: This is a common trait that many dual bay eSATA enclosures share and is more the fault of the eSATA standard than that of the manufacturer. For this reason this wonky behavior can not and will not be held against any enclosure that exhibits it. If we ever test an enclosure that does not exhibit this behavior and in fact has a built in port multiplier we will make sure to highlight this for you.
Overall this is a FAIL for all tests, as in the real world a dead hard drive would destroy all data. If, however you remove a good hard drive you can rest assured that you can simply reinsert it and everything will be fine when you reinsert it.
RAID 0 (BIG & FAST)
Spanning Mode (AKA “BIG”): The two hard drives will show up as one large single double capacity drive and data is written sequentially filling one hard drive before starting to write to the second drive.
RAID 0 Striping Mode (AKA “FAST”): The two hard drives will show up as one large single double capacity drive and read/write performance will be increased. This is because the array controller splits each piece of data across both HDDs in segments.
For both of the RAID 0 modes the results were the same. When a hard drive was removed from the enclosure the entire array became inaccessible (even in disk management it was not accessible; rather an 8GB unknown partition showed up but you could not format it nor assign a letter to it). When the hard drive was reinserted the array was instantly recognized and all data was not only safe but was also accessible. If the removed hard drive was first formatted the enclosure would also know this and start to rebuild the array which was darn near instantaneous for both modes, of course if Striping mode is used ALL your data on that array is gone but that is what happens when a zero redundancy RIAD level, but at least the new array is quickly available so you can get on with your post hard drive crash life. For SPANNING mode the data on the original drive (hard drive we left alone) is still there just any data from the replaced drive is gone. Either way, with these two modes expect data loss and plan accordingly (i.e. don't store anything that you think is important).
Just as with JBOD this is a FAIL for all sections as a dead hard drive will kill all your data, but once again if you remove a good hard drive you can rest assured that you can simply reinsert it and everything will be fine when you reinsert it.
RAID 1
In RAID 1 the enclosure's hard drives showed as one single drive. Initial array build time was about 2 and half hours.
When one of the hard drives was removed all data was still secure. The only noticeable difference was that the Alert LED went on. If we reinserted the “dead” hard drive the MobileSTOR was smart enough to recognize it and realize that no rebuild was necessary (recognize / quick test phase was about 5 seconds). However if we formatted second drive before insertion the enclosure was even smart enough to know that it needed to rebuild the array which it did so almost immediately (once again after 5 or so seconds). Just as when first created the Array rebuild time took about 2.5 hours to complete. Overall, this is amazingly good performance for an enclosure in this price range and can be considered an easy PASS.
SAFE33 & SAFE50
SAFE33 mode creates a secure RAID 1 array using 33% of the drives’ capacity. The remaining storage will be transformed into a quick RAID 0 (FAST) array, which works perfectly for fast, temporary work storage. As with JBOD you need a port multiplier to see both arrays when using eSATA, as only one RAID will appear. It is always the RAID 1 array but this negates the speed advantage this hybrid RAID level offers. When using USB, both arrays do appear and are accessible. Regardless of connection type, initial build takes about thirty to forty minutes longer than RAID 1 does.
SAFE50 mode creates a secure RAID 1 array using 50% of the drives’ capacity. The remaining storage will be transformed into a quick RAID 0 (FAST) array, which works perfectly for fast, temporary work storage. As with JBOD you need a port multiplier to see both arrays when using eSATA, as only one RAID will appear. It is always the RAID 1 array but this does negate and speed advantage this hybrid RAID level offers. When using USB both arrays do appear and are accessible. Regardless of connection type, and as with SAFE33, initial build takes about thirty to forty minutes longer than RAID 1 does.
The results for both Safe33 or Safe50 are exactly the same. The only difference is the amount of storage space the RAID 1 array has. When a hard drive was removed from the enclosure under USB the RIAD 0 “drive” in Windows disappeared, however all data on the RAID 1 array was still accessible and except for the Alert LED on the front of the enclosure glowing there was no visible indication that anything had changed. When the drive was reinserted the RAID 0 array drive reappeared and all data was fine. The enclosure was smart enough to recognize it and did not instigate a rebuild of the array.
When eSATA was used the enclosure acted the same way except that the RAID 0 array was never visible (as discussed earlier a port multiplier is needed). Regardless of connection type used, if the “dead” hard drive was formatted before reinsertion the array would recognize that a hard drive was insert and begin its rebuild process. Just as the initial building of the array took about 3.5 hours to complete, so did the rebuilding process.
Overall you can consider this a PASS on three tests for the RAID 1 array and for RAID 0 it is a FAIL for all tests (with the same caveats given with JBOD and normal RAID 0).
Disaster Recovery Conclusions
When you take all this data and assimilate it as a whole a few things to tend to stand out. The first and biggest of these is the fact that this is a super fast enclosure when it comes to building arrays. Heck, approximately 150GB an hour for RAID 1 rebuild speed is downright amazing.
The next thing that stands out is that this enclosure is not just fast; it is smart too! With a lot of enclosures (even ones costing a heck of lot more than this one does) once a hard drive is pulled and the array goes into failure mode and the only way out of it is to rebuild the entire array from scratch. This is a long, tedious process that leaves your data vulnerable for hours; whereas this unit is smart enough to not only know when you reinsert the same hard drive back in but knows when the data is still there. This makes this enclosure extremely user friendly and a lot more attractive than its competition.
The only thing that does stand out that is less than optimal is the fact that SAFE33 and SAFE50 are very gimmicky in this specific implementation. If you are going to advertise the benefits of these hybrid arrays you should at the very least include any and all necessary equipment to get the most from them. As it stands those two Hybrid arrays are only practical when the customer uses a USB connection.