Performance Benchmarks
Performance Benchmarks
Testing Methodology
Testing an external storage array is not as simple as putting together a bunch of files, dragging them onto the array's drive folder in Windows and using a stopwatch to time how long the transfer takes. Rather, there are factors such as read / write speed and data burst speed to take into account. To this end Testing will be split into two groups “real world testing” and “synthetic testing”. Real world testing will consist of read and write tests to not only the time it takes to move files to and fro the NAS but also to calculate approximately “real world” performance in MB/S. The synthetic benchmark will be IOzone. All tests were run 4 times and only best results are represented. Jumbo Packets (9000 MTU) was used for both the computer and the NAS during the testing.
For information purposes here is the theoretical maximum each network connection is capable of:
100mbit/s = 12,500 KB/s
1000mbit/s = 125,000 KB/s
Processor: Q6600 @ 3.2GHZ
Motherboard: Gigabyte p35 DS4
Memory: 4GB G.Skill PC2-6400
Graphics card: XFX 7200gt 128mb
Hard Drives: 1x Western Digital Se16 500GB
Power Supply: Seasonic S12 600W
Case: CM 690
Router: 5 port 10/100 D-Link 604 (used solely in these tests for DHCP)
Switch: 5 port Freedom9 freeConnect 500 10/100/1000 switch
Cable: Cat 5e shielded with max distance of 5ft runs
NAS Hard drives: 4 x Western Digital Se16 500GB
USB COPY Quick Test
In our real world testing using an OCZ ATV 2GB thumb drive capable of 25MB/secong read speeds it took the NAS 615 seconds to successfully read 1,974,000 KB of data from it to the NAS. This works out to a mediocre 3,210 KB/second.
IOzone Testing
The IOzone test is a much more thorough and exhaustive test of the network bandwidth and hard drive performance of a given NAS being tested. The true power of IOzone is that it can be configured a near endless variety of ways with the use of command line switches. These switches quite literally tell IOzone what it should use for its testing methodology. We have included a detailed break down of exactly what switches we used but to sum up the command that we have put together, IOzone will be reading and writing files varied from 64KB in small 4KB chunks up to 1GB in 16MB chunks and every combination in-between. This provides us with an extremely complete picture of how the device handles a variety of traffic over the network. The following two commands were used for the testing:
Full Area Graph:
IOzone -Rab 409RaidXXX.wks -i 0 -i 1 -g 1G -+u -f y:\IOzone\001.tmp
Simplified 2-D 64K Graphs
IOzone -Rab 409RaidXXX2D.wks -i 0 -i 1 -+u -q 64K -g 1G -f z:\test\001.tmp
-Rab
is 3 settings in one
R generate excel report
a auto mode 1(the “short” or quick tests mode)
b filename of excel spreadsheet to use
-g 1G sets max size to 1GB for auto mode
-+U Enable CPU utilization output (used to make sure OS caching is not happening)
-i options are tests to run
-i 0 = write/rewrite
-i 1 = read/re-read
-q 64= sets maximum record size (in Kbytes) for auto mode
When first presented with the output area graph of IOzone, people's first reaction is that of information overload (e.g. “pretty looking graph you have there….just one question…..what does it mean!?”). To make it easier to understand the results, and remove and CPU buffering and / or caching that usually occurs at the “peaks”, not only will the full area graph be included but that of a 2D streamlined version consisting of the 64K record size results at all recorded file sizes. 64k record size is has not been selected at random; rather under normal output this is the only record size that is complete across all file sizes and for many online NAS forums it is the defacto standard to use when comparing different NAS results.
READ
The read bandwidth is quite good and nicely shows how fast this unit can be for pushing data to a system. The majority of its read bandwidth is in the 40,000 -50,000 KB/s range with peaks hitting nearly 51,000 KB/s. These are very good performance numbers for any unit that costs less than $1,000 let alone a mere $600.
WRITE
As you can see from the above area graph the majority of the 409’s bandwidth is in the 20,000 to 25,000 KB/s range with peaks of nearly 27,600 KB/s. While these numbers are a little on the low side they do highlight how hard the Marvel processor has to work to write not only to multiple hard drives, but also to calculate parity for the array as well. Please do not get the wrong idea since these numbers are certainly respectable but for a RAID 5, 4 HDD array with Jumbo packets (9000 mtu) they are lower than one would like to see.
Simplified Read
For small to medium size files it appears that the different RAID levels do not significantly effect performance, as their transfer speeds are so similar as to be within normal tolerance / error range. This graph also does a great job of highlighting not only how good the TS 109 is, but how mediocre this unit’s performance is. When it comes to large files it apparently also does not matter what RAID level you chose since all perform below expectations.
Simplified Write
It is quite self-evident that RAID 5 is faster for small to medium size files. However, once the size of the files start to get big the Marvel processor just can not keep up causing RAID 5 speeds to plummet. Even RAID 1 suffers from this, just to a lesser extent. It is too bad that QNAP did not use either a more powerful single processor (or two low of these low power, low heat processors) as this unit has all the other ingredients of very good, high performance NAS. As it stands the 409 Pro Turbo is only a mediocre performer at best.
Real World Data Transfers
No matter how good a synthetic benchmark like IOzone is it can not really tell you how your NAS will perform in “real world” situations. All of us here at Hardware Canucks strive to give you the best, most complete picture of a review item’s true capabilities. To this end we will be running timed data transfers over a typical network to give you a general idea of how its performance relates to real life use. In general a NAS will be used on a day to day basis for either transferring large multi Gigabyte files (for example .ISO’s) or a random mix of small to medium files (for example .mp3 and album art). To help replicate these conditions we will be we will transfer a 4.32GB contiguous RAR file and a folder containing that includes 40 subfolders and over 4000 files varying in length from 8mb to 6kb (6.50 GB total). Testing will include transfer too and transferring from the devices, timing each process individually to provide an approximate Read and Write performance.
As you can see, even though the 409 has four times more hard drives and a new faster processor the older single bay TS-109 Pro handily beat it in all tests. While a few seconds one way or the other would not have been a concern, these numbers are significantly different and should give one a good idea of what to expect performance wise from this NAS.
Extended Runtime Testing
Where these units are marketed towards the S/MB environment, it is reasonable to expect them to be able to handle extended usage, with random multiple requests for data. To test how robust this unit is, and how well its cooling would work under a heavy workload, the QNAP 409 Pro Turbo was subjected to a 96 hour nonstop session. During this time the NAS was directed to not only create then fill and empty its RAID arrays but was also expected to handle not only multiple (and sometimes simultaneous) read and write requests from different computers on the network but also print multiple 100 page documents and download multiple CD size torrents (the latest Linux distros). At no time was the load less than 4 simultaneous process (1 bit torrent, 1 usb copy, 1 print job, 1 data request) and peaked at 9 (1 torrent, 2 print jobs, 3 requests for data, 2 requests to save data on the RAID arrays, 1 USB copy). For this test the RAID array was set up as two RAID 1 drives with four WD SE16 500 AAKS hard drives being used. For the print server portion of the testing one Canon i850 ink jet printer and one Lexmark 5450 inkjet printer was used.
After 96hrs of nonstop use the hard drives were running at 35*c, the Marvel processor was running at 47*C @ 100% usage and there was less than .5mb free of ram. Overall, it was very impressive to see the NAS not hang even once during those stressful hours. While yes, things to did slow down and the rate that they slowed down did vary from little (the printer's buffers seemed to help smooth things out and there appeared little to no performance loss there) to a significant slow downs (the torrent files did almost stop and were very slow during high demand periods). However, no matter what the demands put on this NAS it never once failed at any task. This NAS certainly deserves the moniker “Pro” and with its decent speeds even when placed under severe multitasking it also deserves its self appointed title of “Turbo”.