Original Link: http://www.anandtech.com/show/8298/qnap-ts451-bay-trail-nas-performance-review
QNAP TS-451 Bay Trail NAS Performance Reviewby Ganesh T S on July 28, 2014 9:00 AM EST
Introduction and Testbed Setup
The launch of the QNAP TS-x51 series was covered in detail last month. Its introduction has revitalized the premium NAS market for SOHO and power users by providing a powerful enough alternative to the Atom D270x-based NAS units. The 22nm Celeron J1800 in the TS-x51 is a SoC (obviates the necessity for a platform controller hub) and brings a revamped Atom microarchitecture (Silvermont) to the NAS market. QNAP is, to our knowledge, the first off-the-shelf NAS vendor to bring a Bay Trail-based NAS unit to the market. The Celeron J1800 is also one of the few Bay Trail parts to come with the Intel Quick Sync transcoder engine as well as VT-x capabilities. QNAP takes advantage of both in their firmware to provide hardware transcoding capabilities (both offline and real-time) as well as support for virtual machines (i.e, their OS, QTS, can act as a host OS).
The virtualization and multimedia capabilities of the firmware deserve detailed analysis and will not be part of this review. Instead, we will solely concentrate on performance numbers under various scenarios. We have already looked into the market that QNAP is trying to target with this lineup in our launch piece. So, without further digression, let us take a look at the specifications of our TS-451 review unit.
|QNAP TS-451-4G Review Unit Specifications|
|Processor||Intel Celeron J1800 (2C/2T @ 2.41 GHz)|
|RAM||4 GB DDR3L RAM|
|Drive Bays||4x 3.5"/2.5" SATA 6 Gbps HDD / SSD (Hot-Swappable)|
|Network Links||2x 1 GbE|
|External I/O Peripherals||2x USB 3.0, 2x USB 2.0|
|VGA / Display Out||HDMI 1.4a|
|Full Specifications Link||QNAP TS-451 Specifications|
Note that the $759 price point reflects the additional 3 GB of RAM over the baseline 1 GB model (which will retail for $700).
The TS-451 runs Linux (kernel version 3.12.6). Other aspects of the platform can be gleaned by accessing the unit over SSH.
Testbed Setup and Testing Methodology
The QNAP TS-451 can take up to four drives. Users can opt for either JBOD, RAID 0, RAID 1, RAID 5, RAID 6 or RAID 10 configurations. We benchmarked the unit in RAID 5 with four Western Digital WD4000FYYZ RE drives as the test disks. Our testbed configuration is outlined below.
|AnandTech NAS Testbed Configuration|
|Motherboard||Asus Z9PE-D8 WS Dual LGA2011 SSI-EEB|
|CPU||2 x Intel Xeon E5-2630L|
|Coolers||2 x Dynatron R17|
|Memory||G.Skill RipjawsZ F3-12800CL10Q2-64GBZL (8x8GB) CAS 10-10-10-30|
|OS Drive||OCZ Technology Vertex 4 128GB|
|Secondary Drive||OCZ Technology Vertex 4 128GB|
|Tertiary Drive||OCZ Z-Drive R4 CM88 (1.6TB PCIe SSD)|
|Other Drives||12 x OCZ Technology Vertex 4 64GB (Offline in the Host OS)|
|Network Cards||6 x Intel ESA I-340 Quad-GbE Port Network Adapter|
|Chassis||SilverStoneTek Raven RV03|
|PSU||SilverStoneTek Strider Plus Gold Evolution 850W|
|OS||Windows Server 2008 R2|
|Network Switch||Netgear ProSafe GSM7352S-200|
We thank the following companies for helping us out with our NAS testbed:
- Thanks to Intel for the Xeon E5-2630L CPUs and the ESA I-340 quad port network adapters
- Thanks to Asus for the Z9PE-D8 WS dual LGA 2011 workstation motherboard
- Thanks to Dynatron for the R17 coolers
- Thanks to G.Skill for the RipjawsZ 64GB DDR3 DRAM kit
- Thanks to OCZ Technology for the two 128GB Vertex 4 SSDs, twelve 64GB Vertex 4 SSDs and the OCZ Z-Drive R4 CM88
- Thanks to SilverStone for the Raven RV03 chassis and the 850W Strider Gold Evolution PSU
- Thanks to Netgear for the ProSafe GSM7352S-200 L3 48-port Gigabit Switch with 10 GbE capabilities.
- Thanks to Western Digital for the four WD RE hard drives (WD4000FYYZ) to use in the NAS under test.
Hardware Platform and Setup Impressions
The industrial design of the QNAP TS-451 is quite utilitarian. Despite the metal chassis, the drive caddies are themselves made of plastic and feel a bit more flimsy that what we would like. At the price point that QNAP wants to place the product, consumers would be looking for a premium product with proper metal caddies (like the ones that come along with the TS-x70 and the rackmount units). Apart from the main unit, the package consists of the following:
- 2M Cat 5E Ethernet cable
- 90 W external power supply with US power cord
- Getting started guide / warranty card
- Screws for hard drive installation
In terms of chassis I/O, we have a USB 3.0 port in front (beneath the power and backup buttons). On the rear side, we have the power inlet, a USB 3.0 port, two USB 2.0 ports, two GbE ports and a HDMI port. Since we are in the middle of a long-term evaluation (for the virtualization and multimedia capabilities), a teardown hasn't been performed yet, but Legion Hardware disassembled the unit and found two ASMedia ASM1061 SATA to PCIe bridges as well as an Etron EJ168A USB 3.0 host controller (two-port hub chip).
The various components of a Bay Trail-D part (the family to which the Celeron J1800 belongs) are provided in the diagram below.
Obviously, two cores are cut, as are a number of miscellaneous ports, in the Celeron part we are looking at.
As we already discussed in the launch coverage, the USB 3.0 port is connected the upstream port of the Etron EJ168A, while two PCIe 2.0 x1 lanes are connected to the two ASMedia ASM1061 ports. From Legion Hardware's disassembly, the other two PCIe 2.0 x1 lanes are connected to two Intel i210 Ethernet controllers.
Setup and Usage
QNAP's QTS is one of the more full-features NAS operating systems that we have seen from off-the-shelf NAS vendors. A diskless unit can be set up in three ways - the first one is to use QNAP's cloud service (at start.qnap.com) and enter the Cloud ID that comes in the getting started guide. The second one is to use QNAP's QFinder utility and set up the unit through that. The third one is to somehow determine the DHCP IP received by the unit and access the unit directly over the web browser. We chose the second option to get things up and running.
In terms of usage, the web interface allows multi-tasking and provides a desktop environment within the browser. It is a cross between a mobile OS-type app layout and a traditional desktop environment. From our experience, even though the features are awesome, we did find the UI responsiveness to be a bit on the slower side compared to, say, Asustor or Synology. Some of the relevant features are exposed in the gallery below.
We have not dealt with higher-level applications and the mobile app ecosystem in the above gallery. A discussion of those will be made in the upcoming coverage of the virtualization and multimedia capabilities.
The NAS's primary purpose is, of course, the handling of the storage aspects - RAID setup, migration and expansion. Our full test process of starting with one drive, migrating to RAID-1, adding another drive to migrate to RAID-5 and yet another one to expand the RAID-5 volume using a total of 4x 4 TB WD Re drives successfully completed with no issues whatsoever.
We simulated drive failure by yanking out one of the drives during data transfer. The operations from the client didn't face any hiccups, but the NAS UI immediately reported the trouble (alerts can be configured). Inserting a new drive allowed for rebuild. There was a bit of an issue with the NAS not allowing for the hot-swap because of some pre-existing partitions on the hard drive that was inserted as new, but the issue couldn't be reliably reproduced. QNAP suggested the use of drives free of partitions for the empty bays / replacements for reliable expansions / rebuilds.
Single Client Performance - CIFS on Windows
The single client CIFS and iSCSI performance of the QNAP TS-451 was evaluated on the Windows platforms using Intel NASPT and our standard robocopy benchmark. This was run from one of the virtual machines in our NAS testbed. All data for the robocopy benchmark on the client side was put in a RAM disk (created using OSFMount) to ensure that the client's storage system shortcomings wouldn't affect the benchmark results. It must be noted that all the shares / iSCSI LUNs are created in a RAID-5 volume. The TS-451 manages to compare favourably against every other 4-bay NAS unit that we have evaluated with our new methodology. Benchmark numbers are provided in the graphs below.
Single Client Performance - iSCSI on Windows
We created a 250 GB iSCSI target and mapped it on the Windows VM. The same benchmarks were run and the results are presented below. The observations we had in the CIFS section hold true here too.
Linux Client Performance - CIFS and NFS
A CentOS 6.2 virtual machine was used to evaluate NFS and CIFS performance of the NAS when accessed from a Linux client. We chose IOZone as the benchmark for this case. In order to standardize the testing across multiple NAS units, we mount the CIFS and NFS shares during startup with the following /etc/fstab entries.
//<NAS_IP>/PATH_TO_SMB_SHARE /PATH_TO_LOCAL_MOUNT_FOLDER cifs rw,username=guest,password= 0 0
<NAS_IP>:/PATH_TO_NFS_SHARE /PATH_TO_LOCAL_MOUNT_FOLDER nfs rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=2, sec=sys,mountaddr <NAS_IP>,mountvers=3,mountproto=udp,local_lock=none,addr=<NAS_IP> 0 0
The following IOZone command was used to benchmark the CIFS share:
IOZone -aczR -g 2097152 -U /PATH_TO_LOCAL_CIFS_MOUNT -f /PATH_TO_LOCAL_CIFS_MOUNT/testfile -b <NAS_NAME>_CIFS_EXCEL_BIN.xls > <NAS_NAME>_CIFS_CSV.csv
IOZone provides benchmark numbers for a multitude of access scenarios with varying file sizes and record lengths. Some of these are very susceptible to caching effects on the client side. This is evident in some of the graphs in the gallery below.
Readers interested in the hard numbers can refer to the CSV program output here.
The NFS share was also benchmarked in a similar manner with the following command:
IOZone -aczR -g 2097152 -U /nfs_test_mount/ -f /nfs_test_mount/testfile -b <NAS_NAME>_NFS_EXCEL_BIN.xls > <NAS_NAME>_NFS_CSV.csv
The IOZone CSV output can be found here for those interested in the exact numbers.
A summary of the bandwidth numbers for various tests averaged across all file and record sizes is provided in the table below. As noted previously, some of these numbers are skewed by caching effects. A reference to the actual CSV outputs linked above make the entries affected by this effect obvious.
|QNAP TS-451 - Linux Client Performance (MBps)|
|*: Number skewed by caching effect|
Encryption Support Evaluation
Consumers looking for encryption capabilities can opt to encrypt a iSCSI share with TrueCrypt or some in-built encryption mechanism in the client OS. However, if requirements dictate that the data must be shared across multiple users / computers, relying on encryption in the NAS is the best way to move forward. Most NAS vendors use the industry-standard 256-bit AES encryption algorithm. One approach is to encrypt only a particular shared folder while the other approach is to encrypt the full volume. QNAP supports only volume-level encryption for now in QTS.
On the hardware side, encryption support can be in the form of specialized hardware blocks in the SoC (common in ARM / PowerPC based NAS units). In x86-based systems, accelerated encryption support is dependent on whether the AES-NI instruction is available on the host CPU. The Celeron J1800 doesn't have AES-NI capabilities, but the CPU does have some more grunt compared to the previous generation Atoms. The following graphs show its effectiveness.
The encryption-enabled performance numbers of the TS-451 simply blow it past the other contenders. If the absence of AES-NI can still result in this good a performance with a RAID-5 volume, we are left wondering what QNAP could do with AES-NI acceleration in Avoton / Rangeley.
Multi-Client Performance - CIFS on Windows
We put the QNAP TS-451 through some IOMeter tests with a CIFS share being accessed from up to 25 VMs simultaneously. The following four graphs show the total available bandwidth and the average response time while being subject to different types of workloads through IOMeter. The tool also reports various other metrics of interest such as maximum response time, read and write IOPS, separate read and write bandwidth figures etc. Some of the interesting aspects from our IOMeter benchmarking run can be found here. Since we have two network links, they can be teamed in 802.3ad dynamic link aggregation mode.
The real competition here is the Thecus N4800, which is based on the previous-generation Atom D2700. While the QNAP TS-451 wins out in some of the multi-client benchmarks (in terms of both total bandwidth as well as average response time), the N4800 does manage to win a few too. Ultimately, it is the target market (home users) as well as the power consumption numbers (which we shall see in the next section) that tend to make the Bay Trail platform an attractive upgrade option for NAS owners with previous-generation Atom-based units.
Miscellaneous Aspects and Concluding Remarks
It is expected that most users would configure the QNAP TS-451 in RAID-5 for optimal balance of redundancy and capacity. Hence, we performed all our expansion / rebuild testing as well as power consumption evaluation with the unit configured in RAID-5. The disks used for benchmarking (Western Digital WD4000FYYZ) were also used in this section. The table below presents the average power consumption of the unit as well as time taken for various RAID-related activities.
|QNAP TS-451-4G RAID Expansion and Rebuild / Power Consumption|
|Single Disk Init (4TB in JBOD)||-||19.47 W|
|4 TB JBOD (1D) to 4 TB RAID-1 (2D)||8h 42m 12s||31.73 W|
|4 TB RAID-1 (2D) to 8 TB RAID-5 (3D)||24h 58m 25s||42.36 W|
|8 TB RAID-5 (3D) to 12 TB RAID-5 (4D)||26h 25m 47s||52.84 W|
|12 TB RAID-5 Rebuild (4D)||9h 14m 48s||54.37 W|
Similar to the Seagate NAS 4-bay we saw last week, the rebuild process takes much less time compared to the RAID expansion process. The time taken for the various RAID modifications are amongst the lowest of all the four-bay NAS units that we have evaluated so far. Starting with the next review, we will have comparison graphs for these aspects.
QNAP also enables access to the NAS over the Internet through automatic port forwarding (UPnP). There is also a Cloud Link beta service which utilizes relay servers operated by QNAP to achieve the same functionality without the port forwarding setup. Our experience indicated that the beta tag for Cloud Link is not unwarranted. The myQnapCloud.com service also ties in with the Qfile mobile app for access to the NAS contents over the Internet from a mobile device.
The differentiating aspects of the TS-x51 series are obviously the virtualization and transcoding features. In our evaluation so far, the QEMU-based Virtualization Station app works really great for the average consumer (though power users coming off a Hyper-V or VMWare background may miss some features that they take for granted). The hardware transcoding features have not been evaluated in full depth yet, but, in our limited experience, there are still quite a few rough edges - we can't say for sure yet whether it is substantially better than the Intel Evansport-based Synology DS214play. Based on paper specifications, it should definitely be. Hopefully, by the time we get to detailed coverage in a month or two, we can decide one way or the other.
Coming to the business end of the review, we find that QNAP's choice of Bay Trail for this market segment makes quite a bit of sense. For the SMB and high-end SOHO markets, the Intel Avoton and Rangeley platforms are the best bet. For home consumers and power users looking for a media server solution with mobile app support and ability to take advantage of Quick Sync, NAS units based on Bay Trail parts such as the Celeron J1800 make more sense. With the TS-451, QNAP has delivered a NAS that can handle a large number of simultaneous connections without drop in performance, a pre-requisite for a powerful media server. It also ticks all the boxes for a full-featured solution.
We do have use-cases in mind for evaluating the virtualization and transcoding features of the TS-451 for the upcoming articles. That said, it would be interesting to see what readers want to see evaluated when it comes to running virtual machines on a NAS, as well as what they would like to see done for evaluation of the real-time transcoding features. The feedback would be of great aid in shaping up the second and third parts of our TS-451 review.