Building the 2012 AnandTech SMB / SOHO NAS Testbed
by Ganesh T S on September 5, 2012 6:00 PM EST- Posted in
- IT Computing
- Storage
- NAS
Memory
The eight DIMM slots in the Z9PE-D8 WS allow up to 64GB of DRAM. Intel NASPT (one of our anticipated VM workloads) requires a minimum of 1GB of DRAM and doesn’t really like 4GB (as it introduces caching effects and leads to anomalous benchmarking results). Even low end clients in SMB environments come with a minimum of 2GB of DRAM nowadays, and hence, we decided to go with 2GB of DRAM for each VM. As SMB NAS speeds approach 200 MBps, it is sometimes necessary to have data sources and sinks capable of handling such speeds for file transfer tests. One option would be to have a really fast SSD or SSDs in RAID0. However, this introduces unnecessary extra variables into the mix. A RAM disk makes for a better solution, and in our build, also acts as a common storage resource for multiple VMs.
Keeping the above factors in mind, we decided to max out the capabilities of the Z9PE-D8 WS by installing 64GB of DRAM. We used G.Skill’s RipjawsZ F3-12800CL10Q2-64GBZL (8Gx8) modules. This quad-channel DDR3 kit is rated for operation at 1.5V and 1600 MHz with a CAS latency of 10-10-10-30. For our application, no overclocking was necessary. The Z9PE-D8 WS BIOS actually set it to 1333 MHz by default. We did find the performance at that setting to be good enough for our workloads, but decided to run the DIMMs at the native setting in the final configuration. Even though G.Skill targets the X79 platform, we had no trouble using it with the Z9PE-D8 WS. The combination of high capacity and efficiency made the G.Skill RipjawsZ a good choice for our testbed.
Storage
The storage subsystem is one of the most important aspects in a build meant to host multiple VMs concurrently. In our earlier NAS testbed, we used to run our VMs off a Seagate 2TB hard drive which had the host OS and the VMs in separate partitions. However, this is not a suitable solution for running multiple VMs concurrently. Hence, we made a decision to devote one physical disk to each VM. Fortunately, the Z9PE-D8 WS had 14 SATA ports.
Our planned workload doesn’t involve the storage of massive media files or any other such data which calls for hard disk drives in the testbed. The only exception is the robocopy test where we transfer a Blu-ray folder structure (with a size of 10.7GB) to the NAS and back. It is quite easy to handle that with a RAM disk, and hence, we decided to go with an SSD-only build.
We decided to equip the testbed with a 128GB OCZ Vertex 4 SSD for the host OS (Windows Server 2008 R2) and devote one 64GB OCZ Vertex 4 SSD to each VM. During the initial installation, we found that Windows Server 2008 R2 recommended at least 80GB of disk space for the primary partition. With the necessity to store temporary files for 12 VMs at the same time, we would have run the host OS SSD very close to full capacity. To resolve this, we installed another 128GB SSD to store the VM data and other necessary softwares.
The 128GB OCZ Vertex 4 provides up to 560 MBps / 430 MBps sequential read / write performance, and 90K / 120K IOPS for 4KB random reads and writes. At idle, the power consumption is 1.3W and it shoots up to 2.5W when fully active. These numbers remain the same for the 64GB OCZ Vertex 4. However, the sequential read / write performance drops down to 460 MBps / 220 MBps and the IOPS for 4K random reads and writes come in at 70K / 85K. Our aim in going with an SSD-only build was to make sure that the system’s storage subsystem didn’t end up being a bottleneck for our VMs. The much lower power consumption (compared to several distinct hard disk drives) ends up being an added bonus.
Facebook
Twitter
RSS
74 Comments
View All Comments
dertechie - Wednesday, September 5, 2012 - link
That is a helluva test bench.I'd love to see what a HP N40L Microserver does with 4 disks in it if you throw that at it (use the on-motherboard USB port for the OS). It's certainly not a plug-and-play solution like most NAS boxes, but assuming the performance is there it should be a far more flexible one for the money if you throw a *nix based OS on it.
bsd228 - Wednesday, September 5, 2012 - link
I've taken advantage of the 5th internal port of the N36L to add an SSD that is used by ZFS for both read and write caching. Strictly speaking, mirrored write caches are advised, but it's connected to a UPS to eliminate much of that risk.I think HP has given us the perfect platform for low power, high performance with flexibility.
extide - Thursday, September 6, 2012 - link
Cache? or L2ARC?Mirrored Cache drives are NOT suggested for ZFS, but Mirrored L2ARC devices are.
coder543 - Wednesday, September 5, 2012 - link
running Windows Server.........ganeshts - Wednesday, September 5, 2012 - link
What alternatives do you have in mind?We needed a platform which was well supported by the motherboard. To tell the truth, I found Hyper-V and the virtualization infrrastructure to be really good and easy to use compared to VMWare's offerings.
ender8282 - Wednesday, September 5, 2012 - link
I assume coder543 was going for a Linux based host, and possibly Linux based clients as well. If you had gone with Linux you wouldn't have needed extra software for SSH or the ram disk. It even looks like IOMeter is supported for Linux. Had you gone that route you likely could have automated the whole task so that it was just a matter of typing go on the host and coming back hours later to collect the results. OTOH most of your audience is probably more likely to be using Windows clients so it probably makes more sense to provide information clearly relevant to the average reader.I found the article interesting. The one thing that I'd be curious about is whether or not there were any major performance differences using Samba/CIFS type shares vs NFS, or a mixture of the two.
I'd love to see more Linux coverage in general, but I respect that you know your audience and write the articles that they generally want to read.
Great Job keep it up!
Ratman6161 - Thursday, September 6, 2012 - link
I should run on that platform just great. On the other hand, when all is said and done, as nice as this setup is, to me it is basically a full blown server/virtualization platform; not really a "NAS" at all. I would typically think of a NAS as being a dedicated storage device - possibly used as an IScsi target with the brains of the operation living elsewhere.ganeshts - Thursday, September 6, 2012 - link
This is a testbed for evaluating NAS units, not a NAS. Not sure why readers are getting an impression that this is a NAS by itself.bsd228 - Wednesday, September 5, 2012 - link
Ganesh- I think this test bed sets up very well for testing the $500-1000 4 bay type NAS devices we've been seeing of late that could actually serve a small office. However, I'm less sure that it delivers meaningful data to the home crowd. Like with your SSD tests, I see a place for a "light" load versus the heavy. I think testing against 4 VMs with, for sake of example, the following load types would work:1- 2 VMs streaming video - 1 DVD, 1 H.264 HDTV - are there any interruptions?
2- 1 VM streaming audio off a mt-daapd (or actual itunes since you're using windows as the server) - again, is there any dropoffs.
3- same VM as #2 is also doing content creation - like importing 1000 RAW images into Lightroom using this storage space
4- last VM is copying large files (or small) to the storage server.
The Thecus 4800 should handle this with ease, but there are many cheaper solutions out there that may or may not meet this level of need. I got so tired of poorly performing consumer units that 4 years ago I switched to an AMD x2 4800 running solaris, and more recently to the HP36L and 40L. At $300 plus $60 for 8 gigs of ECC I think this is a better value than the Thecus for those who can run solaris or even Windows Home Server. You're not reliant on the release of modules to support a particular service.
Also, it seems that all of these benchmarks are based on SMB transfers. It's worth checking to see if nfs and iscsi performance (when made available by the NAS) shows different numbers. In the past, it certainly did, especially on the consumer devices where NFS smoked SMB1. But perhaps this is a moot point with SMB2/windows 7 where it seems like the NIC or the hard drives are the limiting factors, not the transfer protocol.
Rick83 - Thursday, September 6, 2012 - link
I agreem test the different protocols provided by the devices.iSCSI, SMB, NFS as well as the media streaming protocols, FTP and whatever else it offers.
If encrypted transfers are offered, test those as well (eg. sshfs / scp).
Additionally, have a look at one of the cluster-ssh solutions, that allows simultaneous connections/commands to all machines.