Testbed Setup and Testing Methodology

The rising popularity of USB 3.1 (both Gen 1 and Gen 2) Type-C direct-attached storage (DAS) devices and the upcoming Thunderbolt 3 DAS units made it clear that I had to work on some updates to our direct-attached storage testbed. Originally based on the Haswell platform, the DAS testbed used a Thunderbolt 2 PCIe add-on card and the USB 3.0 ports hanging off the PCH. For a brief while, I also added USB 3.1 Gen 2 Type-A and Type-C PCIe cards to evaluate a few DAS units.

The introduction of Skylake has been quite interesting from the viewpoint of fast local storage. While the 100-series chipset doesn't have native USB 3.1 Gen 2 support, it does have plenty of high-speed PCIe 3.0 lanes that enable high-speed bridges to other protocols. Motherboard vendors have decided to enable USB 3.1 on entry-level boards with an ASMedia bridge chip. However, premium boards can be equipped with Intel's own Alpine Ridge controller. As mentioned in the previous section, Thunderbolt 3 and Intel's Apline Ridge are interesting for a few reasons:

  • In addition to Thunderbolt 3, Alpine Ridge also integrates a USB 3.1 Gen 2 (10 Gbps) host controller
  • Thunderbolt 3 works over a Type-C interface, and supports a couple of additional protocols - USB 3.1 Gen 2 and DisplayPort 1.2

Considering these aspects, it made sense to migrate to Skylake for our DAS testbed. In particular, I looked out for a board with Alpine Ridge integrated. Ian published the review of the GIGABYTE Z170X-UD5 TH, and it turned out that the board perfectly fit the requirements. Note the DisplayPort output and PCIe 3.0 x 4 lanes from the Z170 PCH getting into the Alpine Ridge controller before producing two Type-C ports that can be used as '2x Intel Thunderbolt 3' or '2x DisplayPort' or '2x USB Type-C'.

Intel provided us with a sample of the Core i5-6600K to use in the board. G.Skill also came forward with four 8GB DDR4 DIMMs to give the testbed 32GB of DRAM (the same as our Haswell-based testbed)

The Corsair Carbide Series Air 540 chassis in our Haswell-based testbed has been great in terms of footprint, ventilation and easy access to components. Two hot-swap internal SATA slots turned out to be a boon for quick secure erases of SSDs as well as benchmarking of internal HDDs meant for NAS usage in the single-disk mode. However, this unintended usage model (I wasn't planning on doing this frequently when I first opted for the Corsair Air 540) was a bit of a hassle, since one of the chassis sides had to be dismounted to access the hot-swap slots. I wanted to address this issue in the new testbed.

In the lookout for a ATX chassis for the new testbed, I had three main requirements:

  • Hot-swap bays accessible without the need to open up the unit (similar to the drive slots in hot-swap NAS units)
  • Portability in terms of being easy to shift from one location in the lab to another (something I realized as important when trying to test daisy chaining with a Thunderbot 2 DAS unit last year)
  • Cubical footprint with horizontal motherboard orientation in order to better fit in a workbench and enable easy swapping out of PCIe cards in the future

The Cooler Master HAF XB EVO perfectly fit our requirements. The two X-Dock bays fulfilled our need for hot-swap bays for both 3.5" and 2.5" drives. Since the unit is marketed as a LAN box, it has two rigid carry handles on the side panels to enable portability. The unit can also easily serve as a testbench. Only the top cover (held in place by two screws at the back) needs to be removed in order to access the PCIe cards. The PSU slot also extends slightly out, enabling easier cable management inside the chassis. With plenty of additional drive slots in addition to the X-Dock, it was a no-brainer to go with the Cooler Master HAF XB EVO.

We have traditionally gone with the chassis vendor for the PSU also in our testbeds. Cooler Master suggested the fully modular V750 for use in our system.

Even though a 750W PSU is an overkill for a system with no discrete GPUs, the rating makes sure that we have the option in the future. The fully modular nature also helped greatly in cable management.

In addition to the above, we made use of a few components that were salvaged from earlier reviews / unused components from previous builds - a Corsair Hydro Series H105 liquid CPU cooler, a Samsung SM951 NVMe PCIe 3.0 x4 SSD for the boot drive, and an Intel 730 series 480 GB SSD and a Corsair Neutron XT 480 GB SSD for use as staging drives for temporary data. The gallery below provides some more pictures from our build process.

Evaluation of DAS units (both Thunderbolt 3-based and USB 3.x-based ones) on Windows is being done with the testbed outlined in the table below.

AnandTech DAS Testbed Configuration
Motherboard GIGABYTE Z170X-UD5 TH ATX
CPU Intel Core i5-6600K
Memory G.Skill Ripjaws 4 F4-2133C15-8GRR
32 GB ( 4x 8GB)
DDR4-2133 @ 15-15-15-35
OS Drive Samsung SM951 MZVPV256 NVMe 256 GB
SATA Devices Corsair Neutron XT SSD 480 GB
Intel SSD 730 Series 480 GB
Add-on Card None
Chassis Cooler Master HAF XB EVO
PSU Cooler Master V750 750 W
OS Windows 10 Pro x64
Thanks to Cooler Master, GIGABYTE, G.Skill and Intel for the build components

Our direct-attached storage testing involves artificial benchmarks (ATTO and CrystalDiskMark) as well as real-world data transfer scenarios (photographs, videos and documents). In addition, we run the PCMark 8 Storage Bench for select multimedia editing workloads. Finally, for simultaneous multi-target testing (as in, multiple drives in a JBOD, or, two or more daisy-chained systems), we utilize Iometer to get an idea of the total performance.

The Nuts and Bolts of Thunderbolt 3 Direct-Attached Storage Performance
Comments Locked

60 Comments

View All Comments

  • name99 - Thursday, April 14, 2016 - link

    There is obviously a market for this sort of thing for SOME people, but if you're trying to save money, the way you do it is to use standard (high-performance) USB-3 cases and software RAID-1; there's no need to do that in the case.
    OSX and Linux obviously support RAID out of the box; I've no idea about Windows but I'd assume they're also on board. The only reason I can see that you need this hidden behind HW is if you need to move the device between different OS's.
  • danbob999 - Friday, April 15, 2016 - link

    well there is this one for $50:
    http://www.newegg.com/Product/Product.aspx?Item=9S...

    if you don't need a real case there are various dual dock for about $30. I also found dual 2.5" USB3 cases for $25.
  • Great_Scott - Thursday, April 14, 2016 - link

    At $100 over the price of a normal enclosure, no one that really needs the extra features this offers will care. Thus the item will stay niche and the price will stay high.

    This is self-reinforcing problem.

    TB is putting up a good fight, but the future doesn't look bright. No interface has ever survived higher prices by having better features. See: Microchannel, Token Ring, EISA, ATM, e-SATA and, yes, FireWire.
  • epobirs - Saturday, April 16, 2016 - link

    Look at the Cineraid portable models. I picked up one from Newegg a couple of years ago for about $20 because the things weren't selling. Either the target market didn't know they existed or just didn't see the value. Mine still sits unpopulated because I don't have any great need for a somewhat faster USB 3.0 drive, unless I'm getting the drives really cheap. At the time I bought the unit I was getting a lot of requests for laptop SSD upgrades. The customers would either get their original hard put in an enclosure or just let me keep it for a token amount. (I'd label it and put it in a drawer for a month or so, in case the SSD proved defective.)

    I expected I'd have a couple of drives to use or sell it to somebody converting two laptops but neither situation came along since obtaining the Cineraid.
  • HideOut - Thursday, April 14, 2016 - link

    The good news for us is that you will be asleep until hell freezes over. No more trolling from you.
  • danbob999 - Thursday, April 14, 2016 - link

    At $378 I might as well buy a PC... which will be able to hold at least 4 drives.
  • jbrizz - Thursday, April 14, 2016 - link

    People are going to plug these things into their Mac Pro, then do some 4k video editing on their 5k screen. You are not it's target audience (although you still shouldn't be so ignorant as to think that just because you don't want/need it no one else will).
  • danbob999 - Thursday, April 14, 2016 - link

    That's just one more reason why a Mac Pro is a crap PC. Such a professional PC should include room for at least 4 hard drives. Requiring an external $378 case just to get close to the same performance that you would get with internal drives is a joke.
  • apoctwist - Friday, April 15, 2016 - link

    I had to log into the site for the first time because of how silly this comment is. People working in the pro audio/pro video side are not going to rely on internal storage for their work or software RAID. First of all what happens when the machine you are using dies? What happens with your software RAID array on all those internal drives? With this device due to hardware RAID I can just plug in a cable on a new machine and I'm up and running in minutes. I don't have to worry about taking hard drives out of the PC, I don't have to worry about rebuilding the array (if that's even possible since you are using software RAID tied to you OS). All I have to worry about is a cable.

    That's why devices like these exist. In Pro video workflows external RAID arrays are common and encouraged. You see less heartache in the long run that way.

    As an audio professional I have all of my projects/audio/recording on an external TB raid array. if my machine dies tomorrow I can pick up where I left off the next day. I also have multiple machines for DAW work and I can just plug my TB cable to them and continue working on whatever project I need to with no worries.

    You are looking at a device like this from a consumer level but that's not what it's made for and the price tag is rather in line with what you will find out there for a TB enclosure.
  • theduckofdeath - Saturday, April 16, 2016 - link

    It might sound like harsh words, but, he is right. At the moment,. everything Mac related is stupid solutions to problems we didn't have. Thunderbolt has to die. It's a way to milk consumers for even more money on relatively limited sales.
    There are a ton of ways to make a better solution for your external storage you actually need. The first choice is exactly what he said, design the workstation case to allow for stortage expansion, instead of selling people an overpriced and under-powered garbage bin.
    There's a reason why companies like Oculus and HTC Vice refuse to support Mac OS these days. Because Mac devices are designed to work against the consumer from the foundation, Thunderbolt being one of the key culprits.

Log in

Don't have an account? Sign up now