Testbed Setup and Testing Methodology

The rising popularity of USB 3.1 (both Gen 1 and Gen 2) Type-C direct-attached storage (DAS) devices and the upcoming Thunderbolt 3 DAS units made it clear that I had to work on some updates to our direct-attached storage testbed. Originally based on the Haswell platform, the DAS testbed used a Thunderbolt 2 PCIe add-on card and the USB 3.0 ports hanging off the PCH. For a brief while, I also added USB 3.1 Gen 2 Type-A and Type-C PCIe cards to evaluate a few DAS units.

The introduction of Skylake has been quite interesting from the viewpoint of fast local storage. While the 100-series chipset doesn't have native USB 3.1 Gen 2 support, it does have plenty of high-speed PCIe 3.0 lanes that enable high-speed bridges to other protocols. Motherboard vendors have decided to enable USB 3.1 on entry-level boards with an ASMedia bridge chip. However, premium boards can be equipped with Intel's own Alpine Ridge controller. As mentioned in the previous section, Thunderbolt 3 and Intel's Apline Ridge are interesting for a few reasons:

  • In addition to Thunderbolt 3, Alpine Ridge also integrates a USB 3.1 Gen 2 (10 Gbps) host controller
  • Thunderbolt 3 works over a Type-C interface, and supports a couple of additional protocols - USB 3.1 Gen 2 and DisplayPort 1.2

Considering these aspects, it made sense to migrate to Skylake for our DAS testbed. In particular, I looked out for a board with Alpine Ridge integrated. Ian published the review of the GIGABYTE Z170X-UD5 TH, and it turned out that the board perfectly fit the requirements. Note the DisplayPort output and PCIe 3.0 x 4 lanes from the Z170 PCH getting into the Alpine Ridge controller before producing two Type-C ports that can be used as '2x Intel Thunderbolt 3' or '2x DisplayPort' or '2x USB Type-C'.

Intel provided us with a sample of the Core i5-6600K to use in the board. G.Skill also came forward with four 8GB DDR4 DIMMs to give the testbed 32GB of DRAM (the same as our Haswell-based testbed)

The Corsair Carbide Series Air 540 chassis in our Haswell-based testbed has been great in terms of footprint, ventilation and easy access to components. Two hot-swap internal SATA slots turned out to be a boon for quick secure erases of SSDs as well as benchmarking of internal HDDs meant for NAS usage in the single-disk mode. However, this unintended usage model (I wasn't planning on doing this frequently when I first opted for the Corsair Air 540) was a bit of a hassle, since one of the chassis sides had to be dismounted to access the hot-swap slots. I wanted to address this issue in the new testbed.

In the lookout for a ATX chassis for the new testbed, I had three main requirements:

  • Hot-swap bays accessible without the need to open up the unit (similar to the drive slots in hot-swap NAS units)
  • Portability in terms of being easy to shift from one location in the lab to another (something I realized as important when trying to test daisy chaining with a Thunderbot 2 DAS unit last year)
  • Cubical footprint with horizontal motherboard orientation in order to better fit in a workbench and enable easy swapping out of PCIe cards in the future

The Cooler Master HAF XB EVO perfectly fit our requirements. The two X-Dock bays fulfilled our need for hot-swap bays for both 3.5" and 2.5" drives. Since the unit is marketed as a LAN box, it has two rigid carry handles on the side panels to enable portability. The unit can also easily serve as a testbench. Only the top cover (held in place by two screws at the back) needs to be removed in order to access the PCIe cards. The PSU slot also extends slightly out, enabling easier cable management inside the chassis. With plenty of additional drive slots in addition to the X-Dock, it was a no-brainer to go with the Cooler Master HAF XB EVO.

We have traditionally gone with the chassis vendor for the PSU also in our testbeds. Cooler Master suggested the fully modular V750 for use in our system.

Even though a 750W PSU is an overkill for a system with no discrete GPUs, the rating makes sure that we have the option in the future. The fully modular nature also helped greatly in cable management.

In addition to the above, we made use of a few components that were salvaged from earlier reviews / unused components from previous builds - a Corsair Hydro Series H105 liquid CPU cooler, a Samsung SM951 NVMe PCIe 3.0 x4 SSD for the boot drive, and an Intel 730 series 480 GB SSD and a Corsair Neutron XT 480 GB SSD for use as staging drives for temporary data. The gallery below provides some more pictures from our build process.

Evaluation of DAS units (both Thunderbolt 3-based and USB 3.x-based ones) on Windows is being done with the testbed outlined in the table below.

AnandTech DAS Testbed Configuration
Motherboard GIGABYTE Z170X-UD5 TH ATX
CPU Intel Core i5-6600K
Memory G.Skill Ripjaws 4 F4-2133C15-8GRR
32 GB ( 4x 8GB)
DDR4-2133 @ 15-15-15-35
OS Drive Samsung SM951 MZVPV256 NVMe 256 GB
SATA Devices Corsair Neutron XT SSD 480 GB
Intel SSD 730 Series 480 GB
Add-on Card None
Chassis Cooler Master HAF XB EVO
PSU Cooler Master V750 750 W
OS Windows 10 Pro x64
Thanks to Cooler Master, GIGABYTE, G.Skill and Intel for the build components

Our direct-attached storage testing involves artificial benchmarks (ATTO and CrystalDiskMark) as well as real-world data transfer scenarios (photographs, videos and documents). In addition, we run the PCMark 8 Storage Bench for select multimedia editing workloads. Finally, for simultaneous multi-target testing (as in, multiple drives in a JBOD, or, two or more daisy-chained systems), we utilize Iometer to get an idea of the total performance.

The Nuts and Bolts of Thunderbolt 3 Direct-Attached Storage Performance
Comments Locked

60 Comments

View All Comments

  • ganeshts - Saturday, April 16, 2016 - link

    I would have agreed with you a year or so back, but the good thing here is that Thunderbolt has emerged successfully out of the Mac shadow. At present, every Thunderbolt 3 device out there is compatible with Windows only, and not Mac.

    From a system perspective, for a very little premium, board vendors can integrate USB 3.1 Gen 2 + Thunderbolt 3 into their notebooks and consumers can get more out of that integrated port compared to a pure USB 3.1 Gen 2 Type-C port. Whether that extra premium is worth it, the consumer should figure out. The board we used - GIGABYTE Z170 board with Alpine Ridge integrated - can be purchased for less than $200. I think at this price, compared to a vanilla system, consumers might be OK with something a bit more future-proof.

    From a device perspective, yes, lots of Thunderbolt peripherals are overpriced. But, eGFX solutions will definitely be attractive to a lot of gaming folks who are looking for portability. Then, there is Thunderbolt networking which is attractive for small workgroups (more of a business use-case there). Daisy chaining is another great feature for thin systems.

    Frankly, a year or two back, I wouldn't have been bullish on Thunderbolt, and would have equated it with Firewire. But, with Thunderbolt 3, I think Intel has finally converged on to the right type of solution.
  • theduckofdeath - Saturday, April 16, 2016 - link

    Mobile gaming with an external GPU will stay a niche, as the bottleneck will still be the poorly cooled mobile technology within the notebook. Especially with DX12 and the likes, processor and motherboard component performance will be a lot more relevant than it has been since Microsoft introduced the hardware extraction layer to Windows last decade.
    And it's not just "some" TB accessories that are overpriced for what you get. Like the previous commenter said, you can build an actual PC for the price of this external case. Use it for storage if you like, or whatever else you'd want to use a spare PC for.
  • bobj3832 - Saturday, April 23, 2016 - link

    I've worked in small offices with 6 people and huge corporations with over 100,000 employees. In every company we always had file servers on the network. Virtually no one kept data locally (except travelling sales people) If your workstation goes down just log in somewhere else and continue working.

    For my personal stuff at home I have a 10 gigabit network. The SATA interface (even for the SSDs I have in the file server) is the bottleneck.

    I just don't have any need for Thunderbolt with a 10G network.
  • bobj3832 - Saturday, April 23, 2016 - link

    And before someone complains of the cost I got used SFP+ cards for $20 each and a switch with 4 SFP+ ports for $520. It also has 24 1G ports.
  • JHBoricua - Saturday, April 23, 2016 - link

    Umm, and I had to log to the site to point how your comment on software raid was even more silly.
    What happens if the machine running a software RAID dies, you ask? Simple. Since it is software RAID, you simply mount the RAID volume on another PC running the same Os. That's the beauty of software RAID, it is not tied to the hardware.

    Let me ask you this. What happens when the failed device happens to be this enclosure with its 'hardware' RAID? Well, you better have a spare one of the same make/model lying around or you won't be able to recover the data. But hey, you'll probably be able to use the same cable.

    Seriously, did you even pause to think before commenting on the merits of software RAID vs this device?
  • damianrobertjones - Thursday, April 14, 2016 - link

    I recall, all those years ago, that USB was supposed to feature daisy chaining.
  • HideOut - Thursday, April 14, 2016 - link

    Never. But I think firewire was. USB has always been hub + spoke layout. FW could chain I think.
  • DanNeely - Thursday, April 14, 2016 - link

    You could fake daisy chaining in USB by connecting hubs to each other; and in theory there was no reason you couldn't make a device with a 2port hub that used 1 port internally and exposed the 2nd for daisy chain type usage. Aside from a few higher end keyboards I've never seen anything like this done.

    FW had direct proper support for daisy chaining; but neither of the FW HDD enclosures I owned supported it.
  • hrrmph - Friday, April 15, 2016 - link

    That's how most high-port-quantity USB hubs do it. They internally daisy chain. A "normal" USB hub is typically 4-ports. Use the last of those 4 ports to daisy chain to 4 more ports. The user sees a physical 7 ports on the outside of the hub.
  • Vidmo - Thursday, April 14, 2016 - link

    Waiting on the eight drive version to replace my current USB 3.0 local DAS backup solution. Backing up 36TB of data takes a long time over USB 3.0.

Log in

Don't have an account? Sign up now