Direct-Attached Storage Performance

Given the presence of two bays, it is clear that the system needs to be fitted with SSDs in order to deliver maximum performance. We chose two Intel DC S3510 240GB SSDs as the members of a RAID-0 array for benchmarking purposes. Prior to taking a look at the real-life benchmarks, we first check what ATTO and CrystalDiskMark have to report.

ATTO shows performance numbers close to 800 MBps for large-sized reads, and it is backed up somewhat by CrystalDiskMark also.

In order to tackle the first real-life use-case, we created three test folders with the following characteristics:

  • Photos: 15.6 GB collection of 4320 photos (RAW as well as JPEGs) in 61 sub-folders
  • Videos: 16.1 GB collection of 244 videos (MP4 as well as MOVs) in 6 sub-folders
  • BR: 10.7 GB Blu-ray folder structure of the IDT Benchmark Blu-ray (the same that we use in our robocopy tests for NAS systems)
Akitio Thunder3 Duo Pro + 2x Intel SSD DC S3510 (Thunderbolt 3)
robocopy Benchmarks (MBps)
  Write Bandwidth Read Bandwidth
Photos 526.00 398.80
Videos 522.75 538.06
Blu-ray Folder 525.68 568.55

The above benchmark run was also instrumented to record the temperature of one of the SSDs as well as the instantaneous transfer rate to the DAS unit during the process. The internal disk temperature never exceeded 38 C even after more than 127 GB of writes and 127 GB of reads continuously.

For the second use-case, we take advantage of PC Mark 8's storage bench. The storage workload involves games as well as multimedia editing applications. The command line version allows us to cherry-pick storage traces to run on a target drive. We chose the following traces.

  • Adobe Photoshop (Light)
  • Adobe Photoshop (Heavy)
  • Adobe After Effects
  • Adobe Illustrator

Usually, PC Mark 8 reports time to complete the trace, but the detailed log report has the read and write bandwidth figures which we present in our performance graphs. Note that the bandwidth number reported in the results don't involve idle time compression. Results might appear low, but that is part of the workload characteristic. This is not the intended use-case for portable hard drives, but the results are just presented here for the sake of completeness

Akitio Thunder3 Duo Pro + 2x Intel SSD DC S3510 (Thunderbolt 3)
PCMark8 Storage Benchmarks (MBps)
  Write Bandwidth Read Bandwidth
Adobe Photoshop (Light) 612.44 167.14
Adobe Photoshop (Heavy) 578.53 225.74
Adobe After Effects 265.09 227.90
Adobe Illustrator 488.68 217.72

While the above numbers are very good, and what one expects from two SSDs in RAID-0, they do not bring out the real capabilities of Thunderbolt 3. For that purpose, we need to daisy chain multiple units.

Prior to looking at the daisy chaining results, let us see how the unit performs when fitted out with a couple of high-capacity hard disks in RAID-0. This is probably the configuration that most users would choose when looking for large amounts of fast and affordable storage. Two 8TB Seagate Enterprise NAS HDDs were installed in one of the units and all benchmarks that were run for the SSD configuration were repeated.

Note that the HDD configuration provides lower bandwidth, but much higher consistency. There is also not much benefit to higher queue depths in this configuration. The consistency aspect is also brought out in the performance consistency test.

Akitio Thunder3 Duo Pro + 2x 8TB Seagate Enterprise NAS HDDs (Thunderbolt 3)
robocopy Benchmarks (MBps)
  Write Bandwidth Read Bandwidth
Photos 346.22 301.99
Videos 376.31 348.88
Blu-ray Folder 372.49 375.63

Akitio Thunder3 Duo Pro + 2x 8TB Seagate Enterprise NAS HDDs (Thunderbolt 3)
PCMark8 Storage Benchmarks (MBps)
  Write Bandwidth Read Bandwidth
Adobe Photoshop (Light) 316.60 9.93
Adobe Photoshop (Heavy) 292.50 12.45
Adobe After Effects 71.11 9.58
Adobe Illustrator 204.77 9.06

We now move on to the results from our daisy-chaining experiments.

Testbed Setup and Testing Methodology Daisy Chaining and Performance Implications
Comments Locked

60 Comments

View All Comments

  • DanNeely - Thursday, April 14, 2016 - link

    Were there any issues with video quality/reliability in your daisy chaining test?

    I'm guessing not since you didn't come close to saturating the bus in this test, but I'd be really interested in seeing what happens if you ever get to play with something like a 10bay SSD enclosure and external GPU that could devour most of the bandwidth for what ever they're running.
  • ganeshts - Thursday, April 14, 2016 - link

    The DisplayPort lanes are muxed together with the PCIe lanes, and, if any throttling were to happen, it would be on the PCIe lanes, and not the DisplayPort ones.

    But, yes, we were way short of saturating the link because we were not equipped properly to test that aspect.
  • DanNeely - Thursday, April 14, 2016 - link

    Ok. I didn't know the DP lanes were given explicit preference in the MUXing; but I suppose that in general it's probably the right way to go.
  • repoman27 - Thursday, April 14, 2016 - link

    I just wanted to clarify that Thunderbolt supports multiple signaling modes over a single port via hardware muxing, however, when operating in Thunderbolt signaling mode, it uses protocol converters and crossbar switches to mux at the packet level. So the Thunderbolt mode is more like iSCSI or other technologies that encapsulate and transport data streams over IP networks alongside other packets. The encapsulation that Thunderbolt performs is incredibly lightweight, though, and Intel even refers to it as a "meta protocol".

    Thunderbolt seems to have fairly solid mechanisms in place for guaranteeing timely delivery of packets that are part of isochronous data streams such as DisplayPort. OG Thunderbolt did have some issues with USB audio adapters, as it had no way of knowing that the PCIe packets destined for the USB host controller in the device were particularly time sensitive.

    Bear in mind that the signaling rate of a Thunderbolt PHY is considerably higher than the versions of PCIe or DisplayPort that it's carrying. Also, it can strip out all the bit-stuffing that is normally used to maintain a constant DisplayPort data rate and just send the packets carrying actual data. Or it can essentially bit-stuff with PCIe packets instead.

    And one last niggle, Thunderbolt cables are really nothing like regular DisplayPort cables aside from sharing the miniDP connector. They're active and have four full-duplex signaling lanes, whereas DP cables are generally passive and only support half-duplex lanes for the main link.
  • DanNeely - Friday, April 15, 2016 - link

    You say USB Audio was a problem with the original generation, does that mean it's not a problem in the 2nd/3rd generation? If so, how was it fixed: Throwing more bandwidth at the problem, or by doing usb packet inspection to ID and prioritize the usb audio stream?

    Also, am I right in assuming a TB dock with audio out would be using a built in usb audio device?
  • repoman27 - Saturday, April 16, 2016 - link

    Here's Anand's description of the original problem: http://www.anandtech.com/show/4832/the-apple-thund...

    Honestly, I have no idea if the issue with the Promise Pegasus R6 drives (one of the very first Thunderbolt devices to make it to market) was ever fully resolved. In the meantime, Apple has released updates to their EFI, Thunderbolt host, device and cable firmware (yup, even the cables have firmware), and USB host controller and audio device drivers. If I had to guess, making the USB host controller / audio device drivers Thunderbolt aware and capable of isochronous bandwidth reservation (a la FireWire) might have solved the problem. However, Anand's conclusions about the root cause are different than mine, so I could be way off base.

    And yes, AFAIK Thunderbolt docks all use USB devices in some form or another for audio I/O.
  • danbob999 - Thursday, April 14, 2016 - link

    "the street price of $378 sounds reasonable"

    Sorry but no, a hard drive case is not worth $378. Wake me up when it cost less than $30.
  • NCM - Thursday, April 14, 2016 - link

    On the off chance that you're just ignorant, not a troll, I'll point out that it's not a "hard drive case." This enclosure holds two drives, it provides a selection of hardware RAID options, and it has very high speed connectivity via Thunderbolt 3. It's also one of the first products of of it kind. Each one of those things adds cost.

    Yes it's not cheap, but this kind of product will become less expensive over time. For comparison purposes we have a bunch of 2.5" drive RAID enclosures at the office that run about $270 each (empty), see http://www.bhphotovideo.com/c/product/882789-REG/C...
  • danbob999 - Thursday, April 14, 2016 - link

    This is a 2x3.5" case. It's too bulky to be useful for 2.5" SSDs. It's too expensive and no faster and a plain USB3 case when you put two 3.5" HDDs.
    I wasn't trolling, I seriously don't see any use case for it. The fact that it needs a fan to operate just make it worse.
  • Guspaz - Thursday, April 14, 2016 - link

    Can you point out a 2x3.5" USB 3 hard drive enclosure with hardware RAID support for under $30? I get that you think $378 is too much, but $30 seems far more unreasonable a price than $378.

Log in

Don't have an account? Sign up now