Worst-Case Consistency, Thermals, and Power Consumption

The performance of the drives in various real-world access traces as well as synthetic workloads was brought out in the preceding sections. We also looked at the performance consistency for these cases. Power users may also be interested in performance consistency under worst-case conditions, as well as drive power consumption. The latter is also important when used with battery powered devices such as notebooks and smartphones. We analyze each of these in detail below.

Flash-based storage devices tend to slow down in unpredictable ways when subject to a large number of small-sized random writes. Many benchmarks use that scheme to pre-condition devices prior to the actual testing in order to get a worst-case representative number. Fortunately, such workloads are uncommon for direct-attached storage devices, where workloads are largely sequential in nature. Use of SLC caching as well as firmware caps to prevent overheating may cause drop in write speeds when a flash-based DAS device is subject to sustained sequential writes.

Our Sequential Writes Performance Consistency Test configures the device as a raw physical disk (after deleting configured volumes). A fio workload is set up to write sequential data to the raw drive with a block size of 128K and iodepth of 32 to cover 90% of the drive capacity. The internal temperature is recorded at either end of the workload, while the instantaneous write data rate and cumulative total write data amount are recorded at 1-second intervals.

Sequential Write to 90% of Disk Capacity - Performance Consistency

Our primary focus is on the performance in the Haswell testbed. The Extreme PRO v2 starts off around 1800 MBps for up to 30GB of continuous writes before dropping down to around 1500 MBps for the remainder of the workload. There is a 22C rise from 56C to 78C in this process. The P50 also starts off similarly, sustaining around 1800 MBps for 13GB of continuous writes before moving down to 1400 MBps. However, the worrisome part is the drops to sub-50 MBps for 3-5 seconds at a time before recovering for around 10-15 seconds in the latter half of the workload. This symptom is seen after around 610GB of continuous writes. We do not see this problem when the drive is connected via the eGFX enclosure - however, the fill rate is around 1280 MBps consistently for the full duration of the workload in that configuration. With the USB 3.2 Gen 2 port of Alpine Ridge, the P50 is again consistent around 850 MBps. The performance of the other drives in other configurations are along expected lines, as can be observed in the above graphs.

Bus-powered devices can configure themselves to operate within the power delivery constraints of the host port. While Thunderbolt 3 ports are guaranteed to supply up to 15W for client devices, USB 3.0 ports are guaranteed to deliver only 4.5W (900mA @ 5V). In this context, it is interesting to have a fine-grained look at the power consumption profile of the various drives. Using the Plugable USBC-TKEY, the bus power consumption of the drives was tracked while processing the CrystalDiskMark workloads (separated by 30s intervals). The graphs below plot the instantaneous bus power consumption against time, while singling out the maximum and minimum power consumption numbers.

Drive Power Consumption - CrystalDiskMark Workloads

The system lock-up during the RND4K Q32T16 workload component is evident in the above graphs, where the power consumption tracker on a different machine continues to keep tracking the power numbers while the system and the drive itself are frozen for all practical purposes. The most interesting cases for the above set are with the testbed demonstrating maximum performance - the Haswell testbed. The Extreme PRO v2 has a peak of 7.23W, and a minimum of 2.91W. The corresponding numbers for the P50 are 8.13W and 3.17W. Interestingly, we see the peak for the Extreme PRO v2 and the P50 with the eGFX configuration - 7.41W and 8.19W respectively. Using the USB SuperSpeed 10Gbps port, the drives run a bit cooler - Extreme PRO v2 peaks at 6.59W, while the P50 peaks at 7.39W. It must be noted that the P50 comes with a status LED (which glows white when the drive is connected), while the Extreme PRO v2 doesn't.

PCMark 10 Storage Bench - Real-World Access Traces Miscellaneous Aspects and Concluding Remarks
Comments Locked

81 Comments

View All Comments

  • six_tymes - Monday, October 5, 2020 - link

    spot on. thank you for posting truths.
  • vol.2 - Monday, October 5, 2020 - link

    When I read it, it sounded like the issue was with USB 3.X devices operating on a USB4.0 chipset. My assumption, and I don't think it was explicitly addressed, is that USB4.0 would be "full speed" per it's own specs. Of course, it wasn't explicitly addressed (from what I took away from this), so I phrased my comment as a question; "So wait for USB4 devices then?"
  • repoman27 - Monday, October 5, 2020 - link

    You’re reading is pretty much on the money, but the answer to your question is a bit trickier.

    The first USB4 hosts to hit the market will (probably) be Intel Tiger Lake based products which have integrated Thunderbolt 4 and support USB4 40Gbps. The USB4 spec requires backwards compatibility with USB 3.2, including both the Gen 1 (5Gbps) and Gen 2 (10Gbps) PHYs. It does not, however, require USB3 dual-lane operation (Gen 2x2, 20Gbps), and Intel has not included this optional feature in the controller integrated into Tiger Lake CPUs.

    If Intel doesn’t have any plans for integrated USB 3.2 20Gbps, I fail to see how it becomes widespread, unless Apple and AMD both embrace it in their future chipsets. On the other hand, USB 3.2 devices are probably always going to be cheaper than Thunderbolt or USB4 gear. Paying the premium for a USB 3.2 20Gbps device today is somewhat questionable, unless you have a capable host or the performance difference compared to other products when connected to a 10Gbps port is worth it to you.
  • magreen - Monday, October 5, 2020 - link

    Why will USB3.2 20Gbps devices always be cheaper than USB43 gear? It seems likely USB3.2 20 Gbps will be a niche product and without being produced in high volume, street price won't come down. USB4 might be initially expensive, but volume production and competition may bring street prices down to what we see today for USB3.2 Gen 1 5Gbps devices.
  • repoman27 - Monday, October 5, 2020 - link

    Because USB4 is essentially Thunderbolt, but even more complicated. It will always take way more silicon and way more power than USB3 on the same node. Thus it’s never going to be as cheap. Economies of scale can’t solve everything.
  • vol.2 - Thursday, October 8, 2020 - link

    Sounds about right. I don't have a sudden need for USB 3.2 dual lane. It seems like the best case for more parties to forgo support for it altogether and push USB 4.0. On a related note, USB 3.X has always been very buggy and unstable/unreliable for me, so hopefully 4.X fixes some of that.
  • Meteor2 - Monday, October 26, 2020 - link

    Not if you plug a USB4-20Gbps or a USB4-40Gbps SSD into them.
  • YB1064 - Tuesday, October 6, 2020 - link

    Why is the ASM2364 winning in every scenario compared to Thunderbolt, if TB offers higher bandwidth? Am I reading this incorrectly?

    WTF is this godawful nomenclature dumpster fire??? The idiots on the USB standards committee need to be flogged with extreme prejudice.
  • repoman27 - Tuesday, October 6, 2020 - link

    The ASMedia ASM2364 is the PCIe NVMe to USB3 20Gbps bridge chip used by the devices being tested for this review.

    The ASMedia ASM3242 is the USB3 20Gbps host controller that was used for testing these USB3 20Gbps drives.

    The Intel JHL6540 is a Thunderbolt controller which includes an integrated USB3 10Gbps host controller for interoperability with USB3 devices. This was used to test these USB3 drives while connected to a USB3 10Gbps host. The ASM3242 is winning because it supports twice the USB3 signaling rate as the JHL6540.

    When connected to a Thunderbolt 3 device, the JHL6540 supports up to 40 Gbit/s. In some of the tests, the ASM3242 card was actually plugged into a Thunderbolt 3 enclosure and connected to the host PC via the JHL6540.
  • Googer - Tuesday, October 6, 2020 - link

    It's kind of sad to think that USB is now faster than SATA. Will there ever be a SATA 4 for SSD and future bulk storage technologies?

Log in

Don't have an account? Sign up now