Single-Threaded Performance

This next batch of tests measures how much performance can be driven by a single thread performing asynchronous I/O to produce queue depths ranging from 1 to 64. With these fast SSDs, the CPU can become the bottleneck at high queue depths and clock speed can have a major impact on IOPS.

4kB Random Reads

With a single thread issuing asynchronous requests, all of the SSDs top out around 1.2GB/s for random reads. What separates them is how high a queue depth is necessary to reach this level of performance, and what their latency is when they first reach saturation.

Random Read Throughput
Throughput: IOPS MB/s
Latency: Mean Median 99th Percentile 99.999th Percentile

For the Optane SSDs, the queue depth only needs to reach 4-6 in order to be near the highest attainable random read performance, and further increases in queue depth only add to the latency without improving throughput. The flash-based SSDs require queue depths well in excess of 32. Even long after the Optane SSDs have reached saturation an latency has begun to climb, the Optane SSDs continue to offer better QoS than the flash SSDs.

4kB Random Writes

The Optane SSDs offer the best single-threaded random write performance, but the margins are much smaller than for random reads, thanks to the write caches on the flash-based SSDs. The flash SSDs have random write latencies that are only 2-3x higher than the Optane SSD's latency, and the throughput advantage of the Optane SSD at saturation is less than 20%.

Random Write Throughput
Throughput: IOPS MB/s
Latency: Mean Median 99th Percentile 99.999th Percentile

The Optane SSDs saturate around QD4 where the CPU becomes the bottleneck, and the flash based SSDs follow suit between QD8 and QD16. Once all the drives are saturated at about the same throughput, the Optane SSDs offer far more consistent performance.

128kB Sequential Reads

With the large 128kB block size, the sequential read test doesn't hit a CPU/IOPS bottleneck like the random read test above. The Optane SSDs saturate at the rated throughput of about 2.4-2.5GB/s while the Micron 9100 MAX and the Intel P3608 scale to higher throughput.

Sequential Read Throughput
Throughput: IOPS MB/s
Latency: Mean Median 99th Percentile 99.999th Percentile

The Optane SSDs reach their full sequential read speed at QD2, and the flash-based SSDs don't catch up until well after QD8. The 99th and 99.999th percentile latencies of the Optane SSDs are more than an order of magnitude lower when the drives are operating at their respective saturation points.

128kB Sequential Writes

Write caches again allow the flash-based SSDs to approach the write latency of the Optane SSDs, albeit at lower throughput. The Optane SSDs quickly exceed their specified 2GB/s sequential write throughput while the flash-based SSDs have to sacrifice low latency in order to reach high throughput.

Sequential Write Throughput
Throughput: IOPS MB/s
Latency: Mean Median 99th Percentile 99.999th Percentile

As with sequential reads, the Optane SSDs reach saturation at a mere QD2, while the flash-based SSDs need until around QD8 to scale up to full throughput. By the time the flash-based SSDs reach their maximum speed, their latency has at least doubled.

Performance VS Transfer Size Mixed Read/Write Performance
Comments Locked

58 Comments

View All Comments

  • AlaskanDruid - Thursday, November 9, 2017 - link

    Ok. what am I missing? What is the use of these when you can have one in a m.2 form which takes up MUCH less room?

    I know I'm missing something, but what is it?
  • Drumsticks - Thursday, November 9, 2017 - link

    The drives dissipate 12-18W, which is 2-3x more than flash based SSDs. I don't think the m.2 form factor would make heat dissipation easy, unless you scale the design back down to the 32-120GB range.
  • Billy Tallis - Thursday, November 9, 2017 - link

    More than just capacity, you'd almost certainly have to scale down performance by a lot. It's possible to do better than the current PCIe x2 32GB module, but a 7 channel controller would be hard to fit physically, and it would have to run slower to not melt. I think a ~3 channel M.2 drive is plausible, but I'm not sure its write speeds would be high enough.
  • DanNeely - Thursday, November 9, 2017 - link

    it looks like you could physically do 7 (or even 8) channels on a dual sided 22110 m.2 module with the current package size. The thermals would be a real challenge though; and are why I really wish U.2 would've gained at least a little traction in the consumer market.
  • Billy Tallis - Thursday, November 9, 2017 - link

    I was thinking more along the lines of a 22x80mm card. I agree that 7 3D XPoint packages is probably possible on 22x110, but I'm not sure how dense the controller's BGA pad can get.
  • DanNeely - Thursday, November 9, 2017 - link

    If you went 2 sided you could fit 6 on a 2280 card. With a bare bottom they're just short on enough room for a 3rd package on top; move some of that misc stuff underneath and you'd free up the PCB area needed.
  • Rwbeckman - Thursday, November 9, 2017 - link

    This is the datacenter model, good for releasing higher capacity sooner with its size. Plus rackmount cooling design will help. U.2. Will be for hot swap, the card will not. The M.2 enthusiast consumer model is the 900P
  • Samus - Thursday, November 9, 2017 - link

    XPoint technology is just one of the reasons Intel released the "ruler" form factor for future SSD's. I suspect a mobile variant will replace m2 in the future.

    https://www.anandtech.com/show/11702/intel-introdu...
  • CheapSushi - Thursday, November 9, 2017 - link

    Or just use the new M.3 format (wider) that Samsung came out with.
  • Drumsticks - Thursday, November 9, 2017 - link

    Great review Billy, thank you!

    Optane as a gen 1 product is pretty fantastic. I can't wait to see how the rest of the market will compete, and what Intel and Micron (where are their drives?) can do for round two.

Log in

Don't have an account? Sign up now