Random Read

Random read speed is the most difficult performance metric for flash-based SSDs to improve on. There is very limited opportunity for a drive to do useful prefetching or caching, and parallelism from multiple dies and channels can only help at higher queue depths. The NVMe protocol reduces overhead slightly, but even a high-end enterprise PCIe SSD can struggle to offer random read throughput that would saturate a SATA link.

Real-world random reads are often blocking operations for an application, such as when traversing the filesystem to look up which logical blocks store the contents of a file. Opening even an non-fragmented file can require the OS to perform a chain of several random reads, and since each is dependent on the result of the last, they cannot be queued.

Our first test of random read performance looks at the dependence on transfer size. Most SSDs focus on 4kB random access as that is the most common page size for virtual memory systems and it is a common filesystem block size. Maximizing 4kB performance has gotten more difficult as NAND flash has moved to page sizes that are larger than 4kB, and some SSD vendors have started including 8kB random access specifications. It is worth noting that 3D XPoint memory, from a fundamental standpoint, does not impose any inherent block size restrictions on the Optane SSD, but for compatibility purposes the P4800X by default exposes a 512B sector size.

Queue Depth 1

For our test, each transfer size was tested for four minutes and the statistics exclude the first minute. The drives were preconditioned to steady state by filling them with 4kB random writes twice over.

Random Read
Vertical Axis scale: Linear Logarithmic

The Optane SSD starts off with about eight times the throughput of the other drives for small random reads. As the transfer sizes grow past 16kB the Optane SSD's performance starts to level off and the flash SSDs start to catch up, with the Micron 9100 overtaking the Intel P3700. At 1MB transfer size the Optane SSD is only providing an additional 50% higher throughput than the Micron 9100.

Queue Depth >1

Next, we consider 4kB random read performance at queue depths greater than one. A single-threaded process is not capable of saturating the Optane SSD DC P4800X with random reads so this test is conducted with up to four threads. The queue depths of each thread are adjusted so that the queue depth seen by the SSD varies from 1 to 64, with every single queue depth from 1 through 16, then 18, 20, and factors of four up to 64 (so 24, 28, 32... to 64). The timing is the same as for the other tests: four minutes for each tested queue depth, with the first minute excluded from the statistics.

Looking just at the range of throughputs and latencies achieved, it is clear that the Optane SSD DC P4800X is in a different league entirely from the flash SSDs. The Optane SSD saturates part way through the test with a throughput +30% higher than what the Micron 9100 can deliver even at QD64, and at the same time its 99.999th percentile latency is half of the Micron 9100's median latency.

Between the two flash SSDs, the Intel P3700 has better performance on average through most of the test, but its maximum achieved throughput is slightly lower than the Micron 9100's peak and the 9100 offers lower latency at the high end. The Micron 9100 also has much better 99.999th percentile latency across almost the entire range of queue depths.

Random Read Throughput
Vertical Axis units: IOPS MB/s

In absolute terms, the Optane SSD's performance is uncontested. Even though the Optane SSD's random read throughput is saturating at QD8, by QD6 it's outperforming what either flash SSD can deliver at any reasonable queue depth. Beyond QD8 the Optane SSD does not deliver even incremental improvement in throughput and increasing queue depth just adds latency. This test stops at QD64, which isn't enough to saturate either flash SSD. The Micron 9100 MAX is rated for a maximum of 750k random read IOPS, but clearly the Optane SSD delivers far better performance at the kinds of queue depths that are reasonably attainable.

Random Read Latency
Mean Median 99th Percentile 99.999th Percentile

All three SSDs show median latency growing slowly across a wide range of queue depths. At QD1 the 99th percentile curves are very close to the median latency curves, but at high queue depths the 99th percentile latency is around twice the median. For the Optane SSD and the Micron 9100 MAX, the 99.999th percentile latency is higher by another factor of two or so, but the Intel P3700 cannot deliver such tight regulation and its worst-case latencies are well over a millisecond.

Random Write

Flash memory write operations are far slower than read operations. This is not always reflected in the performance specifications of SSDs because writes can be deferred and combined, allowing the SSD to signal completion before the data has actually moved from the drive's cache to the flash memory. The 3D XPoint memory used by the Optane SSD DC P4800X does have slower writes than reads, and it was commented that Intel did not specificy read latency when Optane was initially announced, but our results show that the disparity is not as large. With inherently fast writes and no page size and erase block limitations, the Optane SSD should be far less reliant on write combining and large spare areas to offer high throughput random writes. The drive's translation layer is probably far simpler than what flash SSDs require, potentially giving a latency advantage.

Queue Depth 1

As with random reads, we first examine QD1 random write performance of different transfer sizes. 4kB is usually the most important size, but some applications will make smaller writes when the drive has a 512B sector size. Larger transfer sizes make the workload somewhat less random, reducing the amount of bookkeeping the SSD controller needs to do and generally allowing for increased performance.

Random Write
Vertical Axis scale: Linear Logarithmic

The Micron 9100 really doesn't like random writes smaller than 4kB, but both Intel drives handle it relatively well. The Optane SSD DC P4800X has only a 30% higher throughput result than the P3700 for transfer sizes of 4kB and smaller. The Intel P3700 (owing mainly to its relatively low capacity) doesn't benefit very much as transfer sizes grow beyond 4kB, as it saturates soon after. The Optane SSD maintains a clear lead for transfers of 8kB and larger, averaging about twice the throughput of the Micron 9100 as both show diminishing returns from increased transfer sizes.

Queue Depth >1

The test of 4kB random write throughput at different queue depths is structured identically to its counterpart random write test above. Queue depths from 1 to 64 are tested, with up to four threads used to generate this workload. Each tested queue depth is run for four minutes and the first minute is ignored when computing the statistics.

The QD1 starting points for all three drives are somewhat close together, with the fastest drive (the Optane SSD, of course) only offering about twice the random write throughput than the Micron 9100, with less than half the average latency. From there, the gaps widen quickly. The Intel P3700 reaches its maximum throughput very quickly and then the latency just piles up. The Micron 9100 keeps its median and 99th percentile latency reasonably well controlled until reaching its maximum throughput, which is half of what the Optane SSD can deliver.

Random Write Throughput
Vertical Axis units: IOPS MB/s

While QD64 wasn't enough to completely saturate the flash SSDs with random reads, here with random writes, QD8 is enough for any of the drives, and the P3700 is done around QD2. The Micron 9100 starts out as the slowest of the three but soon overtakes the Intel P3700.

When examining the latency statistics, we should keep in mind that all three drives reached their full throughput by QD8. At queue depths higher than that, latency increases with no improvement to throughput. A well-tuned server will generally not be operating the drives in that regime, so the right half of these graphs can be mostly ignored.

Random Write Latency
Mean Median 99th Percentile 99.999th Percentile

Median latency for these drives is quite flat until they reach saturation. 99th percentile latency for the flash SSDs shoots up when they're operated at unnecessarily high queue depths. The 99.999th percentile latency of the Intel P3700 is never less than 1ms and actually exceeds 10ms at the end of the test. The Micron 9100's 99.999th percentile latency is fairly close to that of the Optane SSD until the 9100 hits QD4, where it spikes and surpasses 1ms shortly before the drive reaches full throughput. Meanwhile, the Optane SSD's 99.999th percentile latency only climbs up to a third of a millisecond even at QD64.

Checking Intel's Numbers Sequential Access Performance
POST A COMMENT

117 Comments

View All Comments

  • masouth - Tuesday, May 2, 2017 - link

    With ddriver and RAM? I've only skimmed ddriver's posts but I believe a summary would be.

    1) RAM is faster than this product so adding more RAM would be a better option than adding a middle man that is only faster than the data storage device but still slower than RAM.
    2) RAM has much more endurance than these drives
    3) Servers tend to stay on 24/7 and have back up power solutions (UPS, generators, etc) to allow for a RAM data flush to a non-volatile data storage device prior to any power loss so it renders Optane's advantage of being non-volatile fairly moot.

    ddriver believes these reasons result in this product having very niche uses yet Intel keeps hyping this as a solution for every user while hiding behind synthetic benchmarks instead of demonstrating real world applications which would reveal that more RAM would lead to a superior solution in many/most cases.

    I may have missed something but I think that sums up what I have read so far.
    Reply
  • masouth - Tuesday, May 2, 2017 - link

    oops, in the last part I forgot that he saying they are using the benchmarks to hide the fact that it's not as far ahead of NAND speads (although it is ahead) as they claim. Reply
  • AnTech - Saturday, April 29, 2017 - link

    Is Intel XPoint Optane a fiasco? Check out:
    Intel crosses an unacceptable ethical line
    http://semiaccurate.com/2017/03/27/intel-crosses-u...
    Reply
  • Santoval - Tuesday, June 6, 2017 - link

    A few days ago I registered here on Anandtech and I found it very odd that such a very knowledgeable website provided (only) unsecure cleartext registration and log-in forms. I felt awkward and uncomfortable, because that is a very no no for me. I wanted to register though, so I used the Tor Browser, to risk being sniffed only by the exit node. Now I see that Charlie (which I used to read ages ago) has taken this quite a few steps further..
    The guy sells $1,000 annual "professional subscriptions" on a completely private, crystal clear transparent, as public as it gets, 100% unencrypted page. I am utterly dumbfounded... And I lost all appetite to read his article or anything from him ever again. For life. Click your link and then click the "Become a subscriber" link on the top to enjoy this adorable (in)security atrocity..
    Reply
  • tsk2k - Thursday, April 20, 2017 - link

    You forgot one thing, CRYSIS 3 FPS?!?! Reply
  • philehidiot - Thursday, April 20, 2017 - link

    I find the go faster stripes on my monitor screen make a massive difference to my FPS. I have many, many more FPS as a result. It's due to the quality of the paint - Dulux one-coat just bring down my latency to the point whe...

    .... I've sniffed too much of this paint, haven't I?
    Reply
  • ddriver - Friday, April 21, 2017 - link

    If you use this instead of ram it will most likely be 3 FPS indeed :) Reply
  • mtroute - Friday, April 21, 2017 - link

    never has Intel claimed that this product is faster the DRAM...Your indignation is not proportional to even your perceived slight by Intel. You work for SK Hynix or more likely Powerchip don't you? Reply
  • ddriver - Sunday, April 23, 2017 - link

    Nope, I am self employed. I never accused intel of lying about hypetane being faster than dram. I accused them of lying how much faster than NAND it is and how close to dram it is. And I have only noted that it is hundreds of times slower than dram, making the population of dimm slots (which some intel cheerleaders claim will magically make hypetane faster) is a very bad prospect in 99.99% of the use cases.

    I don't have corporate preferences either, IMO all corporations are intrinsically full of crap, yet the amount of it varies. I also do realize that "nicer" companies are only nicer because they are it a tough situation and cannot afford to not be nice.

    What annoys me is that legally speaking, false advertising is a crime, yet everyone is doing it, because it has so many loopholes, and what's worse, the suckers line up to cheer at those lies.
    Reply
  • MobiusPizza - Sunday, April 23, 2017 - link

    It is still a first gen product and I think it has potential in servers and scientific computing. First gen SSDs were also crappy with low capacity. Give it 5 years I think it will make more sense. Reply

Log in

Don't have an account? Sign up now