Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

Given the higher over-provisioning and an enterprise-oriented controller, it's no surprise that the SSD 750 has excellent steady-state random write performance. 

Steady-State 4KB Random Write Consistency

The consistency is also very good, although the SSD 750 can't beat the 850 Pro if just focusing on consistency. When considering that the SSD 750 provides nearly three times the performance, it's clear that the SSD 750 is better out of the two. 

Intel SSD 750 1.2TB (PCIe 3.0 x4 - NVMe)
Default

At the initial cliff the performance drops to around 15K IOPS, but it quickly rises and seems to even out at about 22-23K IOPS. It actually takes nearly an hour for the SSD 750 to reach steady-state, which isn't uncommon for such a large drive but it's still notable. 

I couldn't run tests with added over-provisioning because NVMe drives don't support the usual ATA commands that I use to limit the LBA of the drive. There is similar command set for NVMe as well, but I'm still trying to figure out how to use them as there's isn't too much public info about NVMe tools.

Intel SSD 750 1.2TB (PCIe 3.0 x4 - NVMe)
Default
Introduction, The Drive & The Test AnandTech Storage Bench - The Destroyer
Comments Locked

132 Comments

View All Comments

  • KAlmquist - Thursday, April 2, 2015 - link

    It's too bad that Anandtech didn't benchmark the 400 GB model, since that's the one most people are going to be most interested in buying. I assume that it's a case of Intel not making the 400 GB model available for review, rather than Anandtech deciding not to review it.
  • jwilliams4200 - Thursday, April 2, 2015 - link

    Agreed, the 400 GB model is more interesting to consumers.

    Also, I hope that if Anandtech does test the 400GB model, that they re-run the tests of the comparison SSDs so that the competitors are overprovisioned to 400GB usable capacity (from 512GB or whatever nominal capacity). That is the only reasonable way to compare, since anyone who wants high sustained performance and is willing to try a drive with only 400GB to achieve it would obviously be willing to overprovision, for example, a 512GB Samsung 850 Pro to only 400GB usable to achieve higher sustained performance.
  • Kristian Vättö - Thursday, April 2, 2015 - link

    That is something that I've had on my mind for a while now and I even have a way to do it now (the Storage Bench traces are a bit tricky since they are run on a raw drive, but thankfully I found an hdparm command for limiting the far LBA count). The only issue is time because it takes roughly two days to test one drive through the 2015 suite, so I may be include a drive or two as comparison points but I definitely can't test all drives with added OP.
  • Kristian Vättö - Thursday, April 2, 2015 - link

    Not far LBA count, but raw LBA count, obviously :)
  • Stahn Aileron - Friday, April 3, 2015 - link

    Honestly, I'd rather have AnandTech test drives and components as-is ("stock" from the manufacturer) and publish those results rather than spend time doing tests on non-standard, customized configurations. Let the customers do that if they truly need that type of set-up or leave it to integrators/specialists.

    As far as I know, most customers of a product just want to use it immediately right of the box, no mucking with special settings. Most products are advertised that way as well.

    Really, just test the product(s) as advertised/intended by the manufacturer first and foremost to see if it matches their claims and properly serves the target userbase. Specialty cases should only be done if that is actively advertised as a feature, there is truly high interest, something makes you curious, and/or you have the time.
  • jwilliams4200 - Friday, April 3, 2015 - link

    If this were a review site for the totally clueless, then you might have a point. But anandtech has always catered to enthusiasts and those who either already know a lot about how computer equipment works, or who want to learn.

    The target audience for this site would certainly consider something as simple as overprovisioning an SSD if it could significantly increase performance and/or achieve similar performance at lower cost relative to another product. So it makes sense to test SSDs configured for similar capacity or performance rather than just "stock" configuration. Anyone can take an SSD and run a few benchmarks. It takes a site as good as anandtech to go more in-depth and consider how SSDs are actually likely to be used and then present useful tests to its readers.
  • Kristian Vättö - Thursday, April 2, 2015 - link

    That is correct. I always ask for all capacities, but in this case Intel decided to sample all media with only 1.2TB samples. I've asked for a 400GB, though, and will review it as soon as I get it.
  • Mr Alpha - Thursday, April 2, 2015 - link

    Has anyone managed to find this mythological list of compatible motherboards?
  • Kristian Vättö - Thursday, April 2, 2015 - link

    I just asked Intel and will provide a link as soon as I get one. Looks like it's not up yet as they didn't have an answer right away.
  • tstones - Thursday, April 2, 2015 - link

    Older chipsets like z77 and z87 will support NVMe?

Log in

Don't have an account? Sign up now