Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the IOPS divided by standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

It looks like Crucial has finally taken steps to improve steady-state performance, although the additional over-provisioning is partially to thank for the increase. One criticism I always had about Crucial's SSDs was the relatively bad steady-state performance, but the MX200 finally brings the performance closer to other high-end drives.

Steady-State 4KB Random Write Consistency

The consistency is very good as well and far better than what the BX100 offers. 

Crucial MX200 250GB

The IO consistency appears to behave differently from the MX100 and the graph resembles 850 EVO and Pro quite a bit by dropping quickly in performance and then slowly increasing before evening out. The 1TB model is an exception, though, as it seems that the firmware can't properly handle such a large capacity, which results in worse performance and considerably higher variation. Unfortunately, the MX200 wouldn't respond to the hdparm command that I use for over-provisioning testing, so I don't have any results with added over-provisioning at this point.

Crucial MX200 250GB
Introduction, The Drives & The Test AnandTech Storage Bench - The Destroyer
Comments Locked

62 Comments

View All Comments

  • beginner99 - Friday, May 22, 2015 - link

    SSD market has becoming just as boring as CPU. Time for new chipste and NVMe drives which will finally deliver an actual improvement. However for clients, it doesn't really matter...even my intel G2 was good enough.
  • JackF - Friday, May 22, 2015 - link

    I continue to be happy with my 1TB Samsung EVO drive. For $359, it is/was a good value, and consistently performs high on these comparison charts.

    Just out of curiosity, why is the Mushkin Reactor not included on any of the comparison charts? When I was considering an upgrade, it was at the top of my list, but you never include it in the comparison charts. You gave it a Anandtech recommendation back in February?
  • Teknobug - Friday, May 22, 2015 - link

    Sounds like what Kingston did with the SSDNow series, older SSDNow drives were fast but the newer ones are ridiculously slow, so now the same with Crucial?
  • MrSpadge - Saturday, May 23, 2015 - link

    Take another look at the BX100 - that's a really good value drive and anything but slow.
  • jabber - Saturday, May 23, 2015 - link

    The Kingston SSDs are great value as they are usually the cheapest SSDs on Amazon etc. And in most cases are probably being used to upgrade SATA II equipped hardware. In which case they will push 260MBps all day long. Seen many labour for days over SSD specs and reviews when in fact the machine they want to upgrade doesn't have SATA III.
  • der - Friday, May 22, 2015 - link

    Crucial for CRUCIAL performance. Eek!
  • eanazag - Friday, May 22, 2015 - link

    In the SSD market I only see excitement in PCIe/NVMe designs. I believe there is a place for SATA drives, but the differentiation needs to innovative; i.e. beyond performance. Warranty, reliability, consistency, software tools, and RAID support are areas for differentiation.

    After looking at the value & mainstream performance products in the chart the Samsung Evo is a compelling product. Crucial's BX line is a price undercutting product compared to the Evo and it doesn't do that. I say this because the BX feature set is sub-par to the Evo. The MX is the Evo's direct competitor.
  • KaarlisK - Friday, May 22, 2015 - link

    I do not get TRIM validation. There was never doubt that TRIM works in the sense that they LBAs contain zeroes. The question always was whether TRIMming a drive would restore degraded performance. And for some drives, it would not.
    Why isn't this verified any more? Or have I missed something?
  • creed3020 - Friday, May 22, 2015 - link

    Thanks so much for the review Kristian. Now I can finally compare this in Bench to others drives when making recommendations for clients.
  • zodiacfml - Friday, May 22, 2015 - link

    Price and warranty. Anyway, it is just getting more difficult to compete with Samsung which is the case and it will just get worse.

Log in

Don't have an account? Sign up now