Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the IOPS divided by standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

Steady-state performance has never been Crucial's nor Silicon Motion's biggest strength, but 5K IOPS is fairly average for a value-oriented SSD and for the target group the performance is just fine. 

Steady-State 4KB Random Write Consistency

As some of you have mentioned in the comments, displaying standard deviation alone isn't very useful because it totally ignores the IOPS, so I've changed the graph to IOPS divided by standard deviation during the last 400 seconds of the run. The SM2246EN has always been very inconsistent and it appears that the BX100 is no exception since it has the worst consistency we've tested so far. 

Crucial BX100 250GB
Default
25% Over-Provisioning

Putting the IOPS under a microscope shows that the consistency really is poor. That's nothing new to the SM2246EN and the graph is very similar to the other SM2246EN drives we have tested, although I was hoping that Crucial would have tuned the firmware for better consistency. On the other hand, the users buying the BX100 are unlikely to put the drive under harsh 4KB random write workloads, so all in all the consistency isn't much of an issue.

Crucial BX100 250GB
Default
25% Over-Provisioning
A Look at Storage Executive - Crucial's SSD Toolbox AnandTech Storage Bench - The Destroyer
Comments Locked

67 Comments

View All Comments

  • Kristian Vättö - Saturday, April 11, 2015 - link

    We've run some basic tests, but nothing too in-depth for the review. Testing an internal drive is a bit more complicated, but hopefully we'll be able to follow up with more thorough testing once we get the initial review out.
  • repoman27 - Saturday, April 11, 2015 - link

    Especially when it's not just internal but full on embedded. We may need to wait for an iFixit or Chipworks teardown to get a better picture of how this is implemented.
  • zodiacfml - Saturday, April 11, 2015 - link

    I'd take the Samsung with longer warranty and fat pixels... i mean, NAND. The 250GB version has more DRAM compared to the BX100 if that is any useful.

    Random read/write is what differentiates these great products.
  • zodiacfml - Saturday, April 11, 2015 - link

    Speaking of random performance, isn't the random read results swapped with the random write chart?
  • digiguy - Friday, April 10, 2015 - link

    Agree with Christian, the limitations of SATA 6GB hardly justify an SSD like the 850 pro or the Extreme pro (which I have), especially with PCIe virtually here (tough still rare in laptops). Something like this SSD (or the MX100 which I also have is perfectly adequate even for enthusiasts). I even wonder how many years the SATA interface will survive... I suspect that in 10 years SATA SSDs might well be a thing of the past... (especially in the 2.5 inches format...).
  • CaedenV - Friday, April 10, 2015 - link

    I think it depends on the use. I would imagine that SSDs are going to transition to M.2 or embedded options as the primary interface. However, I think that SATA, even SATA3, has a very long life ahead as a traditional HDD interface for bulk storage. My bet is that we will see 1-2 M.2 connectors for system drive SSDs, and continue to see 2-4 SATA connectors with RAID support for HDD installs for a very long time. They simply do not cost much, and they don't take a lot of space, so it will sort of be like how PS/2 ports keep showing up even though practically nobody uses them, or how parallel ports hung on a good 10 years after they were useful.
  • Murloc - Saturday, April 11, 2015 - link

    kb-0133s never die, and so neither does PS/2.
  • lordken - Sunday, April 12, 2015 - link

    I do use PS/2, as long as my old M$ multimedia keyboard is going to live...no reason to buy new usb keyboard only because this one is like 10y old :)
    Also using good old mx518 (which is usb ofc)
  • Jaybus - Thursday, April 16, 2015 - link

    I don't think multiple M.2 connections will be likely. A single M.2 is great for small form factors, but it takes up too much board space.
  • Pissedoffyouth - Friday, April 10, 2015 - link

    Awesoem stuff. I have an mSata m500 and its been flawless, unlike the bad performance I've had with samsung 840's.

    Looks like this is the drive to recommend to people, I might even get myself one of the 250gb as portable drive

Log in

Don't have an account? Sign up now