Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the IOPS divided by standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

Steady-state performance has never been Crucial's nor Silicon Motion's biggest strength, but 5K IOPS is fairly average for a value-oriented SSD and for the target group the performance is just fine. 

Steady-State 4KB Random Write Consistency

As some of you have mentioned in the comments, displaying standard deviation alone isn't very useful because it totally ignores the IOPS, so I've changed the graph to IOPS divided by standard deviation during the last 400 seconds of the run. The SM2246EN has always been very inconsistent and it appears that the BX100 is no exception since it has the worst consistency we've tested so far. 

Crucial BX100 250GB
Default
25% Over-Provisioning

Putting the IOPS under a microscope shows that the consistency really is poor. That's nothing new to the SM2246EN and the graph is very similar to the other SM2246EN drives we have tested, although I was hoping that Crucial would have tuned the firmware for better consistency. On the other hand, the users buying the BX100 are unlikely to put the drive under harsh 4KB random write workloads, so all in all the consistency isn't much of an issue.

Crucial BX100 250GB
Default
25% Over-Provisioning
A Look at Storage Executive - Crucial's SSD Toolbox AnandTech Storage Bench - The Destroyer
Comments Locked

67 Comments

View All Comments

  • stickmansam - Friday, April 10, 2015 - link

    Any thoughts on how performance would be like if the BX100 didn't have the Samsung like OP?
  • bricko - Friday, April 10, 2015 - link

    Been researching ssd for a bit, these seem ok, but after watching the Intel discussion of their massive new Intel 750 NVMe, PCIe 3, 1.2 Tb will make one cry, 2-4 times the speed of these old SATA stuff. Half height card for pcie slot. But massive cost, like 1100 for the 1.2 Tb, but the charts are scary. Better with new X99 mobo

    http://www.pcper.com/reviews/Storage/Intel-SSD-750...

    http://www.anandtech.com/show/9090/intel-ssd-750-p...
  • Sunburn74 - Saturday, April 11, 2015 - link

    But no real world performance benefit for 99% of us.
  • just4U - Friday, April 10, 2015 - link

    I think the only real problem I have with the BX100 is ... believe it or not pricing. Here in Canada I've bought 4 of these drives but I've had to scour the net for price match deals to even come close to the MX100 which.. actually was 10-20% lower in costs. Ticks me off really.. their coming close to price parity with the older model but still not there yet.. The Sandisc Ultra2 is cheaper but it's almost always out of stock.. they tried to price this damn thing like the higher end drives. :(
  • stickmansam - Saturday, April 11, 2015 - link

    Canada has always suffered by worse off pricing and stock issues.
  • frombauer - Saturday, April 11, 2015 - link

    Would this be a tangible empirical upgrade over a 256GB Samsung 840 Pro?
  • Margalus - Saturday, April 11, 2015 - link

    no, nothing you would be able to notice in usage.
  • Morawka - Saturday, April 11, 2015 - link

    whats up with the 850 EVO scores? do you guys only have laptop drives or something? Wanted to see how it compares to a 2.5" 850 Evo, and obviously the Msata scores dont compare to it.
  • Kristian Vättö - Saturday, April 11, 2015 - link

    I haven't had time to put the 2.5" 850 EVOs through our new SSD suite yet, but I have the scores for the mSATA/M.2 versions since we just reviewed those. The performance should essentially be the same though since the hardware is no different.
  • Nordlicht - Saturday, April 11, 2015 - link

    It would have been nice to have BX100 being compared to Transcend SSD370. Both use the same controller. However, SSD370 uses Micron's 20nm NAND whereas BX100 uses the next generation 16nm. Performance is similar?

    On the surface the SSD370 could be more reliable due to bigger feature size. BX100 commands a small price premium, though.

Log in

Don't have an account? Sign up now