Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the IOPS divided by standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

Steady-state performance has never been Crucial's nor Silicon Motion's biggest strength, but 5K IOPS is fairly average for a value-oriented SSD and for the target group the performance is just fine. 

Steady-State 4KB Random Write Consistency

As some of you have mentioned in the comments, displaying standard deviation alone isn't very useful because it totally ignores the IOPS, so I've changed the graph to IOPS divided by standard deviation during the last 400 seconds of the run. The SM2246EN has always been very inconsistent and it appears that the BX100 is no exception since it has the worst consistency we've tested so far. 

Crucial BX100 250GB
Default
25% Over-Provisioning

Putting the IOPS under a microscope shows that the consistency really is poor. That's nothing new to the SM2246EN and the graph is very similar to the other SM2246EN drives we have tested, although I was hoping that Crucial would have tuned the firmware for better consistency. On the other hand, the users buying the BX100 are unlikely to put the drive under harsh 4KB random write workloads, so all in all the consistency isn't much of an issue.

Crucial BX100 250GB
Default
25% Over-Provisioning
A Look at Storage Executive - Crucial's SSD Toolbox AnandTech Storage Bench - The Destroyer
POST A COMMENT

67 Comments

View All Comments

  • Hulk - Friday, April 10, 2015 - link

    Wow, that validates all of the other reviews I've been reading about this drive. Great performance, great value, and no TLC NAND worries. I'm getting one for my new laptop. Reply
  • Uplink10 - Saturday, April 11, 2015 - link

    If you want a great value you better wait for a few months (maybe more than a few) till 3D NAND drives are going to come out from SanDisk, Micron, Intel, Toshiba. Then Samsing will not be the only company with 3D NAND drives and prices will probably going to come down. Reply
  • Harry_Wild - Saturday, April 25, 2015 - link

    Thanks for the info! I will wait it out to get the best 3D NAND drives. Reply
  • Hace - Friday, April 10, 2015 - link

    I'm kinda surprised you didn't draw more attention to the 850 EVO as a competitor, which is neck-in-neck with pricing. Reply
  • digiguy - Friday, April 10, 2015 - link

    Yes, similar value for the money, and definitely more than the 840 EVO.... Reply
  • Kristian Vättö - Friday, April 10, 2015 - link

    I have to admit that I totally overlooked the 850 EVO as I was kinda assuming that its price would be higher. I put this review together on a very short notice as Ryan couldn't get the MacBook review finished for today, hence my mistake of not paying enough attention to the prices (even though I updated the table today...). Anyway, I've updated the conclusion to take the 850 EVO into account because as you said, the pricing is very close and it does provide a little higher performance. Reply
  • Hulk - Friday, April 10, 2015 - link

    While the 850EVO is definitely a competitor for the BX100 the fact that it uses TLC NAND vs. MLC for the BX100 is a big deal for many people who aren't as yet convinced the 850EVO won't be affected by the read issue that is still a problem with the 840EVO. I realize the cell size for the 850 EVO's 3D NAND is much greater but as I wrote above many people, myself included, don't see the need to roll the dice on the 850EVO when the BX100 will provide basically similar performance. Reply
  • just4U - Friday, April 10, 2015 - link

    The 850Evo is $20-$50 more on all models here in Canada. Not sure what stateside or Euro Pricing is like. Reply
  • Margalus - Saturday, April 11, 2015 - link

    I just bought a 1TB EVO for $350 on Amazon, USA Reply
  • repoman27 - Saturday, April 11, 2015 - link

    Kristian, hopefully you'll be helping Ryan with that MacBook review by covering the NVMe(!) SSD it ships with in depth. Looks to be a PCIe 2.0 x4 connected device dubbed "AP0256H", so possibly a semi-custom Apple controller based on the Marvell 88SS1093. Reply

Log in

Don't have an account? Sign up now