Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the IOPS divided by standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

Steady-state performance has never been Crucial's nor Silicon Motion's biggest strength, but 5K IOPS is fairly average for a value-oriented SSD and for the target group the performance is just fine. 

Steady-State 4KB Random Write Consistency

As some of you have mentioned in the comments, displaying standard deviation alone isn't very useful because it totally ignores the IOPS, so I've changed the graph to IOPS divided by standard deviation during the last 400 seconds of the run. The SM2246EN has always been very inconsistent and it appears that the BX100 is no exception since it has the worst consistency we've tested so far. 

Crucial BX100 250GB
Default
25% Over-Provisioning

Putting the IOPS under a microscope shows that the consistency really is poor. That's nothing new to the SM2246EN and the graph is very similar to the other SM2246EN drives we have tested, although I was hoping that Crucial would have tuned the firmware for better consistency. On the other hand, the users buying the BX100 are unlikely to put the drive under harsh 4KB random write workloads, so all in all the consistency isn't much of an issue.

Crucial BX100 250GB
Default
25% Over-Provisioning
A Look at Storage Executive - Crucial's SSD Toolbox AnandTech Storage Bench - The Destroyer
Comments Locked

67 Comments

View All Comments

  • Vepsa - Friday, April 10, 2015 - link

    I wonder how two of these would do in RAID1 attached to a HP P410/256 controller. My poor little HP N40L could stand a capacity boost (2x250GB HDDs for booting) as well as a performance boost.
  • owbert - Friday, April 10, 2015 - link

    Performance besides power consumption is so close to the mx100.

    Current prices have both bx100 and mx100 around the same price. Would it be a smarter buy to pick the mx100 because it offers a few gigs more storage at each tier?
  • CaedenV - Friday, April 10, 2015 - link

    If I were purchasing today I would spend the extra $5 for a 1TB EVO. However, the EVO isn't likely to see many more price drops over it's life span, while the BX drives are brand new and will probably see a few price drops after the initial release. 6mo from now it might be substantially cheaper than the EVO and be a true budget drive (granted sub $400 for a 1TB SSD is not a bad price at all!).

    For my next build I think I am going to stick with m.2 for the added throughput and ability to have less stuff cluttering my box (looking at ITX next time). I wonder if there will be a 'budget' M.2 drive available by then that will offer better performance than SATA3 or SATA Express options.
  • MrCommunistGen - Friday, April 10, 2015 - link

    Due to the different controller in the 1TB 850 EVO (and the generally lower performance profile it offers compared to the 500GB model) I'd personally skip the 1TB EVO...

    Then again I'm not really in the market for a new SATA SSD at the moment either. My 500GB 840 EVO is *good enough* until I can get some future, shiny PCI-E, NVMe, 3D NAND SSD. Of course I'll need a new machine too since Z77 isn't going to know how to boot a NVMe SSD...
  • Margalus - Friday, April 10, 2015 - link

    the performance difference between the 500GB and the 1TB is negligible. Plus the 500GB is too small. I just bought the 1TB evo a couple weeks ago for $350. I wish they had a 2TB...
  • bunsenbunner - Friday, April 10, 2015 - link

    Nothing like ordering two of these last night (based on other reviews thus far) for a video editing setup only to see this review drop the next day. No stress at all reading through this to verify I made the right purchase! :-)

    Samsung's handling of the 840 EVO issues had me hesitant to go with the 850 EVO (even though I know the 3D NAND in the 850 EVO is a different beast from the TLC electrical drift issues in the 840 EVO). Samsung has had some stellar drives in recent years, but they've also not had any after-the-fact issues to really deal with. A "fix" followed by a second "fix" didn't give me a lot of confidence in Samsung's ability to truly resolve the issue. And this is from a current owner of two 840 EVOs.

    I of course wanted to hold out for the Intel 750 Series, but it skews a little far on the performance vs. value per GB for my needs.
  • Elixer - Friday, April 10, 2015 - link

    Storage Executive installs JAVA, that is why it is so huge.
    In theory, using JAVA means that this app could be ported to linux or Macs fairly easily.

    Personally, I rather not install anything that has to do with JAVA, they should have went with C# or C++ like the other OEMs are using.
  • Mr Perfect - Saturday, April 11, 2015 - link

    Oh. Java? I've actually removed that from my systems. Nothing I have uses it, and it's got as many security flaws as Flash.

    I'm also a little disappointed that this isn't a standalone application, but then I have all of these horrible flashbacks of Internet Explorer updates hopelessly breaking browser-based applications. *IE6 flashback*
  • dave_the_nerd - Friday, April 10, 2015 - link

    Bought one of these (500GB) the other day for a family member's build. I thought I had read a review on AT already, but I probably got it mixed up with the MX100.

    Nevertheless, I was impressed with the performance, especially for the price. (I guess I lucked out.) Seems like the arguments in favor of the BX100 come down to:

    1) Cheap
    2) Fast enough
    3) Crucial

    I'm okay with that.

    Also, I ordered a V4 once upon a time, then read some reviews of it and cancelled the order. So I guess that's twice I've lucked out.
  • jabber - Friday, April 10, 2015 - link

    Bought one of the 250GB ones a few weeks ago.

    My review - Works just as well as any other SSD I've bought over the past two years. No disappointment.

    Erm that's about as much as you need to know.

Log in

Don't have an account? Sign up now