Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

Barefoot 3 has always done well in steady-state performance and the Vector 180 is no exception. It provides the highest average IOPS by far and the advantage is rather significant at ~2x compared to other drives.

Steady-State 4KB Random Write Consistency

But on the down side, the Vector 180 also has the highest variation in performance. While the 850 Pro, MX100 and Extreme Pro are all slower in terms of average IOPS, they are a lot more consistent and what's notable about the Vector 180 is how the consistency decreases as the capacity goes up. 

OCZ Vector 180 240GB
Default
25% Over-Provisioning

Looking at the scatter graph reveals the source of poor consistency: the IOPS reduce to zero or near zero even before we hit any type of steady state. This is known behavior of the Barefoot 3 platform, but what's alarming is how the 480GB and 960GB drives frequently drop to zero IOPS. I don't find that acceptable for a modern high-end SSD, no matter how good the average IOPS is. Increasing the over-provisioning helps a bit by shifting the dots up, but it's still clear that 240GB is the optimal capacity for Barefoot 3 because after that the platform starts to run into issues with consistency due to metadata handling.

OCZ Vector 180 240GB
Default
25% Over-Provisioning
SSD Guru: The New OCZ Toolbox AnandTech Storage Bench - The Destroyer
Comments Locked

89 Comments

View All Comments

  • sfc - Wednesday, March 25, 2015 - link

    Fool me once, shame on, shame on me. A fool me can't get fooled again.

    I *KNEW* OCZ was a garbage company after all the havoc they caused in the late 90s/early 00s with their crap memory products. It sounded alarm bells in my head that reminded me of their fake address that was literally an empty storefront.

    But I read all the press, heard how it was just the same name but a different backing company. So I bought one of their SSDs like a fool, only to send it back multiple times and everytime have it die again. I still have an 80GB intel SSD I bought several years before the OCZ that's still kicking.

    After all that, I find out the same crook selling crap memory was behind the "new" OCZ. Pulling his same parlor tricks giving review sites hand-picked models and sending bottom barrel reject flash to customers. You should just refuse to review any more of their hardware, they're crooks and their wares are trash.
  • Jahzah_1 - Wednesday, March 25, 2015 - link

    Just picked up a Samsung 850 Evo for $204 at Microcenter (price matched to Newegg), last Saturday. Don't understand the justification by OCZ to price the 480GB version at $275.
  • Jahzah_1 - Wednesday, March 25, 2015 - link

    That is $204 for the 500GB Evo.
  • mapesdhs - Wednesday, March 25, 2015 - link

    As I understand the intent of this product, it's aimed at the likes of the Extreme Pro
    and 850 Pro. The Arc 100 is the mainstream competitor to the EVO, which is $196
    on newegg for the 480GB.

    Ian.
  • Jahzah_1 - Thursday, March 26, 2015 - link

    oh, I see. What I didn't take into account was the fairly inexpensive nature of 3D-Nand production. So Samsung has an edge it seems, to set their mid-range drives at that price.
  • rocketman122 - Thursday, March 26, 2015 - link

    Ive had nothing but bad experience with OCZ. I had mem go bad, rma and sold them. I had the core 64 SSD that the company knew were problematic and still didnt have integrity to not sell them. that core SSD cost me quite chunk of money and I suffered with that. I never for

    I hope they go out of business and stop selling their gear to the public. we need companies with reliable gear.
  • ocztosh - Monday, March 30, 2015 - link

    Hello rocketman122, thank you for your comments and sorry to hear that you had issues with the Core Series. The old company no longer exists and the IP was purchased by Toshiba. As OCZ Storage Solutions - A Toshiba Group Company we have completely redone our products and processes and there has been a great focus on quality throughout the organization. Everything from the product design cycle through manufacturing has been updated. By implementing our own in-house controller and firmware technology and having access to premium Toshiba NAND we are now able to impact this better than ever. We believe we have a very competitive offering today when it comes to reliability and product quality and hope that we will have the opportunity to prove it to you in the future. Thank you again for your previous business.
  • loimlo - Thursday, March 26, 2015 - link

    Good SSD review as usual. Kudos to Kristian's efforts.
    Btw, I wonder the M-I-A BX100 review. Can we expect it ?
  • Kristian Vättö - Friday, March 27, 2015 - link

    It's coming. This month has been full of NDAs, which have postponed the BX100 review, but once I'm done with next week's NDAs the BX100 will be getting my full attention :)
  • loimlo - Saturday, March 28, 2015 - link

    Thanks for clarification. Take your time to do it ~~

Log in

Don't have an account? Sign up now