Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

Barefoot 3 has always done well in steady-state performance and the Vector 180 is no exception. It provides the highest average IOPS by far and the advantage is rather significant at ~2x compared to other drives.

Steady-State 4KB Random Write Consistency

But on the down side, the Vector 180 also has the highest variation in performance. While the 850 Pro, MX100 and Extreme Pro are all slower in terms of average IOPS, they are a lot more consistent and what's notable about the Vector 180 is how the consistency decreases as the capacity goes up. 

OCZ Vector 180 240GB
Default
25% Over-Provisioning

Looking at the scatter graph reveals the source of poor consistency: the IOPS reduce to zero or near zero even before we hit any type of steady state. This is known behavior of the Barefoot 3 platform, but what's alarming is how the 480GB and 960GB drives frequently drop to zero IOPS. I don't find that acceptable for a modern high-end SSD, no matter how good the average IOPS is. Increasing the over-provisioning helps a bit by shifting the dots up, but it's still clear that 240GB is the optimal capacity for Barefoot 3 because after that the platform starts to run into issues with consistency due to metadata handling.

OCZ Vector 180 240GB
Default
25% Over-Provisioning
SSD Guru: The New OCZ Toolbox AnandTech Storage Bench - The Destroyer
POST A COMMENT

89 Comments

View All Comments

  • nils_ - Wednesday, March 25, 2015 - link

    I distinctly remember that when I replaced a SSD in my workstation the Acronis tool, instead of copying my data from the source SSD to the new SSD copied the data to another, unrelated HDD in the system, happily overwriting the Linux partitions stored thereon... I had to unplug everything from the mainboard safe for the old and new SSD to make sure that it doesn't destroy any more of my data. Reply
  • MikeMurphy - Wednesday, March 25, 2015 - link

    Macrium Reflect is free and wonderful to use. Reply
  • mapesdhs - Wednesday, March 25, 2015 - link


    Yep, MR is what I use, it works very well and has a good interface.

    Ian.
    Reply
  • JonnyDough - Tuesday, March 31, 2015 - link

    Thirded. It works wonderfully. Reply
  • The_Assimilator - Tuesday, March 24, 2015 - link

    OCZ is still beating that Barefoot 3 dead horse for all it's worth. No wonder they went bankrupt. If you don't innovate, you die. Reply
  • ocztosh - Tuesday, March 24, 2015 - link

    Hello The_Assimilator. Thanks for your comments. Innovation is definitely a key area of focus for us on both the client and enterprise sides of our SSD business. We currently have a lot of resources put on next generation controllers and have been working hard on client SSDs leveraging the latest Toshiba NAND flash, which includes TLC. While some of these products are on the horizon it was natural for us to update our Vector Series with A19 NAND flash, and rather than just make a NAND change we wanted to add new features not normally found in our client class products like power fail management plus (PFM+) to further improve reliability in applications that blur the line between enthusiast and workstation.

    It is true that Barefoot 3 has, and continues to be, a very strong platform for us as we have shipped so many drives based on this in-house controller, and it has been so solid that we have not had to rev silicon a single time. We will continue to push to innovate when it comes to SSD performance, features and cost and are committed to delivering more value for all our customers. Thanks again for your feedback.
    Reply
  • Minion4Hire - Tuesday, March 24, 2015 - link

    The bigger problem is that the SSD market has stagnated. When their weren't major players (like Samsung) the little guy could cobble something together and make a decent living filling what was a niche market. Now SSDs are mainstream, and through volume alone the big guys can overpower the little guys, let alone R&D, etc. Until the market shifts away from SATA there's no room for niche innovation or clever advances. It's all down to margins right now. Reply
  • Samus - Tuesday, March 24, 2015 - link

    You do realize the two best-selling controllers are Marvell and Sandforce, and both are older than Barefoot 3. The SF2281 is 3+ years old and still ships in a variety of Mushkin, ADATA, Intel, etc SSD's.

    Barefoot 3 is thoroughly modern, but it does lack some power saving and drive encryption features. I don't think it makes sense for OCZ to update it when PCIe is around the corner and will require a new controller since a bridge will be expensive and not much faster.

    Aside from that, Barefoot 3 is incredibly innovative. I consider it the best controller available aside from Intel's 3rd gen controller (equally as old) and Samsung's MEX. ASMedia is still a little inconsistent and featureless, Marvell has an aging indirection table implementation that yields average performance, Phison is clearly entry-level with relatively low performance and consistency, Silicon Motion and Fusion-IO are power hungry, have quirks with certain NAND varieties and are not cost competitive.
    Reply
  • Guspaz - Tuesday, March 24, 2015 - link

    "There's an old saying in Tennessee - I know it's in Texas, probably in Tennessee - that says, fool me once, shame on - shame on you. Fool me - you can't get fooled again."

    Yeah, George, I'm not planning on getting fooled by OCZ again.
    Reply
  • blue_urban_sky - Tuesday, March 24, 2015 - link

    Is that the saying from Tennessee where they cant't remember the proper saying? Reply

Log in

Don't have an account? Sign up now