Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

Barefoot 3 has always done well in steady-state performance and the Vector 180 is no exception. It provides the highest average IOPS by far and the advantage is rather significant at ~2x compared to other drives.

Steady-State 4KB Random Write Consistency

But on the down side, the Vector 180 also has the highest variation in performance. While the 850 Pro, MX100 and Extreme Pro are all slower in terms of average IOPS, they are a lot more consistent and what's notable about the Vector 180 is how the consistency decreases as the capacity goes up. 

OCZ Vector 180 240GB
Default
25% Over-Provisioning

Looking at the scatter graph reveals the source of poor consistency: the IOPS reduce to zero or near zero even before we hit any type of steady state. This is known behavior of the Barefoot 3 platform, but what's alarming is how the 480GB and 960GB drives frequently drop to zero IOPS. I don't find that acceptable for a modern high-end SSD, no matter how good the average IOPS is. Increasing the over-provisioning helps a bit by shifting the dots up, but it's still clear that 240GB is the optimal capacity for Barefoot 3 because after that the platform starts to run into issues with consistency due to metadata handling.

OCZ Vector 180 240GB
Default
25% Over-Provisioning
SSD Guru: The New OCZ Toolbox AnandTech Storage Bench - The Destroyer
POST A COMMENT

89 Comments

View All Comments

  • KAlmquist - Tuesday, March 24, 2015 - link

    Vector 180 vx. MX 100:

    I think that the Crucial MX 100 will be a bit faster than the Vector 180 under typical usage, though the Vector 180 does outperform the MX 100 on some benchmarks. Both drives have partial power loss protection. The Vector 180 has a 5 year warranty vs. the 3 year warranty on the MX 100, but a lot of people will be looking to upgrade from SATA to PCIe SSD's before the 3 year warranty expires. In short, I don't see any reason to pay a premium price for the Vector 180.
    Reply
  • MrCommunistGen - Tuesday, March 24, 2015 - link

    If only HIPM+DIPM worked (yes I know its a hardware limitation of the platform) this looks like it'd be a great laptop SSD due to such low power consumption in the various workloads measured here. Reply
  • djsvetljo - Tuesday, March 24, 2015 - link

    Two dead OCZ drives - there will never be a 3rd one. One of the drives was from the era of the bitcoins boom ( when they were easy to mine). Lost 150 coins there ( that's over $50 000). They stupid thing locked up due to power issues ( too many power cycles).

    STAY AWAY FROM OCZ
    Reply
  • Antronman - Wednesday, March 25, 2015 - link

    Well the thing is, the internals are drastically different right now.

    I've been in the market for a good SSD for a while. These drives seem to perform well. Sadly outperformed by the 850 Pro drives, but I do think that the enclosure is actually very aesthetically pleasing. If the cost could be driven down a lot, I'd be very interested in the 180.
    Reply
  • mapesdhs - Wednesday, March 25, 2015 - link

    You're right, they need to have a price advantage with the 180 to pull people in. Also,
    availability needs to be good - in the past it's been rather sketchy with the 150, which
    means prices tend to creep up from a small number of suppliers.

    Ian.
    Reply
  • mapesdhs - Wednesday, March 25, 2015 - link

    More than 30 OCZ drives, all working fine. Some had bad luck, others used Marvell ports
    and blame OCZ. It varies.

    As for coins, well boohoo, not real money unless they're converted back to $.

    Ian.
    Reply
  • ocztosh - Monday, March 30, 2015 - link

    Hi mapesdhs, thank you for your feedback and great to hear the drives are working well for you. We greatly appreciate both your business and support. Reply
  • FalcomPSX - Tuesday, March 24, 2015 - link

    I've owned i think three OCZ drives in the past, they've all bricked on me within about a year. Never again will I purchase their shoddy products. While i was able to get warranty replacments each time, and the customer service is decent, the product itself is just not reliable in any way. I don't know if the new ones are improved at all, but i'm not risking MY data to find out. Reply
  • ocztosh - Wednesday, March 25, 2015 - link

    Hello FalcomPSX, thank you for your comments and sorry to hear that you has a problem with previous drives. We are a new organization under Toshiba and have made significant changes to everything from processes to production. We understand how you feel and hope that one day we will have the opportunity to demonstrate the reliability of our current products. Thank you again for your feedback Reply
  • NeatOman - Wednesday, March 25, 2015 - link

    Battery backup should be built into the PSU for workstations and servers, google does this for their data centers.. why shouldn't servers and workstations. All you need is 1 minute for the average workstation to go into hibernation until power is back on IMO. Reply

Log in

Don't have an account? Sign up now