Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

Barefoot 3 has always done well in steady-state performance and the Vector 180 is no exception. It provides the highest average IOPS by far and the advantage is rather significant at ~2x compared to other drives.

Steady-State 4KB Random Write Consistency

But on the down side, the Vector 180 also has the highest variation in performance. While the 850 Pro, MX100 and Extreme Pro are all slower in terms of average IOPS, they are a lot more consistent and what's notable about the Vector 180 is how the consistency decreases as the capacity goes up. 

OCZ Vector 180 240GB
Default
25% Over-Provisioning

Looking at the scatter graph reveals the source of poor consistency: the IOPS reduce to zero or near zero even before we hit any type of steady state. This is known behavior of the Barefoot 3 platform, but what's alarming is how the 480GB and 960GB drives frequently drop to zero IOPS. I don't find that acceptable for a modern high-end SSD, no matter how good the average IOPS is. Increasing the over-provisioning helps a bit by shifting the dots up, but it's still clear that 240GB is the optimal capacity for Barefoot 3 because after that the platform starts to run into issues with consistency due to metadata handling.

OCZ Vector 180 240GB
Default
25% Over-Provisioning
SSD Guru: The New OCZ Toolbox AnandTech Storage Bench - The Destroyer
POST A COMMENT

89 Comments

View All Comments

  • Shark321 - Wednesday, March 25, 2015 - link

    Tosh, it's a pity PFM does not work on the internal cache of the drive. You can still get file system damage during a power loss. Reply
  • AVN6293 - Sunday, December 20, 2015 - link

    Does this drive support Opal 2.0 eDrive (FIPS/Hippa compliance) ? Reply
  • AVN6293 - Sunday, December 20, 2015 - link

    ...And can the over provisioning be increased by the user ? Reply
  • ats - Wednesday, March 25, 2015 - link

    Actually, all consumer drives need power loss protection and they realistically need it much more than drives targeted at the actual enterprise side of the market. It comes down to simple probabilities. The average enterprise SSD is going to be backed by at least 1 additional layer of power loss prevention (UPS et al), have a robust backup infrastructure, and likely mirroring (offsite) on top.

    In contrast, consumer drives are unlikely to have any power loss prevention, unlikely to have anything approaching a backup infrastructure, and highly unlikely to have robust data resiliency(offsite mirroring et al).

    So like many others, Anandtech gets it exactly wrong wrt PLP and SSDs. The fact that manufacturers have been able to get away without providing PLP on consumer SSDs is almost criminal. The fact that review sites accept this as perfectly OK is pretty much criminal on their part.

    And what should pretty much be a rage storm for consumers is the actual cost of providing PLP on an SSD is literally a couple of $ in capacitors. Not to mention many consumer drives without PLP have enterprise drives using the exact same PCB with PLP. That we as consumers have allowed companies to have PLP as a point of differentiation is to our great detriment, esp when the actual cost of PLP is in the noise even for cheap low capacity SSDs.

    If a drive cannot survive a power loss with data integrity then it certainly shouldn't get a recommendation nor should any consumer even consider it.
    Reply
  • Shiitaki - Wednesday, March 25, 2015 - link

    You do raise some very good points. I think the enterprise still needs it because they want as many ways to protect the data that they can get, after all it's only a couple of bucks. The consumer would benefit to a greater degree since that is likely all they would have is the caps in the SSD. However the consumer is their own worse enemy, a couple of bucks makes a difference for most consumers.

    I've had no issues, and until I read this article, gave no thought to pulling power on a system using an SSD! And I've done it ALOT! Not a single bad block yet! And that is with 6 SSD's in various machines from 4 manufacturers and 8 product lines. Though none of them with Windows, all Linux and OS X.

    Sometimes I wonder just how wide spread issues really are. On the internet it's hard to tell since it's the angry people doing most of the posting.

    In the end though whether you area company or individual, if it isn't backed up. you really don't need it.
    Reply
  • trparky - Wednesday, April 29, 2015 - link

    I do an image of my system SSD every week and my computer is always plugged into a UPS, and yes, that's my home setup. The power is my area is known to be dirty power, not complete drop-outs but if you measured the voltage output it would make most electrical engineers shake their heads and smack their foreheads. Reply
  • zodiacsoulmate - Tuesday, March 24, 2015 - link

    also the Arconis 2013 is basically useless since it only runs on windows 7.... Reply
  • ocztosh - Tuesday, March 24, 2015 - link

    Hi Zodiassoulmate, just wanted to confirm that the Vector 180 drives are shipping with Acronis 2014. Reply
  • DanNeely - Tuesday, March 24, 2015 - link

    That's a step in the right direction; but is still last years product. Acronis 2015 is already out. Am I overly cynical for thinking Acronis offered the 2014 version at a discount hoping to make it up by convincing some of the SSD buyers to upgrade to the new version after installing? Reply
  • Samus - Tuesday, March 24, 2015 - link

    Acronis TrueImage 2015 is complete shit. Check the Acronis forums: most people (like myself - a paying annual customer since 2010) have gone back to 2014. The most recent update (October) still did not fix issues with image compatibility, GPT partition compatibility (added for 2015) and UEFI boot mapping. Aside from the lingering compatibility, reliability and stability issues, the interface is terrible. They've basically turned it into a backup product for single PC's instead of a imaging product. Even the USB bootable ISO I typically boot off a flash drive for imaging/cloning is inherently unstable and occasionally even corrupts the destination. Nobody has confirmed the "Universal Restore" works for Windows 7, yet another broken feature that worked FINE in 2014.

    Acronis lost me as a long-time customer to Miray because 2015 was SO botched and after waiting months for them to fix it, I gave up and had to find a product that could adequately clone UEFI OS's installed on GPT partitions. I use this product almost daily to upgrade PC's to SSD's. Unfortunately Miray's boot environment is a little slower, even with the verification disabled and "fast copy" turned on, likely because it runs a different USB stack.

    I don't blame OCZ for sticking with 2014 like every other Acronis licensee has, including Crucial and Intel. 2014 is mature and stable, but it is not the modern solution - especially with Windows 10 around the corner. Acronis will forfeit this market to Miray or in-house solutions like Samsungs' Clonemaster if they don't get their act together. It's just astonishing how well Acronis was doing until 2015.
    Reply

Log in

Don't have an account? Sign up now