Performance Consistency

In our Intel SSD DC S3700 review I introduced a new method of characterizing performance: looking at the latency of individual operations over time. The S3700 promised a level of performance consistency that was unmatched in the industry, and as a result needed some additional testing to show that. The reason we don't have consistent IO latency with SSDs is because inevitably all controllers have to do some amount of defragmentation or garbage collection in order to continue operating at high speeds. When and how an SSD decides to run its defrag and cleanup routines directly impacts the user experience. Frequent (borderline aggressive) cleanup generally results in more stable performance, while delaying that can result in higher peak performance at the expense of much lower worst case performance. The graphs below tell us a lot about the architecture of these SSDs and how they handle internal defragmentation.

To generate the data below I took a freshly secure erased SSD and filled it with sequential data. This ensures that all user accessible LBAs have data associated with them. Next I kicked off a 4KB random write workload at a queue depth of 32 using incompressible data. I ran the test for just over half an hour, no where near what we run our steady state tests for but enough to give me a good look at drive behavior once all spare area filled up.

I recorded instantaneous IOPS every second for the duration of the test. I then plotted IOPS vs. time and generated the scatter plots below. Each set of graphs features the same scale. The first two sets use a log scale for easy comparison, while the last set of graphs uses a linear scale that tops out at 40K IOPS for better visualization of differences between drives.

The first set of graphs shows the performance data over the entire 2000 second test period. In these charts you'll notice an early period of very high performance followed by a sharp dropoff. What you're seeing in that case is the drive alllocating new blocks from its spare area, then eventually using up all free blocks and having to perform a read-modify-write for all subsequent writes (write amplification goes up, performance goes down).

The second set of graphs zooms in to the beginning of steady state operation for the drive (t=1400s). The third set also looks at the beginning of steady state operation but on a linear performance scale. Click the buttons below each graph to switch source data.


              

Here we see a lot of the code re-use between the Vector and Vertex 4 firmware. Vector performs like a faster Vertex 4, with all of its datapoints shifted up in the graph. The distribution of performance is a bit tighter than on the Vertex 4 and performance is definitely more consistent than the 840 Pro. The S3700 is obviously in a league of its own here, but I do hope that over time we'll see similarly consistent drives from other vendors.

The next set of charts look at the steady state (for most drives) portion of the curve. Here we'll get some better visibility into how everyone will perform over the long run.


              

The source data is the same, we're just focusing on a different part of the graph. Here the Vector actually looks pretty good compared to all non-S3700 drives. In this case the Vector's performance distribution looks a lot like SandForce. There's a clear advantage again over the 840 Pro and Vertex 4.

The final set of graphs abandons the log scale entirely and just looks at a linear scale that tops out at 40K IOPS. We're also only looking at steady state (or close to it) performance here:


              

If we look at the tail end of the graph with a linear scale, we get a taste of the of just how varied IO latency can be with most of these drives. Vector looks much more spread out than the Vertex 4, but that's largely a function of the fact that its performance is just so much higher without an equivalent increase in aggressive defrag/GC routines. The 840 Pro generally manages lower performance in this worst case scenario. The SandForce based Intel SSD 330 shows a wide range of IO latencies but overall performance is much better. Had SandForce not been plagued by so many poorly handled reliability issues it might have been a better received option today.

From an IO consistency perspective, the Vector looks a lot like a better Vertex 4 or 840 Pro. Architecturally I wouldn't be too surprised if OCZ's method of NAND mapping and flash management wasn't very similar to Samsung's, which isn't a bad thing at all. I would like to see more emphasis placed on S3700-style IO consistency though. I do firmly believe that the first company to deliver IO consistency for the client space will reap serious rewards.

Performance vs. Transfer Size AnandTech Storage Bench 2011
POST A COMMENT

151 Comments

View All Comments

  • wsaenotsock - Tuesday, November 27, 2012 - link

    A 5 year warranty is a pretty solid commitment on the part of a manufacturer. I don't think they would have done that if they didn't trust the stability of the hardware, so they really put their money where their mouth is.

    Other thing: is the Indilinx co-processor 'Argon' or 'Aragon'? Pic differs from your text description.
    Reply
  • alacard - Wednesday, November 28, 2012 - link

    Nah, you've got it all wrong unfortunately - they've bet the farm on this drive and if it fails they won't be around in five years to honor those warranties.

    When you've got nothing, you've got nothing to lose.
    Reply
  • Kjella - Wednesday, November 28, 2012 - link

    Well that, but I'm glad to see OCZ committing more to their drives... on my local price price check there's Agility, Colossus, Enyo, Ibis, Lightfoot, Octane, Onyx, Petrol, RevoDrive, Synapse, Vertex and Z-drive not counting numbering or variations like Vertex EX, Vertex Limitied Edition, Vertex Turbo and using a zillion different controllers and stuff. The warranty is also an indication this is the technology they'll continue working on and fixing bugs for, which is good because their attention span has been too short and spread too thin. It's better with a few models that kick ass than dozens of models that are all shoddy. Reply
  • MrSpadge - Wednesday, November 28, 2012 - link

    Some of the drives you list are several years old.. Reply
  • JonnyDough - Friday, November 30, 2012 - link

    Furthermore, the Intel 520 (which I just purchased) got dropped off the enterprise iometer 4KB random write chart. That don't make a lick-a sense! Reply
  • Hood6558 - Wednesday, November 28, 2012 - link

    alacard may be right, OCZ is sliding closer to the cliff as we speak. There's so much competition in the SSD market, someone's got to go sooner or later, and it will probably be the less diversified companies that will go first. I recently bought a Vertex 4 128 for my boot drive, and it lasted only 15 days before it disappeared and refused to be recognized in BIOS. The Crucial M4 128 that replaced it has the problem of disappearing every time the power is shut off suddenly (or with the power button after Windows hangs), but comes back after a couple of reboots and a resetting of your boot priorities. And it's regarded as one of the most reliable drives out there.
    So in order for OCZ to remain solvent, the Vector must be super reliable and stable, and absolutely must stay visible in BIOS at all times. If it's plagued by the same problems as the Vertex 4, it's time to cash out and disappear before the bankruptcy court has it's way.
    Reply
  • Sufo - Tuesday, December 04, 2012 - link

    Windows hanging? I smell problems with the user... Reply
  • djy2000 - Wednesday, July 31, 2013 - link

    That warranty doesn't cover the most important thing. The data on the drive. Reply
  • Bull Dog - Tuesday, November 27, 2012 - link

    Is that an m-SATA connector on the other side of the PCB? Reply
  • philipma1957 - Tuesday, November 27, 2012 - link

    good question if I open the case can i use this as a msata?

    a 512gb msata is very hard to find.
    Reply

Log in

Don't have an account? Sign up now