Performance Consistency

In our Intel SSD DC S3700 review I introduced a new method of characterizing performance: looking at the latency of individual operations over time. The S3700 promised a level of performance consistency that was unmatched in the industry, and as a result needed some additional testing to show that. The reason we don't have consistent IO latency with SSDs is because inevitably all controllers have to do some amount of defragmentation or garbage collection in order to continue operating at high speeds. When and how an SSD decides to run its defrag and cleanup routines directly impacts the user experience. Frequent (borderline aggressive) cleanup generally results in more stable performance, while delaying that can result in higher peak performance at the expense of much lower worst case performance. The graphs below tell us a lot about the architecture of these SSDs and how they handle internal defragmentation.

To generate the data below I took a freshly secure erased SSD and filled it with sequential data. This ensures that all user accessible LBAs have data associated with them. Next I kicked off a 4KB random write workload at a queue depth of 32 using incompressible data. I ran the test for just over half an hour, no where near what we run our steady state tests for but enough to give me a good look at drive behavior once all spare area filled up.

I recorded instantaneous IOPS every second for the duration of the test. I then plotted IOPS vs. time and generated the scatter plots below. Each set of graphs features the same scale. The first two sets use a log scale for easy comparison, while the last set of graphs uses a linear scale that tops out at 40K IOPS for better visualization of differences between drives.

The first set of graphs shows the performance data over the entire 2000 second test period. In these charts you'll notice an early period of very high performance followed by a sharp dropoff. What you're seeing in that case is the drive alllocating new blocks from its spare area, then eventually using up all free blocks and having to perform a read-modify-write for all subsequent writes (write amplification goes up, performance goes down).

The second set of graphs zooms in to the beginning of steady state operation for the drive (t=1400s). The third set also looks at the beginning of steady state operation but on a linear performance scale. Click the buttons below each graph to switch source data.


              

Here we see a lot of the code re-use between the Vector and Vertex 4 firmware. Vector performs like a faster Vertex 4, with all of its datapoints shifted up in the graph. The distribution of performance is a bit tighter than on the Vertex 4 and performance is definitely more consistent than the 840 Pro. The S3700 is obviously in a league of its own here, but I do hope that over time we'll see similarly consistent drives from other vendors.

The next set of charts look at the steady state (for most drives) portion of the curve. Here we'll get some better visibility into how everyone will perform over the long run.


              

The source data is the same, we're just focusing on a different part of the graph. Here the Vector actually looks pretty good compared to all non-S3700 drives. In this case the Vector's performance distribution looks a lot like SandForce. There's a clear advantage again over the 840 Pro and Vertex 4.

The final set of graphs abandons the log scale entirely and just looks at a linear scale that tops out at 40K IOPS. We're also only looking at steady state (or close to it) performance here:


              

If we look at the tail end of the graph with a linear scale, we get a taste of the of just how varied IO latency can be with most of these drives. Vector looks much more spread out than the Vertex 4, but that's largely a function of the fact that its performance is just so much higher without an equivalent increase in aggressive defrag/GC routines. The 840 Pro generally manages lower performance in this worst case scenario. The SandForce based Intel SSD 330 shows a wide range of IO latencies but overall performance is much better. Had SandForce not been plagued by so many poorly handled reliability issues it might have been a better received option today.

From an IO consistency perspective, the Vector looks a lot like a better Vertex 4 or 840 Pro. Architecturally I wouldn't be too surprised if OCZ's method of NAND mapping and flash management wasn't very similar to Samsung's, which isn't a bad thing at all. I would like to see more emphasis placed on S3700-style IO consistency though. I do firmly believe that the first company to deliver IO consistency for the client space will reap serious rewards.

Performance vs. Transfer Size AnandTech Storage Bench 2011
Comments Locked

151 Comments

View All Comments

  • Sabresiberian - Tuesday, November 27, 2012 - link

    So, a longer warranty is only good if you use it? Otherwise you're paying for something you don't need?

    And, you're paying extra for a 5-year warranty here? What, so all these top end SSDs, whose prices are lower than ever, are in fact over-priced with fake expensive warranties, so should come out with 1-year warranties and lower prices?
  • coder543 - Tuesday, November 27, 2012 - link

    a refurbished SSD? I'm not even sure what that means. That's like going to McDonald's and getting a refurbished McFlurry. It doesn't even make sense.

    This isn't a laptop, where worn parts can be replaced. This is a limited lifespan, consumable product, where replacing any parts is equivalent to throwing the old one away and pulling out a brand new one. If the warranty actually says this, then please, point me to it, but otherwise, I'm gonna have to call this bluff and say it's not practical.
  • Beenthere - Wednesday, November 28, 2012 - link

    The point that some of you seem to not understand is that the 5 year warranty does NOT mean that an SSD or other product is any better quality than a product with a one year warranty. And yes you are paying for the extended warranty no matter what the current price. SSD prices are dropping as cost to produce them is dropping. This particular OCZ model is not a highend model by any stretch, it's just the SSD-of-the-week to be superceded by a new model in a month or two.

    Refurbished can mean hand soldered chip replacement or other poorly executed repairs that would not be acceptable to most technically knowledgeable consumers. Reconditioned can mean it's been laying in the warehouse collecting dust for six months and nothing was actually done to repair it when it was returned defective. You would not believe some of the crap that ships as replacement warranty products.
  • zero2dash - Wednesday, November 28, 2012 - link

    ^^^ I'm with Beenthere.
    A 5 year warranty means a 5 year warranty; nothing more nothing less. The notion that '5 year warranty = great product!' is asinine.

    I think if you want to assume anything based off a 5 year warranty in this case, it's because the product is new, the controller is relatively new, and it's an OCZ SSD product.

    I'm not likely to buy an OCZ SSD anytime soon, but I'd definitely rather buy one with a 5 year warranty than a 1 or 3 year warranty....if I have to buy an OCZ branded SSD because every other brand is sold out.

    I owned a 30GB Vertex. For 9 months, it was great. Then it turned into a big POS. Constant chkdsk errors. I did a sanitary erase/firmware flash and sold it for what I could get for it.
  • melgross - Wednesday, November 28, 2012 - link

    I certainly would not want a refurbished SSD. It would NOT mean new NAND chips, which are the parts most likely to be a problem. Or a new controller. I would never buy a refurbished HDD either. These devices do have lifetimes. Since you have no idea how these drives have been used, or abused, you are taking a very big chance for the dubious opportunity of saving a few bucks.
  • Hood6558 - Wednesday, November 28, 2012 - link

    I can't help but wonder how many replacement SSDs it will take to get to the end of that 5 year warranty. If you go by the track record of Vertex 3 & 4, you can expect a failure about every 90 days, so that's 20 drives, less shipping time to and from, so call it 15 drives with a total downtime of 1.25 years. Wow!.. where can I get one? My Vertex 4 lasted 15 days, but I'm sure that was just a fluke...
  • melgross - Wednesday, November 28, 2012 - link

    I basically agree. From anecdotal reports, OCZ is one of the least reliable vendors, with their drives less reliable than the average HDD. While, so far, the average SSD reliability being about the same as the average HDD, despite people's expectations, this isn't good.

    Most people don't need the really high speeds a few of these drives support, higher reliability would be a much better spec to have. Unfortunately, these reviews can't indicate how reliable these drives will be in the longer term.

    While I see that OCZ seems to be thought of as failing, this is the first I've heard of it. Have their sales collapsed of late? I was surprised to find that their long time CEO whom Anand had communicated so often with in the past is gone.
  • Spunjji - Wednesday, November 28, 2012 - link

    "FYI- A five year warranty ain't worth the paper it's written on if the company no longer exists." <- Depends on how you purchase it. Credit card companies will often honour warranties on products purchased from defunct companies. YMMV.

    "Most state that you will receive a refurbished or reconditioned replacement if your product develops a defect." <- Happily now everyone in the thread after you has used this conjecture to knock OCZ warranties. That's not really your fault, but I don't think anyone here has read the terms of OCZ's warranty on this product yet?

    The point being made here is that OCZ would not offer a 5 year warranty on the product if they thought the cost of honouring that warranty would eclipse their income from sales. This is why 1-year warranties are a red flag. So *something* can be inferred from it; just only about the manufacturer's confidence in their product. You can read into that whatever you want, but I don't generally find that companies plan to be out of business within their warranty period.

    Your comment about it increasing the price of the product is odd, because this product is the same price and specification as models with shorter warranties. So either a) you're wrong, or b) you're trivially correct.
  • JonnyDough - Friday, November 30, 2012 - link

    Here here. I second that. I am so tired of getting worn refurbished parts for things I just bought BRAND NEW. CoolerMaster just did this for a higher end power supply I bought. Why would I want to spend a hundred dollars for a used PSU? Seriously. Now all the components aren't new in it. Once the warranty expires it'll die right away. Where is the support behind products these days?

    It used to be that buying American meant you got quality and customer service. Gone are those days I guess, since all the corporations out there are about to start actually paying taxes.
  • smalM - Tuesday, November 27, 2012 - link

    (e.g. a 256GB Vector appears formatted as a 238GB drive in Windows).

    Oh please Anand, the old "formatted" nonsens of all people from you?
    You really should drop this sentence from your phrase list....

Log in

Don't have an account? Sign up now