AnandTech Storage Bench 2011: Much Heavier

I didn't expect to have to debut this so soon, but I've been working on updated benchmarks for 2011. Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. I assembled the traces myself out of frustration with the majority of what we have today in terms of SSD benchmarks.

Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.

Originally I kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system.

Not too long ago I tweeted that I had created what I referred to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. It's the load you'd put on a drive after nearly two weeks of constant usage. And it takes a *long* time to run.

I'll be sharing the full details of the benchmark in some upcoming SSD articles but here are some details:

1) The MOASB, officially called AnandTech Storage Bench 2011 - Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. My thinking was that it's during application installs, file copies, downloading and multitasking with all of this that you can really notice performance differences between drives.

2) I tried to cover as many bases as possible with the software I incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II & WoW are both a part of the test) as well as general use stuff (application installing, virus scanning). I included a large amount of email downloading, document creation and editing as well. To top it all off I even use Visual Studio 2008 to build Chromium during the test.

Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result I'm going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time I'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, I will also break out performance into reads, writes and combined. The reason I do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.

There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. Lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.

As always I don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea.

The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests. All of the older tests are still run on our X58 platform.

AnandTech Storage Bench 2011 - Heavy Workload

We'll start out by looking at average data rate throughout our new heavy workload test:

AnandTech Storage Bench 2011 - Heavy Workload

Overall performance is decidedly last generation. The 320 is within striking distance of the 510 but is slower overall in our heavy workload test.

The breakdown of reads vs. writes tells us more of what's going on:

AnandTech Storage Bench 2011 - Heavy Workload

AnandTech Storage Bench 2011 - Heavy Workload

The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles, this is just how long the SSD was busy doing something:

AnandTech Storage Bench 2011 - Heavy Workload

AnandTech Storage Bench 2011 - Heavy Workload

AnandTech Storage Bench 2011 - Heavy Workload

Random & Sequential Performance AnandTech Storage Bench 2011 - Light Workload
Comments Locked

194 Comments

View All Comments

  • sean.crees - Tuesday, March 29, 2011 - link

    Have you thought that perhaps the new change was in your best interest? That maybe their trying to fix a problem before you notice it.

    Think about it, as a company, you would want to give your customers what they want. You'd give them all the space they'd want, i mean why not right? You'd want all the biggest numbers possible to slap on your box so you can brag to the entire market your the best. Then the only reason to make it less is to pre-emptively keep a potential problem from happening.

    You have to remember, OCZ isn't Intel, they don't make these parts. They just put it together and repackage it. If their Nand suppliers suddenly have issues, they can't just stop selling products, they would go bankrupt, but a slight change in product specs that most people will never notice in real world usage to prevent future issues because of hardware supply problems?

    I'm not saying this is what happened, but it is just one of many possible reasons why the change was made.

    OCZ doesn't have intels Q&A because they can't afford it. What would you prefer, having only Intel SSD's because no one else can afford to be competative with them in this market and have the same Q&A and parts. Or some competition with some sacrifices? Most would agree this is the lesser of two evils.

    But what OCZ lacks in funding they make up in customer support. You can go to their forums, make a post and within a reasonable amount of time get an actual tech from their company to respond to you with options to fix your problem. You don't get that personal touch from Intel. With Intel, if it's not in the box when you buy it, then you don't get it.
  • jwilliams4200 - Wednesday, March 30, 2011 - link

    Lowering the performance of the product, and giving 5GB less space than advertised (or using lower tier flash memory) is NOT in the best interest of the customer. It was in OCZ's best interest to try to deceive their customers so OCZ could increase its profits, so that is what OCZ did.
  • sean.crees - Wednesday, March 30, 2011 - link

    And you have proof they intentionally used sub standard nand to increase profits? Or are you just making stuff up?
  • jwilliams4200 - Wednesday, March 30, 2011 - link

    OCZ has been using flash chips in some SSDs that have the Micron logo and then the SpecTek logo stamped on top. SpecTek sells flash chips that Micron will not sell. SpecTek is a lower tier.

    Making things up? Hardly. This has been discussed at length in a number of forums, and you can easily find credible pictures of the flash chips in OCZ SSDs from people who opened up the SSDs.
  • Ema Nymton - Tuesday, April 5, 2011 - link

    Funny you say that,
    I just had the same discussion with a friend of mine.
    Then I went with a OCZ deneva and opened it.
    Guest what I found Intel Nan flash in it...
  • sean.crees - Tuesday, March 29, 2011 - link

    Intel had the reliability vs speed in the 1st and 2nd gen SSD's, but the difference then was that the speed was competitive. It's not even close this time.

    Another thing i have against Intel, i was one of their early 1st gen adopters, 2x 80gb drives for $500 a pop. I'm the guy Intel screwed out of TRIM support because they didn't feel like sharing a firmware update that enables it for me. So i still have a bitter taste in my mouth over that, especially now that its been a few years and my 1st gen SSD's are starting to feel more like a fast HDD. :*(

    Also, Sandforce and OCZ seem to have learned how important stability and performance sustainability are in this market. They have some of the best garbage collection without TRIM available. That's important especially for people considering OS's that don't support TRIM, or for RAID usage.

    Intel may have created this market, but its competitors have learned the lessons Intel shared while Intel still needs to learn how to support its existing customer base like OCZ has gotten very good at. This is the problem with being #1 for too long, you begin to think you can do no wrong, and all your choices are the right ones that everyone else should conform to. This is fine as long as you keep your competitive edge, but once you lose that, you risk losing everything.

    Intel may be able to brag about stability for now, but i have a feeling as time goes by people will realize that Sandforce drives are just as stable, and then Intel loses its only Ace left in it's hand.
  • jwilliams4200 - Tuesday, March 29, 2011 - link

    You sound like an OCZ spin doctor.

    No, the Sandforce garbage collection is not one of the best. It is actually highly flawed. If you use the drive heavily, the write speed will drop, and it will stay low, even after TRIM, for up to a week. And Sandforce has done nothing to fix this. OCZ, in typical con-man fashion, claims it is a feature, but that is obviously a lie. If it were a feature, it would be documented under what circumstances it happens, how much the write speed decreases, and for how long. The write speed would drop more, too, since 30% drop in write speed is not going to do much to save the flash, even if the flash were really in danger, which it should not be. And if it were a feature, Sandforce would offer the ability to turn it off, if not by the user, at least by the vendor. No, it is a design flaw.

    Even if OCZ users do not get deceived by OCZ in the future (by OCZ changing the internals to make the drive less reliable or perform worse), it is obvious that Intel will remain more reliable and stable than OCZ. There is just no comparison between the quality control and testing that happens at Intel, as an entire corporate culture, in contrast to OCZ, where the corporate culture is to take short-cuts, use the cheapest parts possible, lie to the customers whenever they feel like it, and then send the spin doctors out on the forums to attempt damage control. Ever since the OCZ spin doctors pretended to be users on hardforum years ago, it has been clear what sort of company OCZ is.
  • Acrono - Tuesday, March 29, 2011 - link

    You sound like an Intel spin doctor.
  • jwilliams4200 - Tuesday, March 29, 2011 - link

    I have no association with Intel, other than appreciating their reliability. But of course there is no way for you to know that. The difference is that OCZ has a history of having their people post on forums, posing as non-OCZ "regular people". Intel has no such shady history.
  • sean.crees - Tuesday, March 29, 2011 - link

    I've actually never owned an OCZ product ever. But i keep up with how different companies in the IT domain behave. I read reviews, and keep up with trends.

    "If you use the drive heavily", which the average consumer won't ever do. If you are going to be using this drive heavily then your making the mistake of buying a consumer product instead of a server product. You get what you pay for. Don't blame the company because your trying to use a product for a purpose it was never intended to be used for.

    It is not the culture of Intel that gives them better quality control, its capital. Intel has a lot more resources available than OCZ. It can afford these premium parts, and better quality control.

    Intel has good reliabilty, and if reliability is your #1 concern over all else, then your right, there is no other option. But most are willing to gamble that they will likely never see that increased reliability with intel, but will notice an increased speed with sandforce.

Log in

Don't have an account? Sign up now