AnandTech Storage Bench 2011

Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. Anand assembled the traces out of frustration with the majority of what we have today in terms of SSD benchmarks.

Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.

Originally we kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system. Later, however, we created what we refer to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. This represents the load you'd put on a drive after nearly two weeks of constant usage. And it takes a long time to run.

1) The MOASB, officially called AnandTech Storage Bench 2011—Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. Our thinking was that it's during application installs, file copies, downloading, and multitasking with all of this that you can really notice performance differences between drives.

2) We tried to cover as many bases as possible with the software incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II and WoW are both a part of the test), as well as general use stuff (application installing, virus scanning). We included a large amount of email downloading, document creation, and editing as well. To top it all off we even use Visual Studio 2008 to build Chromium during the test.

The test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:

AnandTech Storage Bench 2011—Heavy Workload IO Breakdown
IO Size % of Total
4KB 28%
16KB 10%
32KB 10%
64KB 4%

Only 42% of all operations are sequential; the rest ranges from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.

Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result we're going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time we'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, we will also break out performance into reads, writes, and combined. The reason we do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.

There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. It has lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback, as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.

We don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea. The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests.

AnandTech Storage Bench 2011—Heavy Workload

We'll start out by looking at average data rate throughout our new heavy workload test:

Heavy Workload 2011 - Average Data Rate

The SSD 335 performs very well in our Heavy suite. It's on-par with most high-end SSDs and even manages to beat the 520 and 330. The improvement over the 330 is quite good.

Heavy Workload 2011 - Average Read Speed

Heavy Workload 2011 - Average Write Speed

The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles, this is just how long the SSD was busy doing something:

Heavy Workload 2011 - Disk Busy Time

Heavy Workload 2011 - Disk Busy Time (Reads)

Heavy Workload 2011 - Disk Busy Time (Writes)

 

Performance vs. Transfer Size AnandTech Storage Bench 2011 - Light Workload
Comments Locked

69 Comments

View All Comments

  • meloz - Tuesday, October 30, 2012 - link

    >After Anandtech's 840 Pro review sample died, I'm not super inspired with confidence about the plain 840.

    Oops, I had almost forgotten about that, thanks for the reminder.

    Damn, all SSD manufacturers suck in their own ways.
  • jwilliams4200 - Thursday, November 1, 2012 - link

    I'm disappointed that there hasn't been any more information on that 840 Pro that died.

    Anand should really post some more details. Like what it was doing just before it died, the symptoms of how it failed, whether the SMART parameters could still be read, etc.

    Also, Anand should be hounding Samsung to get back to him about it, if they haven't already. The 840 Pro is apparently shipping on Nov 6. If Samsung has not been able to diagnose the problem and report back by then, it looks bad for Samsung.
  • Kristian Vättö - Thursday, November 1, 2012 - link

    Anand was filling the drive with sequential data (preconditioning it for our enterprise tests) and it just died in the middle of the run. After that it was no longer recognized in BIOS, not even when connected using USB to SATA adapter.

    As far as I know, Samsung has not gotten back to us about it yet but let me ask Anand and see if he knows more.
  • jwilliams4200 - Thursday, November 1, 2012 - link

    Thank you for the additional information.

    I am highly interested in what Samsung has to say about the failure. It seems to me that anandtech should be able to put some pressure on Samsung to give them a thorough failure analysis in a timely manner, or else anandtech will report that Samsung was unable to explain the failure and that looks bad for Samsung.
  • mmonnin03 - Tuesday, October 30, 2012 - link

    The flash on SSDs arent going to get more reliable. ECC basically scales exponentially as the process dimensions keep shrinking. As the lines get closer and closer, the number of electrons holding the charge becomes harder and harder to measure. Each cell is 2 bits, so 4 different amounts of electrons need to be measured. Errors occur more frequently and get fixed. And that's not going to make the NAND any faster by going smaller. SSD speed/reliability improvements will/have come at the controller level. If you truly want a reliable SSD, go 34nm SLC. Its still being produced.
  • jeffrey - Monday, October 29, 2012 - link

    Between the Samsung 840 Pro dying during testing and now the endurance issue testing Intel's 335, I believe OCZ should get some more credit for their Vertex 4.

    Everyone wants to mention Intel, Micron/Crucial, and Samsung for SSDs, but the Vertex 4 deserves to be there too.
  • josephjpeters - Monday, October 29, 2012 - link

    Endurance will play a larger role in differentiating future SSD's as the industry continues to move to smaller NAND geometries.

    I'm interested in seeing OCZ's Vector which will use 20nm MLC NAND. It'll be a big test for OCZ to see how their endurance technology stacks up against the competition (or lack thereof in the consumer space).
  • Samus - Monday, October 29, 2012 - link

    I've had my Intel SSD320 160GB for about two years, use it daily, and still have only written 10.6TB to it and the MWI is still 100%.

    Either way you look at it, this drive will last the average user a decade easily, even with less than 1000 P/E
  • IntelUser2000 - Tuesday, October 30, 2012 - link

    This is false: "Based on the data I gathered, the MWI would hit 0 after around 250TB of NAND writes, which translates to less than 1,000 P/E cycles."

    There is a forum that exclusively tests SSD endurance and Intel drives last far past the MWI of 0. In fact, after it reached zero, it started counting up. I remember the original X25-M lasting until the second MWI is significantly greater than 25(it could be 50 I don't remember).

    They thought that after the Media Wearout Indicator reached 0, the drive would die. In fact, none of the drives did. NONE.

    Even 240TB is hell of a lot. My X25-M has 7.6TB written to it and I had it since the year the drive was announced. At this rate, I'll be 30 years older by the time it reaches that point. So its a needless worry about nothing.

    Contrary to Platter HDDs, which die off slowly and more and more data gets corrupted and gets slower and slower until you notice that the drive is dying. Less than 5 years for lot of people around me too.
  • Kristian Vättö - Tuesday, October 30, 2012 - link

    I did not say the drive will die after the MWI hits 1. In fact, I said the opposite:

    "Even after it has hit 1, it's likely the drive can still withstand additional write/erase cycles thanks to MLC NAND typically behaving better than the worst-case estimates."

    The problem here isn't that 1,000 P/E cycles isn't enough for a consumer, but the fact that there seems to be a huge difference in endurance between 20nm MLC and 25nm MLC if our data is correct. Intel claimed that there is no difference, both are 3,000 P/E cycles, but our data contradicts with theirs. Given that the SSD 335 doesn't bring any immediate price cuts, you are getting a worse product for the same money compared to the SSD 330.

    It's of course possible that there is a simple firmware bug which reports wrong MWI or NAND writes, but at least so far Intel has not said anything to suggest that.

Log in

Don't have an account? Sign up now