AnandTech Storage Bench 2011

Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. I assembled the traces myself out of frustration with the majority of what we have today in terms of SSD benchmarks.

Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.

Originally I kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system.

Not too long ago I tweeted that I had created what I referred to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. It's the load you'd put on a drive after nearly two weeks of constant usage. And it takes a *long* time to run.

Here's a high level overview:

1) The MOASB, officially called AnandTech Storage Bench 2011 - Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. My thinking was that it's during application installs, file copies, downloading and multitasking with all of this that you can really notice performance differences between drives.

2) I tried to cover as many bases as possible with the software I incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II & WoW are both a part of the test) as well as general use stuff (application installing, virus scanning). I included a large amount of email downloading, document creation and editing as well. To top it all off I even use Visual Studio 2008 to build Chromium during the test.

Digging a little deeper, the test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:

AnandTech Storage Bench 2011 - Heavy Workload IO Breakdown
IO Size % of Total
4KB 28%
16KB 10%
32KB 10%
64KB 4%

Only 42% of all operations are sequential, the rest range from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.

Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result I'm going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time I'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, I will also break out performance into reads, writes and combined. The reason I do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.

There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. Lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.

As always I don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea.

The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests. All of the older tests are still run on our X58 platform.

AnandTech Storage Bench 2011 - Heavy Workload

We'll start out by looking at average data rate throughout our new heavy workload test:

AnandTech Storage Bench 2011 - Heavy Workload

The breakdown of reads vs. writes tells us more of what's going on:

AnandTech Storage Bench 2011 - Heavy Workload

AnandTech Storage Bench 2011 - Heavy Workload

The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles, this is just how long the SSD was busy doing something:

AnandTech Storage Bench 2011 - Heavy Workload

AnandTech Storage Bench 2011 - Heavy Workload

AnandTech Storage Bench 2011 - Heavy Workload

Random & Sequential Performance AnandTech Storage Bench 2011 - Light Workload
POST A COMMENT

90 Comments

View All Comments

  • dagamer34 - Friday, May 06, 2011 - link

    Great review Anand, but a far more practical question is this, why should the average consumer get the OCZ Vertex 3 at a almost $100 street price premium when the Vertex 2 performs reasonably well in all areas that matter? Almost no regular consumer is actually going to benefit from having sustained 500MB/sec read/writes compared to 200MB/sec because you'd actually need a source/destination that can also dish out or take in data that fast. Sure, compared to a HDD, any SSD is better, but is for example loading Windows in 14 seconds as opposed to 9 seconds really worth that much.

    Of course, for business markets, all bets are off and faster is almost always better since they've got RAID arrays churning to get data access as fast as possible. One SSD can easily replace a RAID 5 Raptor array at a far lower cost.
    Reply
  • Anand Lal Shimpi - Friday, May 06, 2011 - link

    I think for the average consumer simply making the move to an SSD is good enough. From there it's a question of subtlety. I can notice the difference on a Sandy Bridge system between a Vertex 2 and Vertex 3 - is it worth the price premium? For most users I'd say no, particularly if you aren't running a platform with a good 6Gbps controller.

    Take care,
    Anand
    Reply
  • aegisofrime - Friday, May 06, 2011 - link

    Thanks for the answer, I was wondering the same thing as well. With Vertex 2s going for cheap nowadays, I think I can settle for a Vertex 2. Thanks for all the great work Anand, Anandtech is the first site I hit when I wake up :) Reply
  • vol7ron - Friday, May 06, 2011 - link

    It's good advice, but also keep in mind scalability and future-proofing.

    How long do you keep your HDs? I keep mine until failure. Even if I don't have enough SATA ports on my mobo/cards, I'll use USB transformers to make user of the drives.

    How important is it to you to buy an inferior product? To most users, it might not matter and the others might not care; but keep in mind you might use it as an external drive some day and you might want it to be able to saturate, or at least make use of the bandwidth for that external interface - think USB3/4 or Light Peak (Thunderbolt).

    That being said, "deals" are deals for a reason and there's no such thing as a bad deal, just "mistakes" and "favors."
    Reply
  • Rasterman - Monday, May 09, 2011 - link

    That doesn't make any sense, drives can last for 10 years or more, are you really telling me you have a 50MB drive from 10 years ago that you have hooked up to a USB port sucking 10-20W down? At some point it makes more sense to consolidate and upgrade, its only a matter of time before the cost to simply power the drive isn't worth the money. Add to that error rates increase exponentially as the drive ages and your data will simply not be reliable.

    I personally get a new storage drive every 3-4 years, the old ones are re-purposed into encrypted off-site backups as they usually aren't worth anything used.
    Reply
  • jharmon - Friday, May 06, 2011 - link

    Again Anand, thank you so much for an excellent review. However, I was wondering if you could expound a bit more on this comment. I am currently running the X58 with Marvell 9128 controller. It is a sata 6Gbps interfaces, but the max bandwidth on that is 500 MB/s and it seems to be capping performance. Even OCZ warns about this on their website.

    You say you notice a different on SandyBrige, but what about the X58 platform?

    Thanks again!
    Reply
  • vol7ron - Friday, May 06, 2011 - link

    These kinds of questions should go to the forum. Though authors/owners, do respond to some comments, the "Comments" section is not for questions, especially questions of another technology. I think they are meant for readers and users to leave comments on the article, content, or food-for-thought. Reply
  • v8x - Monday, May 09, 2011 - link

    I do not agree, Who are you to decide what goes into the forums and what not? If Anand makes a statement in an article he wrote, it's not more than natural that questions on that statement go in the comments section.
    It is not a question on another technology, it's a question on a statement Anand made.
    Reply
  • Hargak - Friday, May 06, 2011 - link

    Actually, your RAM is usually the secondary device in 99% of the situations. And your RAM can receive data much faster than transferring it to another Vertex 3. Hence the improvements in loading large programs such as photoshop or big gaming programs. Whenever it is loading something you are taking data from the hard drive and caching it to RAM so the CPU can work with it in a much lower latency higher bandwidth situation. Then again, the question is whether or not the data is compressable. If it isn't then your basically back to Vertex 2 speeds. I'm also assuming in RAID 0 situations you would have a large margin of read bandwidth depending on the type of data compressable or uncompressable. I've been sold on SSD's since I found out that I could actually overclock the drive. I suggested to one of the engineers (Tony) hat they sell an overclocked version since there is headroom in added frequency. I overclocked the southbridge which made ~10% increase in performance on an old Core SSD. Down the road we may see performance SSD's possibly in card format with heatsinks and fans on them. Reply
  • Hargak - Friday, May 06, 2011 - link

    Forgot to mention, that's how they came up with the Vertex Turbo, ;) Reply

Log in

Don't have an account? Sign up now