AnandTech Storage Bench 2011: Much Heavier

I didn't expect to have to debut this so soon, but I've been working on updated benchmarks for 2011. Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. I assembled the traces myself out of frustration with the majority of what we have today in terms of SSD benchmarks.

Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.

Originally I kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system.

Not too long ago I tweeted that I had created what I referred to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. It's the load you'd put on a drive after nearly two weeks of constant usage. And it takes a *long* time to run.

I'll be sharing the full details of the benchmark in some upcoming SSD articles but here are some details:

1) The MOASB, officially called AnandTech Storage Bench 2011—Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. My thinking was that it's during application installs, file copies, downloading and multitasking with all of this that you can really notice performance differences between drives.

2) I tried to cover as many bases as possible with the software I incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II & WoW are both a part of the test) as well as general use stuff (application installing, virus scanning). I included a large amount of email downloading, document creation and editing as well. To top it all off I even use Visual Studio 2008 to build Chromium during the test.

Update: As promised, some more details about our Heavy Workload for 2011.

The test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:

AnandTech Storage Bench 2011—Heavy Workload IO Breakdown
IO Size % of Total
4KB 28%
16KB 10%
32KB 10%
64KB 4%

Only 42% of all operations are sequential, the rest range from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.

Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result I'm going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time I'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, I will also break out performance into reads, writes and combined. The reason I do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.

There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. Lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.

As always I don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea.

The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests. All of the older tests are still run on our X58 platform.

AnandTech Storage Bench 2011—Heavy Workload

We'll start out by looking at average data rate throughout our new heavy workload test:

AnandTech Storage Bench 2011—Heavy Workload

While we saw a pretty significant difference between 3Gbps and 6Gbps interfaces with the Intel 510 and Vertex 3, but the same can't be said about Crucial's m4. There's only a 7% performance improvement seen by using a 6Gbps connector on our Sandy Bridge system. Even more interesting is that performance actually drops a bit compared to the C300. We saw this in some of our synthetic Iometer tests and it's definitely reflected here.

The breakdown of reads vs. writes tells us more of what's going on:

AnandTech Storage Bench 2011—Heavy Workload

The drop in sequential and random read performance we noticed seems to be at fault for the m4's lower-than-C300 performance. Looking at write speeds we actually see an improvement over the C300:

AnandTech Storage Bench 2011—Heavy Workload

The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles, this is just how long the SSD was busy doing something:

AnandTech Storage Bench 2011—Heavy Workload

AnandTech Storage Bench 2011—Heavy Workload

AnandTech Storage Bench 2011—Heavy Workload

Sequential Read/Write Speed AnandTech Storage Bench 2011 - Light Workload
Comments Locked

103 Comments

View All Comments

  • MrSpadge - Thursday, March 31, 2011 - link

    Wait for the final hard- and software. Some power saving mode was probably not active in this beta version.

    MrS
  • Out of Box Experience - Friday, April 1, 2011 - link

    The majority of users as of January 2011 were still using XP
    55% used XP
    22% used Windows 7
    So for the Majority of users, Intel's 320 series is still the best SSD to get as it uses the same OS agnostic controller that the last Intel SSD's

    OCZ on the other hand just purchased Indilynx so we may yet see a OS agnostic controller from them in the near future to compete with Intel in the non-Windows 7 space

    This would make perfect sense for OCZ to add the offset to Indilynx controllers to compete for XP users and keep the Sandforce line for Windows 7 users

    Although Indilynx does not have the Raw throughput of a Sandforce SSD, it would give respectable performance on "ANY" OS without the need for partition hacks and should be suitable for dual boots if the proper offset is implemented

    One of the most basic problems with partition offset hacks for XP was that the offset would be lost if you backup and restore a single partition using Acronis True Image

    The only way to keep the offset would be to backup the entire drive!

    O&O defrag Pro v14 has a "Manual" Trim command if anyone is interested ???

    Controllers that do not include the proper offset for XP boxes generally have horrible "USED" performance after several writes but seem to function OK if you only use them for a boot drive and avoid writes (Hard to do with XP)

  • Out of Box Experience - Friday, April 1, 2011 - link

    Correction:
    --------------
    So for the Majority of users, Intel's 320 series is still the best SSD to get as it uses the same OS agnostic controller that the last generation Intel SSD's had
  • 7Enigma - Friday, April 1, 2011 - link

    Could you site your source for those numbers? My hunch is that the people still on XP are likely NOT the same people that will be purchasing these drives. More likely they are on Vista or Win7 which mainly negates your comment.

    Don't get me wrong I love XP still (my laptop and wife's computer are still running it), but for my main gaming rig (only system with a SSD) I've been through 2 OS revisions (Vista and now 7). Why? DX10/11 support.

    And people upgrading existing systems for a SSD are likely upgrading for similar reasons. Few people are going to drop $100-400 for a SSD on a 3-5 year old computer. They are brainwashed into thinking the spyware/virii loaded system is slow because it's old, not because it just needs a fresh install and some proper lockdown/protection.
  • Out of Box Experience - Friday, April 1, 2011 - link

    Sorry, I could not find the exact source that I used but here is a different one with similar results

    http://www.winmatrix.com/forums/index.php?/topic/3...

    Your hunch that people still using XP are likely NOT the same people that will be purchasing these drives is a good bet due to the fact that these drives tend to perform quit badly in a used state when compared to the Intel SSD's on an XP box

    This is also the reason I think that OCZ may have purchased Indilynx
    To add the offset needed to compete with Intel on the XP boxes

    Even if Windows 7 had the same market share "TODAY" that XP enjoys, it is still foolish for SSD makers to ignore that huge XP market at their own detriment

    Windows 7 may be the future, but there's gold in them thar XP hills
  • LeTiger - Thursday, March 31, 2011 - link

    With the major players out of the woods now, I guess it's just time to wait until next year to see prices go down. Overall, I can't decide whether I'm happy over a (potentially) knockout performance by OCZ, or sad at the state of Intel/Crucial/etc... offerings...

    With prices where they still are at this point, I am very satisfied with my 30gb boot drive, and will continue to wait till costs go down until I convert fully over to the SSD realm...
  • Wave Fusion - Thursday, March 31, 2011 - link

    I don't place much faith on that happening.
    SSDs are selling well enough there's not a good enough reason to reduce prices anytime soon.

    At about $1 per GB most SSDs costs more than major parts like the CPU, GPU, RAM and HDD combined.

    Allow me to pose a question for a general consensus: All and SSD does better is open files/programs faster and move large files (drive 1 to drive 2) faster; due to its high read/write speeds and more simultaneous operations.

    BUT; if you've already run a program once since booting, closed it, then opened it again.. I thought the program was likely cached in the RAM. Which means an SSD would no longer improve anything

    The only potential benefit past 1st run would be any data that's never rerun, like media encoding. But the often poorly designed/outdated single threaded programs themselves are a bottleneck way before a HDD is. At least on my machine, even similtaneous

    So then I have to ask myself, assuming recently accessed files/programs cached in RAM isn't some myth; what do I really gain from using an SSD?

    Why would I spend that money when I could instead use it for an even better GPU or something I actually will notice?
  • MilwaukeeMike - Thursday, March 31, 2011 - link

    Valid point.. but when I use my PC i do a few things frequently that would benefit. I have a kid, and therefore lots of pics and videos, and i play some video games, so i deal with loading screens. I don't ever sit and wait and wish i had 100 FPS instead of my current avg 60. The only times i'm ever looking at my screen and waiting for my PC to finish doing something is either loading a webpage (1-2 seconds) or loading a game (often) or copying files (kinda rare).

    You're right, if you only use word and excel, this isn't for you. But if find yourself waiting for game screens to load it's worth it. Also, many people have already perfectly fast CPU's/GPU/s and good enough monitors.

    The HD is the last slow piece of my computer for me to upgrade.
  • Ammaross - Thursday, March 31, 2011 - link

    Yes, most CPUs bought within the last two years will be more than powerful enough to handle common use. Juicing a system with an SSD will be the best way to regain "lost" performance. Rather than telling your parents to get a new computer when their old one is just "too slow," upgrade their hard drive to an SSD. They likely don't reach 120GB as it is.
  • LeTiger - Friday, April 1, 2011 - link

    I would spend the money because for what I do, an SSD greatly benefits my workflow. Tooling around in Lightroom/CS5 all day benefits from an SSD. I just think they are still too much, and when they do get down to the $1 per gig, I will snatch up the biggest one I can afford.

    As far as "something you will actually notice", you couldn't be more mistaken, an SSD will make the biggest noticeable difference in performance in a computer that you will be able to see. As the comment below says, (and I agree), would you rather watch your FPS meter jump from 60 to 100? or have almost everything you do load very, VERY quickly.

    - It made my intel Atom/ION laptop feel as fast as my Quad Core for light usage scenarios, now that is worth spending money on as peace of mind.

Log in

Don't have an account? Sign up now