AnandTech Storage Bench 2011: Much Heavier

I didn't expect to have to debut this so soon, but I've been working on updated benchmarks for 2011. Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. I assembled the traces myself out of frustration with the majority of what we have today in terms of SSD benchmarks.

Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.

Originally I kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system.

Not too long ago I tweeted that I had created what I referred to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. It's the load you'd put on a drive after nearly two weeks of constant usage. And it takes a *long* time to run.

I'll be sharing the full details of the benchmark in some upcoming SSD articles but here are some details:

1) The MOASB, officially called AnandTech Storage Bench 2011—Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. My thinking was that it's during application installs, file copies, downloading and multitasking with all of this that you can really notice performance differences between drives.

2) I tried to cover as many bases as possible with the software I incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II & WoW are both a part of the test) as well as general use stuff (application installing, virus scanning). I included a large amount of email downloading, document creation and editing as well. To top it all off I even use Visual Studio 2008 to build Chromium during the test.

Update: As promised, some more details about our Heavy Workload for 2011.

The test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:

AnandTech Storage Bench 2011—Heavy Workload IO Breakdown
IO Size % of Total
4KB 28%
16KB 10%
32KB 10%
64KB 4%

Only 42% of all operations are sequential, the rest range from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.

Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result I'm going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time I'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, I will also break out performance into reads, writes and combined. The reason I do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.

There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. Lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.

As always I don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea.

The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests. All of the older tests are still run on our X58 platform.

AnandTech Storage Bench 2011—Heavy Workload

We'll start out by looking at average data rate throughout our new heavy workload test:

AnandTech Storage Bench 2011—Heavy Workload

While we saw a pretty significant difference between 3Gbps and 6Gbps interfaces with the Intel 510 and Vertex 3, but the same can't be said about Crucial's m4. There's only a 7% performance improvement seen by using a 6Gbps connector on our Sandy Bridge system. Even more interesting is that performance actually drops a bit compared to the C300. We saw this in some of our synthetic Iometer tests and it's definitely reflected here.

The breakdown of reads vs. writes tells us more of what's going on:

AnandTech Storage Bench 2011—Heavy Workload

The drop in sequential and random read performance we noticed seems to be at fault for the m4's lower-than-C300 performance. Looking at write speeds we actually see an improvement over the C300:

AnandTech Storage Bench 2011—Heavy Workload

The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles, this is just how long the SSD was busy doing something:

AnandTech Storage Bench 2011—Heavy Workload

AnandTech Storage Bench 2011—Heavy Workload

AnandTech Storage Bench 2011—Heavy Workload

Sequential Read/Write Speed AnandTech Storage Bench 2011 - Light Workload
Comments Locked

103 Comments

View All Comments

  • killless - Friday, April 1, 2011 - link

    I frequently use VMWare at work.
    One of the tasks that takes forever is deleting VMWare snapshots. Or even worse one is shrinking VMWare disk foot print. In both of these cases 95% is spent on disk read/write. They take from minutes to hours. They mix random and sequential access.
    More that that they are fairly repeatable...

    Using Kingstone SSDNow instead of harddisk made a huge impact - it is about 3 times faster. However it still takes 2 hours sometimes to shrink disk.

    I think that would be a good repeatable set of tests.
  • CrustyJuggler - Saturday, April 2, 2011 - link

    I just bought a spanking 15" MacBook Pro which i know supports Intels new Sandybridge and the sata6gb interface. I decided to buy with just the stock HDD which i will move to the optical bay after purchasing one of these new spangly SSDs

    After reading just about every review i'm still non the wiser as to which drive would benefit me the most so I think in conclusion i will wait for the leading drives to be released. The OCZ is always favourable in the benchmarks but the horror stories of reliability coupled with dreadful customer support make me nervous.

    The intel 510 seems to have great support and reliability but poor performance.

    I've heard little to nothing about Corsair's ForceGT which should be somewhere in the realms Vertex 3 performance right?

    I'm happy to wait at least another 8 weeks i think and just hope more reviews and testing shows up in that time.. I was using a Force F60 on the laptop i replaced and I had no problems whatsoever so i hope Corsair can have a timely release with the GT.

    I had my eye on Crucial's M4 for a while now but these latest crop of reviews have left me pondering even more. Trim is not supported on Mac as of yet so Anand's take on the M4 makes alarm bells ring regarding trim

    First time poster long time reader, but not very tech savvy so what do to folks?
  • Shark321 - Sunday, April 3, 2011 - link

    Anand is #1 at SSD reviews, but there still a long way to achieve perfection.

    On many workstations in my company we have a daily SSD usage of at least 20 GB, and this is not something really exceptional.

    One hibernation in the evening writes 8 GB (the amount of RAM) to the SSDs. And no, Windows does not write only the used RAM, but the whole 8 GB. One of the features of Windows 8 will be that Windows does not write the whole RAM content when hibernating anymore. Windows 7 disables hibernation by default on system with >4GB of RAM for that very reason!

    Several of the workstation use RAM-Disks, which write a 2 or 3 GB Images on Shutdown/Hibernate.

    Since we use VMWare heavily, 1-2 GB is written contanstly all over the day as Spanshots. Add some backup spanshops of Visual Studio products to that and you have another 2 GB.

    Writing 20 GB a day, is nothing unusual, and this happens on at least 30 workstations. Some may even go to 30-40 GB.

    From all the SSDs we have used over the years, there was about 10% failure rate within 2 years, so daily backups are really necessary. When doing a backup of a SSD to a HDD with True Image, the performance of the individual SSDs differs a lot.

    So future benchmarks for professional users should take into consideration:

    - a daily usage of at least 20 GB (better 40 GB)
    - benchmarking VMWare creating snapshots and shringing images
    - unpacking of large ISO images
    - hibernate peformance
    - backup performance

    Thanks!
  • 789427 - Monday, April 4, 2011 - link

    Anand, can't you design a power measurement benchmark whereby typical disc activity is replicated.

    e.g. Power consumption while video encoding (the time taken shouldn't vary by more than 10% - you could add in the difference in idle power consumption as well!

    Or... copying 100 different 1 Mb sections at 1 sec intervals of a 20 min media file - Mp3, divx mkv

    My point is that Sure, SSD's make your system use more power in most benchmarks because the system is asked to do more. Ask the system to do the same thing over the same period and measure power consumption.

    cb
  • gunslinger690 - Tuesday, April 5, 2011 - link

    Ok all I really care about on the machine I'm considering using this on is gaming and web surfing. That's all I care about. Is this a good drive for this sort of app.
  • VJ - Thursday, April 7, 2011 - link

    "I've already established that 3000 cycles is more than enough for a desktop workload with a reasonably smart controller."

    You haven't "established" much unless you take the free space into consideration. Many desktop users will not have 100GB free space on their drives and with 20GB free space and a write amplification of 10x, the lifetime of a drive may drop to a couple of years.

    A 3D plot of the expected lifetime as a function of available free space (10GB-100GB) and write amplification (1x-20x) would make your argument more sound and complete.
  • Hrel - Saturday, April 16, 2011 - link

    I'd really like to know if using a SSD actually has any real world impact. I mean, benchmarks are great. But come on, it's not gonna make me browse the internet any fast or get me higher FPS in Mass Effect; so what's the point? Really, the question comes down to why should I go out and buy two 2TB hard drives, RAID them then ALSO buy an SSD to install the OS on? So windows boots in 7 seconds instead of 27? Cause quite frankly if it's gonna cost me an extra 120-240 dollars I'll just wait 20 seconds. I almost never boot my machine from scratch anyway.

    Ideally I'd want to use a machine with an SSD in it before buying one. But since that doesn't seem likely I'd love it if you guys could just point a camera at two machines right next to eachother, boot them up, wait for both to finish loading the OS. Then use them. Go online, read some anandtech, boot up Word and type something, open photoshop and edit a picture then start up a game and play for a couple minutes. Cause I'm willing to bet, with the exception of load times, there will be no difference at all. And like I already said, if my game takes 20 seconds to load instead of 5, I can live with that. It's not worth anywhere near 100+ dollars.

    Just to be clear one should have an SSD in it near to the 100 dollar mark, since that's as expensive of an SSD most people can even afford to consider. And the other should have two modern 7200rpm mechanical disks from WD in a striped RAID.
  • Hrel - Saturday, April 16, 2011 - link

    both on 6GBPS on P67.
  • Hrel - Saturday, April 16, 2011 - link

    forgot to mention, the SSD machine should also have the same 2 mechanical disks in striped RAID and all the games and programs should be installed on those. Cause for 100 dollars the only thing you can put on an SSD is the OS and some small programs like the browser and various other things every PC needs to be usable; maybe photoshop too but the photos couldn't be on the SSD.
  • Hrel - Saturday, April 16, 2011 - link

    the games can't be installed on the SSD is my point.

Log in

Don't have an account? Sign up now