AnandTech Storage Bench 2011

Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. Anand assembled the traces out of frustration with the majority of what we have today in terms of SSD benchmarks.

Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.

Originally we kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system. Later, however, we created what we refer to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. This represents the load you'd put on a drive after nearly two weeks of constant usage. And it takes a long time to run.

1) The MOASB, officially called AnandTech Storage Bench 2011—Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. Our thinking was that it's during application installs, file copies, downloading, and multitasking with all of this that you can really notice performance differences between drives.

2) We tried to cover as many bases as possible with the software incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II and WoW are both a part of the test), as well as general use stuff (application installing, virus scanning). We included a large amount of email downloading, document creation, and editing as well. To top it all off we even use Visual Studio 2008 to build Chromium during the test.

The test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:

AnandTech Storage Bench 2011—Heavy Workload IO Breakdown
IO Size % of Total
4KB 28%
16KB 10%
32KB 10%
64KB 4%

Only 42% of all operations are sequential; the rest ranges from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.

Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result we're going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time we'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, we will also break out performance into reads, writes, and combined. The reason we do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.

There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. It has lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback, as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.

We don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea. The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests.

AnandTech Storage Bench 2011 - Heavy Workload

We'll start out by looking at average data rate throughout our new heavy workload test:

Heavy Workload 2011 - Average Data Rate

The SX900 is slightly faster than the Vertex 3 and Force GT here, which is most likely due to the newer firmware as I noted in the incompressible sequential read/write speed section. I asked Anand to test a 120GB ADATA S511, which is based on the same SF-2281 controller and utilizes 25nm Intel MLC NAND. The SX900 and S511 should be as similar as possible; however, they don't share the same firmware version. The SX900 is reported to have firmware version 5.0.1 in Device Manager, whereas the S511 Anand tested was running 3.2.2. That's the most current firmware for the S511 and it dates back to November 2011, so this is likely the reason for the difference in performance.

Heavy Workload 2011 - Average Read Speed

Heavy Workload 2011 - Average Write Speed

The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles, this is just how long the SSD was busy doing something:

Heavy Workload 2011 - Disk Busy Time

Heavy Workload 2011 - Disk Busy Time (Reads)

Heavy Workload 2011 - Disk Busy Time (Writes)

AS-SSD Incompressible Sequential Performance AnandTech Storage Bench 2011 - Light Workload
POST A COMMENT

58 Comments

View All Comments

  • Kristian Vättö - Monday, June 11, 2012 - link

    TheJian,

    Reading your comments actually got me interested in the whole situation. It's kind of odd to see increasing profits given that the companies were "struggling" due to floods just a few months ago. A quick conclusion would be that the HD market was getting too low-profit now that SSDs are becoming more and more popular, so the HD companies decided to blame the floods and use that as a tool to limit supply temporarily and increase prices.

    I'll talk to Anand and see what he thinks. This is definitely an intriguing topic - maybe the HD companies are just ripping people off?
    Reply
  • TheJian - Wednesday, June 13, 2012 - link

    In my opinion, based on the numbers, that isn't really a question. It's a REALITY. There is NO other way to look at it. Hopefully Anand will come to the same conclusion and post an article about it. I see Dailytech picked up the article I referenced and posed it (shame...LOL - At least it wasn't Mick...). I was going to just paste my stuff onto that post from the 11th, but I didn't want to give them more hits for whoring out my post...ROFL). I don't want to see that crap repeated, I wanted it TORN APART! :)

    Instead of excusing them, the post should have discussed why they are getting rich during a supposed catastrophe. That is what needs to be analyzed. Why do they continue to rip retail users off? None of us signed some contract thinking no drives would be available. So why am I forced to pay a higher fee? Just to make Dell/HP feel better about getting ripped off?

    PRODUCTION is in full swing at both WD and Seagate. This quarter is the first they will exceed pre-flood drive sales. Then why am I still be screwed at the store? Why did Externals NEVER go up? The same thing should be true at retail/newegg/amazon etc...Those prices have no reason to be high now with production about to EXCEED pre-flood production. What catastrophe are we in now? Is there another LARGER flood I'm unaware of that continues to hurt drive pricing? It seems completely made up at this point, when looking at the profits they are raking in (RECORD PROFITS!). Just tell Anand to look at their profits for Sept 2011 (pre flood), Dec2011 (the actual damage quarter) and 1st quarter 2012 (the complete RIPOFF quarter). I'd like to see another follow up next month or so when we get the June quarter's numbers as they will show HUGE profits for both no doubt.

    Thanks for looking into it! :)
    Reply
  • jwilliams4200 - Wednesday, June 13, 2012 - link

    Actually, there is another way to look at it -- basic economics, supply and demand.

    The flood caused a "supply shock" (look it up on wikipedia), reducing the overall number of HDDs that could be supplied to consumers. As a result, the market price adjusted sharply higher in order to reduce demand to the amount that could be supplied. That is basic ECON 101 behavior, and not at all mysterious.

    WD's production was cut sharply by the flood, but Seagate's production was not as badly impacted, as some of your figures show. But since WD HDDs and Seagate HDDs are "substitute goods" (wikipedia it), the supply curves are closely linked. So Seagate did indeed profit from the overall reduced supply, since WD lost more production. Again, this is all basic economics, and nothing nefarious is needed to explain it.

    Now that production has nearly returned to pre-flood levels, there is still reason for prices to be higher than before -- pent-up demand. Many consumers have delayed HDD purchases while waiting for prices to come down. As these consumers begin to reenter the HDD market, demand will be higher than it ordinarily would be, and so the price must adjust higher than it would ordinarily be so that supply can meet demand.

    All of these issues are well understood by economists. They are so well understood that they are all taught in introductory economics courses.

    There is one more issue that is not basic economics, but I think it is relevant. Others may argue with me about it, since unlike the above concepts, it is not basic economics. The issue is that I think the HDD industry was not long-term viable at the prices that we saw in early-Fall 2011. Clearly the industry was not profitable enough for Hitachi or Samsung to want to continue producing HDDs. Hitachi and particularly Samsung are huge corporations which produce many products and have immense resources. If they did not think it was worthwhile to allocate their capital for HDDs in the future, then it seems obvious that the HDD industry was in severe decline.

    Therefore, I think somewhat higher prices (than we saw in Fall-2011) are necessary for the HDD industry to be lively going forward. Without a prosperous HDD industry, no one will invest the billions of dollars that are necessary to develop the next generations of HDD technology. So I am actually not disturbed to see prices moderately higher than they were in Fall 2011. I expect the price per GB to start falling again eventually, but I also want to see 5TB and 8TB HDDs in the near future, and I am willing to pay a little more $/GB for that new technology.
    Reply
  • CeriseCogburn - Wednesday, June 13, 2012 - link

    A mrono can understand the issues, the problem is locked in contracts from the big system producers at elevated prices and no relief form the manufacturers.

    In the end any excuse is eliminated by merely seagate and wd reducing prices, which they very well can, and still retain enormous profit they never expected.

    Instead they will keep the inflated contract prices and scarf the hundreds of millions.
    Reply
  • TheJian - Monday, June 11, 2012 - link

    http://www.pcmag.com/article2/0,2817,2405500,00.as...

    One more regarding the duopoly that they've created now (these two control 86% of the market post mergers), and having pricing power to keep them artificially high. June 7th.
    Reply
  • jwilliams4200 - Wednesday, June 13, 2012 - link

    What a terrible article! It amazes me that they failed to mention "Toshiba" at all.

    One of the conditions that government regulators put on the WD acquisition of Hitachi GST is that there must be a third 3.5" HDD producer with at least 20% market share (that number came from Hitachi GST's 3.5" HDD market share).

    So, WD sold Toshiba IP and production facilities to allow Toshiba to produce 3.5" HDDs. I don't know all the details, but it seems likely that WD basically sold Hitachi GST's 3.5" HDD business to Toshiba. Toshiba also sold their Thailand 2.5" HDD business to WD. I don't know the details of the deal, it may have been primarily a swap rather than a cash transaction. Or not.

    The situation is further complicated by all the regulators involved -- China, US, Europe. I believe China required that Hitachi GST continue producing HDDs for an extended period before the acquisition could be completed. Also, I believe the WD/Toshiba deal involved WD/Hitachi GST producing HDDs for Toshiba for some period of time.

    As you can see, it is all very complicated. But the author of that article seems remarkably ignorant of the details of the situation.
    Reply
  • CeriseCogburn - Wednesday, June 13, 2012 - link

    What becomes clear is the governments of the world control production.
    Communism, fascism, or regulation, what's the difference - it's not working and prices are still crazy.

    The pols get their due payola.
    Reply
  • mgl888 - Saturday, June 9, 2012 - link

    Anand = A NAND
    Mind=blown
    Reply

Log in

Don't have an account? Sign up now