AnandTech Storage Bench 2011

Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. Anand assembled the traces out of frustration with the majority of what we have today in terms of SSD benchmarks.

Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.

Originally we kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system. Later, however, we created what we refer to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. This represents the load you'd put on a drive after nearly two weeks of constant usage. And it takes a long time to run.

1) The MOASB, officially called AnandTech Storage Bench 2011—Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. Our thinking was that it's during application installs, file copies, downloading, and multitasking with all of this that you can really notice performance differences between drives.

2) We tried to cover as many bases as possible with the software incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II and WoW are both a part of the test), as well as general use stuff (application installing, virus scanning). We included a large amount of email downloading, document creation, and editing as well. To top it all off we even use Visual Studio 2008 to build Chromium during the test.

The test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:

AnandTech Storage Bench 2011—Heavy Workload IO Breakdown
IO Size % of Total
4KB 28%
16KB 10%
32KB 10%
64KB 4%

Only 42% of all operations are sequential; the rest ranges from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.

Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result we're going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time we'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, we will also break out performance into reads, writes, and combined. The reason we do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.

There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. It has lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback, as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.

We don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea. The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests.

AnandTech Storage Bench 2011—Heavy Workload

We'll start out by looking at average data rate throughout our new heavy workload test:

Heavy Workload 2011—Average Data Rate

The 840 is quite average in our Heavy suite and performs similarly to most SandForce drives. The 840 Pro is a lot faster under heavy workloads, so it should be obvious by now why Samsung is offering two SSDs instead of one like they used to.

Heavy Workload 2011—Average Read Speed

Heavy Workload 2011—Average Write Speed

The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles, this is just how long the SSD was busy doing something:

Heavy Workload 2011—Disk Busy Time

Heavy Workload 2011—Disk Busy Time (Reads)

Heavy Workload 2011—Disk Busy Time (Writes)

Performance vs. Transfer Size AnandTech Storage Bench 2011 - Light Workload
Comments Locked

86 Comments

View All Comments

  • mayankleoboy1 - Monday, October 8, 2012 - link

    Samsung is known to release only a few firmware updates (unlike SF).
    But due to the somewhat quirky nature of TLC NAND, do you expect Samsung to release a newer firmware , with maybe better read performance , or better TRIM support ?
  • Taft12 - Monday, October 8, 2012 - link

    Who cares?? Frequent firmware updates are a sign of an incompetent engineering and testing. I'll stick with the vendors known for getting it right the first time thank you very much.
  • JuneBugKiller - Monday, October 8, 2012 - link

    If Samsung 840 120GB cost's $109.99 and you can get an OCZ Vertex 4 128GB for $104.99, which is faster in 80% of the benchmarks and is $5 cheaper then who's going to want it? So what if the price goes down to $80, are people going to save $25 thinking their hard drive is going to die twice as fast. I have bought (with works money) around 30 ssd's including Intel 80GB & 160GB Silver Case, Intel 710 100GB, Vertex 3 Max IOPS 120GB and Vertex 4 128. So far I've only had 2 ssd's go bad and they were both Intel 80GB ssd's. One wouldn't power on and the other reported at 8MB. Intel replaced them under warranty.

    The point is I have been buying SSD's for years and I just don't see how anyone would want these samsung ssd's. Samsung is famous for huge margins on each product. When a company pockets $240 off of a $500 tablet and their name isn't Apple then something is wrong.
  • JarredWalton - Monday, October 8, 2012 - link

    Wow, talk about skewing the data by cherry picking the worst-case numbers for the 840. The Vertex 4 is only 85% faster in one specific test: AS-SSD Write performance. Of course, in the AS-SSD Read performance it's also 10% faster than the Vertex 4 256GB, but you just ignore that? Samsung also has some of the best SSDs in terms of large reliability figures, so even if the 840 is slightly slower than other drives in some tests, it may be the better option. Also don't forget to factor in that the 840 appears to be well-tuned for light workloads (e.g it's near the top of our light workload results).

    Personally, I think the 840 needs to come in below the current 830 drive prices to make sense, and it probably will not long after the official release. 128GB 830 drives already go for under $100, and 256GB drives have been at $200 for over a month now -- likely all in preparation for the release of the 840. TLC NAND is cheaper to manufacture (per GB), and long-term it will be significantly more profitable for Samsung. Get some good DSPs added into the mix and I wouldn't be surprised to see most SSDs in two generations being TLC based, with MLC moving to the enterprise level and SLC basically going away because it's too expensive.
  • JuneBugKiller - Monday, October 8, 2012 - link

    No I'm talking about counting each benchmark add the total and divide the number by how many OCZ Vertex 4 won and it was over 80% of the total number of benchmarks. How is Samsung a better option? TLC over MLC, OCZ fastest drive to Samsungs slowest new drive. Of course Samsung is going to make more profit but why would you want to spend the same amount on a slower drive with less endurance?
  • JarredWalton - Monday, October 8, 2012 - link

    Sorry, 80% of benchmarks is correct; I read that wrong. But let's put that in perspective:
    http://www.anandtech.com/bench/Product/678?vs=628

    If we look at average performance across all benchmarks, the difference between OCZ Vertex 256GB and Samsung 840 250GB is a 12.4% advantage for OCZ. However, OCZ hasn't exactly been free from firmware issues. That right there is the reason many people will pay a bit more for a Samsung (even if it's slower).

    Would I buy an 840 right now for $200 or whatever? Definitely not -- I'd actually take the 830, just for proven reliability over time. Give the 840 a couple months just to be safe, then check the prices. If it's still more expensive than the Vertex 4, sure, go for OCZ if you'd like. If they're the same price, though, the 12% performance is practically meaningless for most consumer workloads.
  • sean.crees - Monday, October 8, 2012 - link

    Because it's not just about performance, it's also about reliability. This is user data we are talking about. Even a single loss could be catastrophic. OCZ doesn't have the reliability track record that Samsung and Intel has, and for that reason there are many people who will only ever buy from Samsung or Intel. So then OCZ isn't even mentionable. It doesn't matter if they are faster because who cares if your data is at risk?

    Also you have to consider that the REAL WORLD difference between any modern 6Gbps SSD is negligible, so then performance means even less. It ends up being a contest of reliability instead of a contest of speed. In that contest, OCZ loses.
  • krumme - Monday, October 8, 2012 - link

    I would want it :)

    We have about 7-8 ssd in the house, about 3 of those is vertex 3 ssd, and there have been realiability and firmware issues.

    I think ssd speeds since the last 2 years have been good enough and would prioritize reliability any day.

    I had to replace an sandisk u100 in my Samsung 9 series x3c to a faster one, but that was because the u100 was like a return to 4 years ago. Its a good backup now.

    Now hopefully reliability is there, and prices will go down so everyone can afford it. We dont need more firmware updates and shit. Its like the first 3d gfx in the mid 90, - a mess.

    But you are right Samsung is starting to get expensive, and charge for the brand. Wether you like it or not it will certainly mean more Samsung reviews too. Being a big boy, have advantages :)
  • Taft12 - Monday, October 8, 2012 - link

    "If Samsung 840 120GB cost's $109.99 and you can get an OCZ Vertex 4 128GB for $104.99, which is faster in 80% of the benchmarks and is $5 cheaper then who's going to want it?"

    A person who values reliability at a value higher than $0 will choose the non-OCZ product every time.
  • Jaybus - Thursday, October 11, 2012 - link

    Time will tell. The TLC NAND should end up around 30% cheaper per GB than MLC. Then it will come down to buying a 240 GB MLC for $200 or a 320 GB TLC for the same price.

Log in

Don't have an account? Sign up now