AnandTech Storage Bench 2011

Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. Anand assembled the traces out of frustration with the majority of what we have today in terms of SSD benchmarks.

Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.

Originally we kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system. Later, however, we created what we refer to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. This represents the load you'd put on a drive after nearly two weeks of constant usage. And it takes a long time to run.

1) The MOASB, officially called AnandTech Storage Bench 2011—Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. Our thinking was that it's during application installs, file copies, downloading, and multitasking with all of this that you can really notice performance differences between drives.

2) We tried to cover as many bases as possible with the software incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II and WoW are both a part of the test), as well as general use stuff (application installing, virus scanning). We included a large amount of email downloading, document creation, and editing as well. To top it all off we even use Visual Studio 2008 to build Chromium during the test.

The test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:

AnandTech Storage Bench 2011—Heavy Workload IO Breakdown
IO Size % of Total
4KB 28%
16KB 10%
32KB 10%
64KB 4%

Only 42% of all operations are sequential; the rest ranges from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.

Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result we're going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time we'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, we will also break out performance into reads, writes, and combined. The reason we do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.

There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. It has lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback, as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.

We don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea. The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests.

AnandTech Storage Bench 2011—Heavy Workload

We'll start out by looking at average data rate throughout our new heavy workload test:

Heavy Workload 2011—Average Data Rate

Samsung SSD 830 continues to dominate our Heavy suite but in both read and write, but the M3 is the fastest Marvell based SSD and delivers a result 32% faster than the Crucial m4. Overall the M3 is on-par with SandForce—it does worse in read speed but takes that back in write speed.

Heavy Workload 2011—Average Read Speed

Heavy Workload 2011—Average Write Speed

The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles; this is just how long the SSD was busy doing something:

Heavy Workload 2011—Disk Busy Time

Heavy Workload 2011—Disk Busy Time (Reads)

Heavy Workload 2011—Disk Busy Time (Writes)

AS-SSD Incompressible Sequential Performance AnandTech Storage Bench 2011 - Light Workload
Comments Locked

113 Comments

View All Comments

  • epobirs - Thursday, April 5, 2012 - link

    Kind of sad to see a review where Plextor is treated as an unknown. For quite a long time they were the brand against which all others were judged. For one simple reason: ifPlextor said their drive functioned at speed X, it did. If other companies were claiming a new high performance mark and Plextor hadn't produced a matching product yet, it often meant those other companies were lying about their performance.
  • cjcoats - Thursday, April 5, 2012 - link

    I'm a scientific user (environmental model), and I have a transaction-pattern I've never seen SSD benchmarks use:

    My dataset transactions are of the form "read or
    write the record for variable V for time T"
    (where record-size S may depend upon the variable;
    typical values range from 16K to 100 MB).

    The datasets have a matching form:

    * a header that indexes the variables, their
    records, and various other metadata

    * a sequence of data records for the time
    steps of the variables.

    This may be implemented using one of various
    standard scientific dataset libraries (netCDF,
    HDF, ...)

    A transaction basically is of the form:

    Seek to the start of the record for variable
    V, time T

    Read or write S bytes at that location.

    NONE of the SSD benchmarks I've seen have this
    kind of "seek, then access" pattern. I have the
    suspicion that Sandforce based drives will do
    miserably with it, but have no hard info.

    Any ideas?
  • bji - Thursday, April 5, 2012 - link

    SSDs have no "seek". Your program's concept of "seek" is just setting the blocks that it will be reading to those at the beginning of a file, but from an SSDs perspective, there is little to no difference between the random access patterns used for a particular operation and any other random access patterns. The only significant difference is between random and serial access.

    My point being, your case sounds like it is covered exactly by the "random write" and "random read" benchmarks. It doesn't matter which part of the file you are "seeking" to, just that your access is non-sequential. All random access is the same (more or less) to an SSD.

    This is most of the performance win of SSDs over HDDs - no seek time. SSDs have no head to move and no latency waiting for the desired sectors to arrive under the head.
  • cjcoats - Friday, April 6, 2012 - link

    I guessed you'd understand the obvious: seek() interacts with data-compression.

    A seek to a 500MB point may depend upon sequentially decompressing the preceding 500 MB of data in order to figure out what the data-compression has done with that 500MB seek-point!

    That's how you have to do seeks in conjunction with the software "gzlib", for example.

    So how do SandForce drives deal with that scenario ??
  • Cerb - Saturday, April 7, 2012 - link

    Nobody can say exactly the results for your specific uses, but it would probably be best to focus on other aspects of the drives, given performance of the current lot. You might get a 830, while a 520 could be faster at your specific application, but you'd more than likely be dealing with <10% either way. If it was more than that, a good RAID card would be a worthy addition to your hardware.

    If you must read the file up to point X, then that's a sequential read. If you read an index and then just read what you need, then that's a random read.

    Compression of the data *IN*SOFTWARE* is a CPU/RAM issue, not an SSD issue. For that, focus on incompressible data results.

    TBH, though, if you must read 500MB into it to edit a single small record, you should consider seekable data formats, instead of wasting all that CPU time.
  • bji - Saturday, April 7, 2012 - link

    They don't use streaming ciphers. They use block ciphers that encrypt each block individually and independently. Once the data makes it to the drive, there is no concept of 'file', it's just 'sectors' or whatever the block concept is at the SATA interface level. As far as the SSD is concerned, no two sectors have any relationship and wear levelling moves them around the drive in seemingly arbitrary (but designed to spread writes out) ways.

    Basically what happens is that the drive represents the sequence of sectors on the drive using the same linear addressing scheme as is present in hard drives, but maintains a mapping for each sector from the linear address that the operating system uses to identify it, to whatever unrelated actual block and sub-block address on the device that it is physically located at. Via this mapping the SSD controller can write blocks wherever makes the most sense, but present a consistent linear sector addressing scheme to the operating system. The SSD can even move blocks around in the background and during unrelated writes, which it definitely does to reduce write amplification and once again for wear levelling purposes. The operating system always believes that it wrote the sector at address N, and the SSD will always deliver the same data back when address N is read back, but under the covers the actual data can be moved around and positioned arbitrarily by the SSD.

    Given the above, and given that blocks are being written all over the flash all the time regardless of how linearly the operating system thinks it has arranged them, there really isn't any concept of contiguously compressed blocks and having to start back at the beginning of some stream of data to uncompress data.

    Keep in mind also that the Sanforce drives do de-duplication as well (as far as I know), which means that for many blocks that have the same contents, only one copy needs to actually be stored in the flash and the block mapping can point multiple operating system sector addresses at the same physical flash block and sub-block segment that has the data. Of course it would have to do copy-on-write when the sector is written but that's not hard once you have all of the rest of the controller machinery built.

    SSD controllers must be really interesting tech to work on. I can't imagine all of the cool algorithmic tricks that must be going on under the covers, but it's fun to try.
  • BolleY2K - Thursday, April 5, 2012 - link

    Don´t forget about the Yamaha CRW-F1... ;-)
  • Metaluna - Saturday, April 7, 2012 - link

    Heh I still have one of those Yamahas, along with some old Kodak Gold CD-R's. I have no real use for them anymore but hate to toss them.
  • Hourglasss - Thursday, April 5, 2012 - link

    You made a forgivable mistake with OCZ's Vertex 4. You said the M3 was the fastest non-sandforce drive. The Vertex 4 is made with OCZ's new everest-2 controller that they developed in-house after acquiring indillix (don't know if they spelled that right). So the M3 is fast, but it's second for non sandforce.
  • zipz0p - Thursday, April 5, 2012 - link

    I am glad that I'm not the only one who noticed this! :)

Log in

Don't have an account? Sign up now