AnandTech Storage Bench 2011

Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. Anand assembled the traces out of frustration with the majority of what we have today in terms of SSD benchmarks.

Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.

Originally we kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system. Later, however, we created what we refer to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. This represents the load you'd put on a drive after nearly two weeks of constant usage. And it takes a long time to run.

1) The MOASB, officially called AnandTech Storage Bench 2011—Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. Our thinking was that it's during application installs, file copies, downloading, and multitasking with all of this that you can really notice performance differences between drives.

2) We tried to cover as many bases as possible with the software incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II and WoW are both a part of the test), as well as general use stuff (application installing, virus scanning). We included a large amount of email downloading, document creation, and editing as well. To top it all off we even use Visual Studio 2008 to build Chromium during the test.

The test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:

AnandTech Storage Bench 2011—Heavy Workload IO Breakdown
IO Size % of Total
4KB 28%
16KB 10%
32KB 10%
64KB 4%

Only 42% of all operations are sequential; the rest ranges from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.

Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result we're going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time we'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, we will also break out performance into reads, writes, and combined. The reason we do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.

There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. It has lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback, as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.

We don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea. The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests.

AnandTech Storage Bench 2011—Heavy Workload

We'll start out by looking at average data rate throughout our new heavy workload test:

Heavy Workload 2011 - Average Data Rate

The 1.4 update brought a small boost in our Heavy suite. As you can see in the table above, most IOs are small and the 1.4 and 1.5 firmwares brought marginal improvements to small transfer size performance.

Heavy Workload 2011 - Average Read Speed

As I mentioned in the previous page, Vertex 4 still lacks read performance at small transfer sizes and the above graphs supports this. The average write performance is very good however and definitely makes up for the middle of the road read performance.

Heavy Workload 2011 - Average Write Speed

The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles; this is just how long the SSD was busy doing something:

Heavy Workload 2011 - Disk Busy Time

Heavy Workload 2011 - Disk Busy Time (Reads)

Heavy Workload 2011 - Disk Busy Time (Writes)

 

Performance vs. Transfer Size AnandTech Storage Bench 2011 - Light Workload
Comments Locked

60 Comments

View All Comments

  • hasseb64 - Sunday, August 5, 2012 - link

    This OCZ firmware review biz has to stop. Let them get ONE chance to deliver a FINALIZED PRODUCT (review it 1 time, and then it is enogh). We consumers should not accept a half finished product.
  • jwilliams4200 - Sunday, August 5, 2012 - link

    The solution to that is simple. Just do what I do. Never buy OCZ.
  • peevee - Monday, August 6, 2012 - link

    EXACTLY!
    OCZ is a sketchy business. We remember upder0the-table downgrade of Vertex 2. We remember blue-screening Vertex/Agility 3s. Now this.
    OCZ employs incompetent firmware engineers. 15 months later after I purchased my Vertex 3, and the latest firmware is still unstable.
    Now all modern SATA III SSDs are fast enough for desktop use. The main differentiators are reliability and quality of support. And at these metrics OCZ is the worst of the worst. They still cannot upgrade firmware on a system drive from Windows, they still don't work with Mac at all, they still will require you to "Sec erase" your drive with your system (and then spend hundred hours to reinstall and reconfigure everything again) every time they update firmware, they still don't allow you to go to an old firmware (although some on them crash less for some users then their new firmwares) and require you to have working internet connection for update utility to redownload firmware every time... And blame problems on everybody else (while other drives simply work). This company is so bad it should simply die. I don't know why Tom's pays them so much attention, OCZ's reputation is beyond repair at this point.
  • vishwa108 - Sunday, August 5, 2012 - link

    It is never a good moment when the bull being lead by its nose-ring is claiming to be leading the bullring. This OCZ Independence Day saga have had many a worshipful scurrying to atone his altarboy days of their “independent” review and now, after “Version 1.5”, the worship is still akin to the fruit of The Choir Boys At Practice being heard louder than ever.

    First it was OCZ Zist und Zat soon to be followed by photocopied scans of “the reviewers” bottoms and then it was Marvellouski, quickly buried by another version of Amerikana 1.4 – only to end up with some vacuous advice of the, “Don’t go beyond half und you be OK, ja?”, variety. All for a good course/cause in buggery no less. No wonder Amerikans are being offered either the feast of Uncle Tommy und hist Kobbling Kabin Krew or another Mormon extravaganza with the mesmerised numpty vessels ringing louder than ever whenever the word, “War”, is messaged. What? OCZ are not of Yankeeland but rather of those who will soon be receiving their just rewards from across The Straits? Makes you ‘fink, ain’t eet. Perhaps the truth ought to be tried once in awhile – with all holes barred, of course.

    It is self benevolent to realise that Truth is merely another word for Freedom because the freedom that was gifted is about another/others and is not about self – when the gift was taken back. When truth is of self, you can’t even get it out of the blighter after a severe bout of torture. Hands up those who would like to disagree – especially when it is via a freed versioned 1.5 of some SSD doo-dah.
  • ssddaydream - Sunday, August 5, 2012 - link

    what is wrong with you?
  • shunya806 - Sunday, August 5, 2012 - link

    welcome to our website:
    http://commonprosperity.org/simpletrading@hotmail.com

    Dear friends, do you want to have some different things?
    Whether you want to give your relatives and friends,
    take a few different exotic gifts? Whether you want to
    buy some cheap benefits of thing? So please, let us begin now!

    Click on our website
    http://commonprosperity.org/
    simpletrading@hotmail.com
  • mark53916 - Monday, August 6, 2012 - link

    What happens to the lifetime and reliability of the OCZ Technology Vertex 4
    drives with firmware 1.5 when the drive is only running at 1/2 capacity?

    We already know that the performance is improved, but there may be other
    advantages to operating with reduced capacity. These two come to mind:
    1. The drive endurance may be improved
    If it turns out that either the drive endurance is improved if the
    drive is only half used I would be very happy.

    2. The data retention time for powered off devices may be improved.
    Increased powered off data retention time at reduced capacity
    won't affect me much since I couldn't afford solid state for my monthly
    backups even at the full capacity price, but people who have to
    keep around old system disks for long periods would benefit
    if the powered off data retention time were increased

    In addition, I find that the per byte price for SLC drives seems to be about
    10 times what the cost is for MLC drives [even though the manufacturing techniques
    seem to indicate that manufacturing cost is slightly less than 2 or 3 times
    as much for SLC compared to MLC.
    I'd gladly spend 2x the price per byte and run at 1/2 capacity until I actually
    need the additional capacity, even without increased drive endurance or
    powered off retention time, just for the improved performance.
  • FunBunny2 - Monday, August 6, 2012 - link

    SLC and MLC are the same parts, just binned and then programmed for either 1 or 2 bits. That's it.
  • robalm - Sunday, September 16, 2012 - link

    I decided to upgrade from my old Intel x25-m g2 80gb to a new fast ssd.
    It became an OCZ Vertex 4 128gb (1.5) but now I saw this review and this:

    "Small filesize sequential read performance needs work. Thankfully most sequential reads in client workloads tend to be in the sweet spot for the Vertex 4, but there are some applications That do a lot of small sequential IO (eg web browser cache accesses). '

    Have I made ​​a stupid upgrde when it come to web browser performence?
  • ryrynz - Monday, December 15, 2014 - link

    My Vertex 4 128GB had a sudden death. Seems to be fairly common for these drives.
    MAKE SURE YOU'RE BACKED UP. If you get some sudden reboots you know your controller
    is on the way out.. one time it'll happen and poof.. no drive detected, you've been warned.

Log in

Don't have an account? Sign up now