OCZ Vertex 3 MAX IOPS & Patriot Wildfire SSDs Reviewed
by Anand Lal Shimpi on June 23, 2011 4:35 AM ESTAnandTech Storage Bench 2011
Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. I assembled the traces myself out of frustration with the majority of what we have today in terms of SSD benchmarks.
Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.
Originally I kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system.
Not too long ago I tweeted that I had created what I referred to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. It's the load you'd put on a drive after nearly two weeks of constant usage. And it takes a *long* time to run.
1) The MOASB, officially called AnandTech Storage Bench 2011 - Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. My thinking was that it's during application installs, file copies, downloading and multitasking with all of this that you can really notice performance differences between drives.
2) I tried to cover as many bases as possible with the software I incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II & WoW are both a part of the test) as well as general use stuff (application installing, virus scanning). I included a large amount of email downloading, document creation and editing as well. To top it all off I even use Visual Studio 2008 to build Chromium during the test.
The test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:
| AnandTech Storage Bench 2011 - Heavy Workload IO Breakdown | ||||
| IO Size | % of Total | |||
| 4KB | 28% | |||
| 16KB | 10% | |||
| 32KB | 10% | |||
| 64KB | 4% | |||
Only 42% of all operations are sequential, the rest range from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.
Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result I'm going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time I'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, I will also break out performance into reads, writes and combined. The reason I do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.
There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. Lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.
As always I don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea.
The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests. All of the older tests are still run on our X58 platform.
AnandTech Storage Bench 2011 - Heavy Workload
We'll start out by looking at average data rate throughout our new heavy workload test:
The breakdown of reads vs. writes tells us more of what's going on:
The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles, this is just how long the SSD was busy doing something:
Facebook
Twitter
RSS
112 Comments
View All Comments
semo - Friday, June 24, 2011 - link
Thanks Anand. I appreciate your honesty and transparency. If it wasn't for you, Jmicron would have killed the momentum of SSD adoption. I'd hate to see the same thing happen again right under our noses.irev210 - Thursday, June 23, 2011 - link
I have to agree.Anand came out blasting the Intel G2 SSD when it first came out for a very MINOR firmware snafu... yet people angry about Intel SSD's or Samsung 470's are very few and far between.
Anand came out blasting Crucial for having firmware issues as well - with absolutely no follow up. The C300 ended up being an absolutely fantastic drive (though we do see more complaints vs. intel 320/510 and Samsung 470).
It's getting old that you admit that all SSD's share extremely similar performance but continue to recommend SSD's that are FAR more unreliable vs. other brands.
If "real-world" performance among SSD's, you should really look at things that distinguish one from another (reliability, warranty, long-term performance, trim/garbage collection features, raid performance, cost/gig evaluation, etc).
Frankly, I think consumers are at the point where a 1% chance of SSD failure isn't worth .05% increase in performance. While those exact numbers aren't easy to come by - that's why we want you, Anand, to get the dirt for us.
Anand Lal Shimpi - Thursday, June 23, 2011 - link
Intel was held to a higher standard simply because with the X25-M you had to give up performance and the promise was you would have something that was more reliable than the competition.The C300 had several firmware issues to begin with and didn't do well over time as we showed in our TRIM torture tests, it's the former that kept me from recommending it early on and the latter that kept me from being all that interested in it in the long run.
In the past two articles I've recommended the Intel SSD 510 and it was my personal choice of SSD for the past three months. I do have to allow for the fact that I have yet to have a single issue with any SF-2281 drive and some users may feel like they want to take a chance on something that's potentially faster (and has better write amplification characteristics).
If it was my money I'd stick with the 510 but until I see a readily repeatable situation where the SF-2281 drives have issues I have to at least mention them as an option.
Take care,
Anand
jwilliams4200 - Thursday, June 23, 2011 - link
"The C300 had several firmware issues to begin with and didn't do well over time as we showed in our TRIM torture tests, it's the former that kept me from recommending it early on and the latter that kept me from being all that interested in it in the long run."So, now that the Vertex 3 has had firmware issues, and now that your test in this article shows that its speed degrades terribly after torture tests, and somewhat even with TRIM....
Basically, now that the V3 is shown to have the same or worse problems as you complained about with the C300...
The question is, why are you not giving the Vertex 3 the same derogatory treatement that you gave the C300?
Anand Lal Shimpi - Thursday, June 23, 2011 - link
I've had multiple C300s die in my lab, not even trying to torture them (it looks like I may have just had another one die as of last night). Thus far I haven't had any SF-2281 drives die on me and I haven't experienced the BSOD issue first hand.The C300's performance degraded pretty poorly under harsh but still reasonable conditions. If you run the same torture test on a Vertex 3, its performance doesn't degrade.
It's only when you completely fill a SF-2281 drive with incompressible data, then randomly write small block incompressible data all over the drive for an hour that you end up in a situation with reduced performance. While random writes do happen on all drives, it's highly unlikely that you'll take your system drive, fill it with H.264 videos, delete those videos, install Windows on the drive and then run some sort of application that writes purely random data all over the drive. The torture test I created for the SF drives in particular is specifically designed to look at worst case performance if you're running a very unusual workload.
I did an 8-month investigation on SandForce's architecture that proved even in my own personal system I never saw the sort of worst case performance I was concerned about. The four drives we deployed across AT editors came back with an average write amplification of 0.6, as in most of the data that was written to the drive was actually deduped/compressed and never hit NAND. Based on that I don't believe most users will see the worst case performance I put forth on the TRIM page, the exception being if you're using this drive purely for highly compressed media or fully random data.
Take care,
Anand
jwilliams4200 - Thursday, June 23, 2011 - link
"The C300's performance degraded pretty poorly under harsh but still reasonable conditions."You call running HD Tach on an SSD "reasonable conditions"? Seriously?
seapeople - Thursday, June 23, 2011 - link
Yes, I'm sure OCZ loves the fact that Anand mentions the Intel SSD 510 as being the better drive overall considering reliability like five times in this review.Not only that, but he explains in depth on page 3 that the extra performance from the Vertex 3 and other latest generation SSD's doesn't even matter in normal computing situations.
So, Anand's options are this: 1) Say that SSD performance differences don't really matter and you should stop reading review sites like this and just go buy an Intel for reliability, or 2) Mention the irrelevancies of SSD performance differences in passing and continue on to do a full performance review which concludes that the Sandforce drives are, in fact, the fastest drives available today as long as you can get past the BSOD issues which may or may not affect you.
Just because Anand chose option 2 does not mean he is in OCZ's pocket, it just means he likes reviewing SSD performance. This is very fortunate for us readers who enjoy reading such articles.
Anand Lal Shimpi - Thursday, June 23, 2011 - link
You are very correct - I've tested eight (more coming) SF-2281 drives and haven't had any issues. However by the same logic the sample size of complaints on the forums isn't statistically significant either.Despite my sample size being what it is, I continue to have the discussion about quality control and testing in every SF-2281 drive. If there was a repeatable way to bring about the BSOD issue on any (or some?) readily available platforms I'd have no problems completely removing the drive from the discussion altogether. Unfortunately that doesn't seem to be the case.
Instead what I do is lay out the options for the end user. If you want the best overall reliability, Intel's SSD 320 is likely the drive for you. If you want the best balance of performance and reliability then there's the Intel SSD 510. And finally if you want to take a chance but want the drive with the lowest write amp for most users, there's anything SF-2281 based.
For me personally the choice was Intel's SSD 510. I've moved it to a secondary role in my system to try and bug hunt the Vertex 3 on a regular basis.
Take care,
Anand
Tomy B. - Thursday, June 23, 2011 - link
Why Samsung 470 isn't included in any results?Spoogie - Thursday, June 23, 2011 - link
1) The first Vertex II I received was DOA.2) The second died completely after just eight months of light use.
3) The BSODs occurred about once every six sleep modes. The Kingston replacement never gives a BSOD.
Buyer beware.