OCZ Vertex 3 MAX IOPS & Patriot Wildfire SSDs Reviewed
by Anand Lal Shimpi on June 23, 2011 4:35 AM ESTAnandTech Storage Bench 2011
Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. I assembled the traces myself out of frustration with the majority of what we have today in terms of SSD benchmarks.
Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.
Originally I kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system.
Not too long ago I tweeted that I had created what I referred to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. It's the load you'd put on a drive after nearly two weeks of constant usage. And it takes a *long* time to run.
1) The MOASB, officially called AnandTech Storage Bench 2011 - Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. My thinking was that it's during application installs, file copies, downloading and multitasking with all of this that you can really notice performance differences between drives.
2) I tried to cover as many bases as possible with the software I incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II & WoW are both a part of the test) as well as general use stuff (application installing, virus scanning). I included a large amount of email downloading, document creation and editing as well. To top it all off I even use Visual Studio 2008 to build Chromium during the test.
The test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:
| AnandTech Storage Bench 2011 - Heavy Workload IO Breakdown | ||||
| IO Size | % of Total | |||
| 4KB | 28% | |||
| 16KB | 10% | |||
| 32KB | 10% | |||
| 64KB | 4% | |||
Only 42% of all operations are sequential, the rest range from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.
Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result I'm going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time I'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, I will also break out performance into reads, writes and combined. The reason I do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.
There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. Lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.
As always I don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea.
The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests. All of the older tests are still run on our X58 platform.
AnandTech Storage Bench 2011 - Heavy Workload
We'll start out by looking at average data rate throughout our new heavy workload test:
The breakdown of reads vs. writes tells us more of what's going on:
The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles, this is just how long the SSD was busy doing something:
Facebook
Twitter
RSS
112 Comments
View All Comments
TrackSmart - Thursday, June 23, 2011 - link
My problem is not the failure rate of OCZ (or Corsair or Patriot) SSDs. Ordinary hard disks that we've been using forever have a 1-3% failure rate (or worse for some models). The problem is customer support for drives that aren't merely failing, but are *failing to be compatible* with particular hardware configurations.How a company deals with such problems is what determines if they deserve customer loyalty. And let me tell you that sending customers another of the same drive, that they know won't work with their system, is not a way to win customer loyalty. Just issue a refund and let the customer buy a different model. With that kind of treatment, the customer might buy another OCZ drive, but from one of the other product lines - instead of leaving angry.
The customer support issue is the one I want OCZ to address. And if Anandtech has personal communications with them, that is something worth passing on.
Anand Lal Shimpi - Thursday, June 23, 2011 - link
The issue is that, apparently, the majority of customers don't have a problem. There isn't a day that goes by where I'm not testing an SF-2281 drive in some fashion. Now it's possible that my usage model and test scenarios aren't enough to cause the problem, but I suspect that it's more complicated than that.My recommendation continues to be the Intel SSD 510 if you want the best balance of performance/predictable reliability. However if I simply recommend Intel's drive without mentioning a cheaper/faster drive I think I'll get accused of a different sort of bias :-P
Take care,
Anand
techinsidr - Thursday, June 23, 2011 - link
Good point Anand... I'm personally not a fan of OCZ products, but I give you props all the hard work you do on this site. Even if I don't agree with your recommendations 100% of the time, you still have the best hardware review site out there.I hope OCZ and Sandforce gets this problem figured out sooner than later. Performance is definitely important, but at the end of the day I think reliability is even more critical.
TrackSmart - Thursday, June 23, 2011 - link
I agree that it's a small percent, but is it okay to tell 10,000 customers "sorry, use it in a different computer" when they've spent $240 on a storage device? Just because it represents *only* 0.5% of their customers?Thanks for taking the time for a personal reply. And thanks for the superb articles and reporting. We'll just have to agree to disagree on what it means for a company to stand by a product.
Anand Lal Shimpi - Thursday, June 23, 2011 - link
No I agree that's not ok, but to the best of my knowledge OCZ has always facilitated a refund if you ask for one with situations like this. OCZ has apparently even offered to send engineers to your house and loan you a notebook to help diagnose the issue back when they had trouble tracking it down.My preference would be a stop shipment order until the problem is corrected. But in terms of customer service, I believe OCZ typically does take care of its customers.
Take care,
Anand
TrackSmart - Thursday, June 23, 2011 - link
Thanks Anand. I've had good experience with other OCZ products and associated support in the past. And it's possible that the 25nm debacle has caused them to shift their policies. At the time, when I purchased a Vertex 2, I was told replacement with a like item was the only option. Newegg came to my rescue and took the drive back, despite their clearly stated policy that SSD returns have to go to the manufacturer. Good on Newegg for understanding my situation. And hopefully good on OCZ if their policies have improved.A google search turns up people from the past who were turned down for Vertex 2/Agility 2 refunds (responses: firmware is on the way or use it in a compatible system - only exchanges for like models). But shows that Vertex 3 owners have been getting refunds.
So maybe I'm behind the times. And if so, my apologies to OCZ. These changes of policy must be within very recent past. A bit of bad press seems to go a long way to changing corporate policy.
jonup - Thursday, June 23, 2011 - link
Why is this review questionable? His recommendation is based on his personal experience with the drives. He has not experienced any BSODs but he disclosed that there are certain drives sold in retail that have the issue. He recommended OCZs drive as much as he recommended the Patriot's.I wouldn't be supprise if majority of the BSOD result from user tweaks and maybe if it is even a Windows problem.
semo - Thursday, June 23, 2011 - link
I wouldn't be as harsh and say that Anand has had underhand dealings with OCZ but there are bridges there. He gets new OCZ SSDs well before release dates so that he can prepare for in-depth articles. These articles bring a boat load of traffic to his domain.So indirectly, Anand does benefit from being on good terms with OCZ because if he didn't, he wouldn't have those golden samples and those spikes in traffic at launch.
Then again, timing is not the only thing that keeps us techies coming back. The quality of the articles plays a big part as well. I am however starting to question the Integrity of his work given the outcry in the community which is not proportionally reflected in Anand's official articles (this article touches on a few of the issues but the huge outcry from the last article must have played a big part). Also, I'm not just a techie, I'm also a consumer and I totally disagree with OCZ's handling of the 25nm fiasco and their customer relations in general.
Needless to say, I still enjoy Anand's articles but I will never touch an OCZ product again. Other companies provide just as much value and performance as them.
techinsidr - Thursday, June 23, 2011 - link
I couldn't agree more semo... I did a blog article covering this whole fiasco at my website techinsidr.com . The way OCZ has handled this fiasco leaves a lot to be desired..Anand Lal Shimpi - Thursday, June 23, 2011 - link
Access to hardware is never a reason to treat a manufacturer favorably. That may win you some short term gains but long term you lose credibility, which will ultimately and drives away the sort of quality readers you need to have a site truly survive.Also note that while exclusives do bring in more traffic, you actually build more traffic by being consistent and delivering quality over time. The former doesn't guarantee lasting traffic.
I don't believe OCZ is getting favorable treatment here, just the treatment that is representative of my experience with the drives. Now the response from the readership is clear that despite my experience, there's definite unease with actually jumping on the OCZ bandwagon until these sorts of complaints go away entirely. I can completely understand that sentiment and I will make sure it's reflected as we continue to cover this space.
Take care,
Anand