OCZ Vertex 3 MAX IOPS & Patriot Wildfire SSDs Reviewed
by Anand Lal Shimpi on June 23, 2011 4:35 AM ESTAnandTech Storage Bench 2011
Last year we introduced our AnandTech Storage Bench, a suite of benchmarks that took traces of real OS/application usage and played them back in a repeatable manner. I assembled the traces myself out of frustration with the majority of what we have today in terms of SSD benchmarks.
Although the AnandTech Storage Bench tests did a good job of characterizing SSD performance, they weren't stressful enough. All of the tests performed less than 10GB of reads/writes and typically involved only 4GB of writes specifically. That's not even enough exceed the spare area on most SSDs. Most canned SSD benchmarks don't even come close to writing a single gigabyte of data, but that doesn't mean that simply writing 4GB is acceptable.
Originally I kept the benchmarks short enough that they wouldn't be a burden to run (~30 minutes) but long enough that they were representative of what a power user might do with their system.
Not too long ago I tweeted that I had created what I referred to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. It's the load you'd put on a drive after nearly two weeks of constant usage. And it takes a *long* time to run.
1) The MOASB, officially called AnandTech Storage Bench 2011 - Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. My thinking was that it's during application installs, file copies, downloading and multitasking with all of this that you can really notice performance differences between drives.
2) I tried to cover as many bases as possible with the software I incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II & WoW are both a part of the test) as well as general use stuff (application installing, virus scanning). I included a large amount of email downloading, document creation and editing as well. To top it all off I even use Visual Studio 2008 to build Chromium during the test.
The test has 2,168,893 read operations and 1,783,447 write operations. The IO breakdown is as follows:
| AnandTech Storage Bench 2011 - Heavy Workload IO Breakdown | ||||
| IO Size | % of Total | |||
| 4KB | 28% | |||
| 16KB | 10% | |||
| 32KB | 10% | |||
| 64KB | 4% | |||
Only 42% of all operations are sequential, the rest range from pseudo to fully random (with most falling in the pseudo-random category). Average queue depth is 4.625 IOs, with 59% of operations taking place in an IO queue of 1.
Many of you have asked for a better way to really characterize performance. Simply looking at IOPS doesn't really say much. As a result I'm going to be presenting Storage Bench 2011 data in a slightly different way. We'll have performance represented as Average MB/s, with higher numbers being better. At the same time I'll be reporting how long the SSD was busy while running this test. These disk busy graphs will show you exactly how much time was shaved off by using a faster drive vs. a slower one during the course of this test. Finally, I will also break out performance into reads, writes and combined. The reason I do this is to help balance out the fact that this test is unusually write intensive, which can often hide the benefits of a drive with good read performance.
There's also a new light workload for 2011. This is a far more reasonable, typical every day use case benchmark. Lots of web browsing, photo editing (but with a greater focus on photo consumption), video playback as well as some application installs and gaming. This test isn't nearly as write intensive as the MOASB but it's still multiple times more write intensive than what we were running last year.
As always I don't believe that these two benchmarks alone are enough to characterize the performance of a drive, but hopefully along with the rest of our tests they will help provide a better idea.
The testbed for Storage Bench 2011 has changed as well. We're now using a Sandy Bridge platform with full 6Gbps support for these tests. All of the older tests are still run on our X58 platform.
AnandTech Storage Bench 2011 - Heavy Workload
We'll start out by looking at average data rate throughout our new heavy workload test:
The breakdown of reads vs. writes tells us more of what's going on:
The next three charts just represent the same data, but in a different manner. Instead of looking at average data rate, we're looking at how long the disk was busy for during this entire test. Note that disk busy time excludes any and all idles, this is just how long the SSD was busy doing something:
Facebook
Twitter
RSS
112 Comments
View All Comments
L. - Thursday, June 23, 2011 - link
Would you handle a website like this for free forever ?--
Besides, that's not the only huge bias there is on Anandtech, but then it's like that mostly everywhere on the interwebz -- just get used to reading through ;)
The 1% may have easily been a true figure, it's statistics, matter of presentation :
"In the first month, about 1% of the drives shipped (to resellers) have been returned for RMA (by buyers)"
See.. that's one huge % that, taking into account customer laziness, various firmware tests and the stock @ resellers could mean about 30% of the drives fail (which of course cannot be the case or it would be a real riot and not just a few whiners on forums).
It could also be (not here) :
"In the past 10 years, about 1% of the drives shipped (to resellers) have been returned for RMA (by buyers)"
Anand Lal Shimpi - Thursday, June 23, 2011 - link
I outlined where the 0.66% figure came from in the article. Take all complaints received on the forums + tech support channels and divide that number by the total number of drives sold through (not just sold to retailers).I've gotten six more OCZ SF-2281 drives in the past week alone - partially to see if this is something that's caused at the drive level. Given that I still haven't seen it yet, I'm beginning to think that this is an issue that really requires a combination of the right platform and one of these drives.
Take care,
Anand
jwilliams4200 - Thursday, June 23, 2011 - link
"I outlined where the 0.66% figure came from in the article. Take all complaints received on the forums + tech support channels and divide that number by the total number of drives sold through (not just sold to retailers)."But that is absurd. That is nothing but a lower limit on the percentage of people who have trouble.
Doesn't OCZ have any clue about how many people use their computers? The people posting on the forums or contacting tech support are only a fraction of the people who use the products -- and usually the more savvy ones. A lot of people use their computer, and if it crashes, they have no idea what caused the crash. They just reboot it and keep going. Or if they contact tech support, it may be for a completely different product, since they do not know which one caused the crash.
It would not be crazy to guess that only 10% of people who experienced the problem with OCZ SSDs actually identified the cause of the crash as the SSD, and followed through to contact OCZ about it by phone or forum.
So the true scope of the problem could easily be 6%, or even higher. The 0.66% figure is just a useless lower limit spit out by OCZ's juggernaut propaganda machine.
Proph3T08 - Thursday, June 23, 2011 - link
I think you might be exaggerating the number of computers being used with OCZ SSDs.The last time I went to Best Buy I didn't see any of their display model computers shipping with a ssd.
I think in general SSDs are still used mostly by tech savvy users. Arguging that Anand's 0.66% is useless where he actually gave a source of information is ridiculous when your argument is a random guess.
Maybe find out how many OEMs actually ship with OCZ drives then maybe you could come up with a compelling argument.
jwilliams4200 - Thursday, June 23, 2011 - link
You missed the point, Proph3T08. I did not claim to know the percentage, I just gave an example. The point is that OCZ's number is indisputably nothing but an extreme lower limit. To claim otherwise is absurd.name99 - Friday, June 24, 2011 - link
Of course MacBook Airs all ship with SSDs...I don't know the numbers for how many SSDs ship on non-Air macs, but I expect the number is pretty high.
Note also that people are quick to complain that Apple never ships leading edge SSDs --- but it is worth noting that there have been no large scale outcries (even at the level of 1% complains) against the SSDs that Apple does ship.
I think Apple believes (and most customers agree) that reliability is vastly more important than the ability to win a benchmark that, after all is said and done, does not really reflect the real-world experience.
As for OCZ's claims, I'd agree that they are certainly at the lower limit. I've bought three SSDs in my time, one from OCZ, and I've been bitterly disappointed by all three of them, ALL of which hang the machine when fed a long sequence of writes. I've not complained in public forums or tried to get a replacement (which is apparently where OCZ get their data) because, what's the point? They won't give me my money back --- all they'll give me (after I spend $10 on mailing the disk to them) is a replacement crappy drive that behaves in exactly the same way.
Much easier just to conclude that
(a) this is a business populated by charlatans and scam artists
(b) Anand, unfortunately, has been way too tolerant of this sort of crap (all devices bought on his recommendation)
(c) the ONLY vendor of SSDs today that doesn't seem a complete a**hole is buying an SSD built into an Apple product. Sad but true.
sam. - Saturday, June 25, 2011 - link
@name99I have to agree with you there. I never complained about my SSD when it failed, I just went straight to RMA and paid the $11 postage. Almost think they make you pay so much for an SSD because they know they will have to send you another one down the track.
jwilliams4200 - Friday, June 24, 2011 - link
Also consider that there are probably a great many people who have the crashes, who then go to the forums, read the posts others made about the crashes, and think, okay, that is the problem I am having. No need to post, I'll just wait for a fix.velis - Thursday, June 23, 2011 - link
It's OCZ who's pumping the SSD novelities the fastest out there. As fast as they go for OCZ reviews, they also go for C300, m4 and intel ones.Anand is just crazy for anything SSD.
If you don't like it, don't read it...
Dam, now I'm putting my hand into fire for Anand :P }:-)))
TrackSmart - Thursday, June 23, 2011 - link
Ha Ha. I agree on three points raised in this thread: 1) OCZ quickly gets their SSDs out to Anandtech for review, 2) OCZ gets products to market quickly, and 3) Anand is crazy for anything SSD.All of that said, this is one of the better tech sites on the web and I have a lot of respect for Anand and others. To Anand: Keep up the good work and don't take the crazies too much to heart. You have my props for explicitly addressing all of the major "concerns" that have popped up of late. That said, you'll never satisfy all of the people who have raised them. Rational people will continue coming back to the site to read about the latest tech.