Plextor Updates The Firmware on M5 Pro: Promises Increased Performance, We Test It
by Kristian Vättö on December 10, 2012 2:30 PM ESTPerformance Consistency
In our Intel SSD DC S3700 review we introduced a new method of characterizing performance: looking at the latency of individual operations over time. The S3700 promised a level of performance consistency that was unmatched in the industry, and as a result needed some additional testing to show that. The reason we don't have consistent IO latency with SSDs is because inevitably all controllers have to do some amount of defragmentation or garbage collection in order to continue operating at high speeds. When and how an SSD decides to run its defrag and cleanup routines directly impacts the user experience. Frequent (borderline aggressive) cleanup generally results in more stable performance, while delaying that can result in higher peak performance at the expense of much lower worst case performance. The graphs below tell us a lot about the architecture of these SSDs and how they handle internal defragmentation.
To generate the data below I took a freshly secure erased SSD and filled it with sequential data. This ensures that all user accessible LBAs have data associated with them. Next I kicked off a 4KB random write workload at a queue depth of 32 using incompressible data. I ran the test for just over half an hour, no where near what we run our steady state tests for but enough to give me a good look at drive behavior once all spare area filled up.
I recorded instantaneous IOPS every second for the duration of the test. I then plotted IOPS vs. time and generated the scatter plots below. Each set of graphs features the same scale. The first two sets use a log scale for easy comparison, while the last set of graphs uses a linear scale that tops out at 40K IOPS for better visualization of differences between drives.
The first set of graphs shows the performance data over the entire 2000 second test period. In these charts you'll notice an early period of very high performance followed by a sharp dropoff. What you're seeing in that case is the drive allocating new blocks from its spare area, then eventually using up all free blocks and having to perform a read-modify-write for all subsequent writes (write amplification goes up, performance goes down).
The second set of graphs zooms in to the beginning of steady state operation for the drive (t=1400s). The third set also looks at the beginning of steady state operation but on a linear performance scale. Click the buttons below each graph to switch source data.
Wow, that's bad. While we haven't run the IO consistency test on all the SSDs we have in our labs, the M5 Pro is definitely the worst one we have tested so far. In less than a minute the M5 Pro's performance drops below 100, which at 4KB transfer size is equal to 0.4MB/s. What makes it worse is that the drops are not sporadic but in fact most of the IOs are in the magnitude of 100 IOPS. There are singular peak transfers that happen at 30-40K IOPS but the drive consistently performs much worse.
Even bigger issue is that over-provisioning the drive more doesn't bring any relief. As we discovered in our performance consistency article, giving the controller more space for OP usually made the performance much more consistent, but unfortunately this doesn't apply to the M5 Pro. It does help a bit as it takes longer for the drive to enter steady-state and there are more IOs happening in the ~40K IOPS range, but the fact is that most IO are still handicapped to 100 IOPS.
The next set of charts look at the steady state (for most drives) portion of the curve. Here we'll get some better visibility into how everyone will perform over the long run.
Concentrating on the final part of the test doesn't really bring anything new because as we saw in the first graph already, the M5 Pro reaches steady-state very quickly and the performance stays about the same throughout the test. The peaks are actually high compared to other SSDs but having one IO transfer at 3-5x the speed every now and then won't help if over 90% of the transfers are significantly slower.
Facebook
Twitter
RSS
46 Comments
View All Comments
jwilliams4200 - Thursday, December 13, 2012 - link
If SSDs were not to be used for heavy workloads, then the warranty would state the prohibited workloads. But they do not, other than (most of them) giving a maximum total amount TB written.The fact is that SSDs are used for many different applications and many different workloads.
Just because you favor certain workloads does not mean that everyone does.
Fortunately, not everyone thinks like you do on this subject.
JellyRoll - Thursday, December 13, 2012 - link
I would ask you to explain why there is a difference between Enterprise class SSDs and consumer SSDs.If there is no difference in the design, firmware and expectations between these two devices then I am sure that the multi-billion dollar companies such as Google are awaiting your final word on this with baited breath. Why, they have been paying much more for these SSDs that are designed for enterprise workloads for no reason!
You would think that these massive companies, such as Facebook, Amazon, Yahoo, and Google with all of their PHDs and humongous datacenters would have figured out that there is NO difference between these SSDs!
Why for their datacenters (which average 12 football fields in size) they can just deploy Plextor M5s! By the tens of thousands!
*hehe*
JellyRoll - Wednesday, December 12, 2012 - link
Also, operating systems and applications that delete data, such as temp and swap files, that do not end up in the recycle bin are subject to immediate TRIM command issuance. These files, thousands of them, are deleted every user session and are never seen in the recycle bin. Care to guess how many temp and cache files Firefox deletes in an hour?jwilliams4200 - Thursday, December 13, 2012 - link
So now every SSD user must run a web browser and keep the temporary files on the SSD?Who named you SSD overlord?
skytrench - Thursday, December 13, 2012 - link
Though I side with JR, in that the published IO consistency check doesn't show anything useful, modified it might have been an interesting test...(different blocksizes, taking breaks from only writing, maybe a reduction of load to see at which level the drives still perform well etc.etc).I find the test disrespectful to Plextor, who after all have delivered an improved firmware, and Anand to come out of this one lacking in credibility.
JellyRoll - Thursday, December 13, 2012 - link
The point is that since the files are subject to immediate issuance of a TRIM command and this cannot be done when testing without a filesystem, is that you cannot attempt to emulate application performance with trace-based testing and the utility they are using. This doesn't just apply to browsers either, this applies to ALL applications. From games, to office docs, to Win7 (or 8), or a simple media player.All applications, and operating systems, have these types of files that are subject to immediate TRIM commands.
gostan - Tuesday, December 11, 2012 - link
Hi Vatto,possible to get the samsung 830 through the performance consistency test? I know it's been discontinued but it's still available for sale in many places.
Cheers!
Kristian Vättö - Tuesday, December 11, 2012 - link
I don't have any Samsung 830s at my place but I'll ask Anand to run it on the 830. He's currently travelling so it will probably take a while but I'll try to include it in an upcoming SSD review.lunadesign - Tuesday, December 11, 2012 - link
WOW! My jaw hit the floor when I first saw the 1st graph.I totally understand that the consistency test isn't realistic for consumer usage. But what about people using the M5 Pro in low-to-moderate use servers? For example, I'm about to set up a new VMware dev server with four 256GB M5 Pros in RAID 10 (using LSI 9260-8i) supporting 8-10 VMs. In this config, I won't have TRIM support at all.
If I understand this consistency test, the drive is being filled and then its being sent non-stop requests for 30 mins that overwrite data in 4KB chunks. This *seems* to be an ultra-extreme case. In reality, most server drives are getting a choppy, fluctuating mixture of read and write operations with random pauses so the GC has much more time to do it work than the consistency test allowed, right?
Should I be concerned about using the M5 Pro is low-to-moderate use server situations? Should I over-provision the drives by 25% to ameliorate this? Or, worse yet, are these drives so bad that I should return them while I can? Or is this test case so far away from what I'll likely be seeing that I should use the drives normally with no extra over-provisioning?
JellyRoll - Tuesday, December 11, 2012 - link
For a non-TRIM environment this would be especially bad. You are constrained to the lowest speed of each SSD. In practical use in a scenario such as yours you will be receiving the bottom line of performance constantly. The RAID is only as fast as the slowest member, and with each drive dropping into the lower state frequently, one SSD will surely be in this low range constantly. Therefore your RAID will suffer.These SSDs are simply not designed for this usage. They are tailored for low QD, and the test results show that clearly.