Delving Deeper

I had suspicions as to the nature of the problem based on my experience with it in my Mac Pro. The SuperTalent MLC drive in my machine would pause, most noticeably, randomly when I'd want to send an IM. What happens when you send an IM? Your logfile gets updated; a very small, random write to the disk. I turned to Iometer to simulate this behavior.

Iometer is a great tool for simulating disk accesses, you just need to know what sort of behavior you want to simulate. In my case I wanted to write tons of small files to the drive and look at latency, so I told Iometer to write 4KB files to the disk in a completely random pattern (100% random). I left the queue depth at 1 outstanding IO since I wanted to at least somewhat simulate a light desktop workload.

Iometer reports four results of importance: the number of IOs per second, the average MB/s, the average write latency and the maximum write latency. I looked at performance of four drives, the OCZ Core (Jmicron controller MLC), OCZ SLC (Samsung controller), Intel MLC (Intel controller) and the Seagate Momentus 7200.2 (a 7200RPM 2.5" notebook drive).

Though the OCZ core drive is our example, but please remember that this isn't an OCZ specific issue: the performance problems we see with this drive are apparent on all current MLC drives in the market that use a Jmicron controller with Samsung flash.

4KB, 100% random writes, IO queue depth 1 IOs per Second MB/s Average Write Latency Max Write Latency
OCZ Core (JMicron, MLC) 4.06 0.016MB/s 244ms 991ms
OCZ (Samsung, SLC) 109 0.43MB/s 9.17ms 83.2ms
Intel X25-M (Intel, MLC) 11171 43.6MB/s 0.089ms 94.2ms
Seagate Momentus 7200.2 106.9 0.42MB/s 9.4ms 76.5ms

 

Curiouser and curiouser...see a problem? Ignore the absolute ridiculous performance advantage of the Intel drive for a moment and look at the average latency column. The OCZ MLC drive has an average latency of 244 ms, that's over 26x the latency of the OCZ SLC drive and 25.9x the latency of a quick notebook drive. This isn't an MLC problem however, because the Intel MLC drive boasts an average latency of 0.09ms - the OCZ MLC drive has a 2700x higher latency!

Now look at the max latency column, the worst case scenario latency for the OCZ Core is 991ms! That's nearly a full second! This means that it takes an average of a quarter second to write a 4KB file to the drive and worst case scenario, a full second. We complain about the ~100 nanosecond trip a CPU has to take to main memory and here we have a drive that'll take nearly a full second to complete a task - totally unacceptable.

In order to find out if the latency is at all tied to the size of the write I varied the write size from 4KB all the way up to 128KB, but kept the writes 100% random. I'm only reporting latencies here:

100% random writes, IO queue depth 1 4KB 16KB 32KB 64KB 128KB
OCZ Core (JMicron, MLC) 244ms 243ms 241ms 243ms 247ms
OCZ (Samsung, SLC) 9.17ms 14.5ms 21.2ms 28ms 28.5ms
Intel X25-M (Intel, MLC) 0.089ms 0.23ms 0.44ms 0.84ms 1.73ms
Seagate Momentus 7200.2 9.4ms 8.95ms 9.14ms 9.82ms 12.1ms

 

All the way up to 128KB the latency is the same, 0.25s on average and nearly a second worst case for the OCZ Core and other similar MLC drives. If it's not the file size, perhaps it's the random nature of the writes?

For this next test I varied the nature of the writes, I ran the 4KB write test with a 100% sequential workload, 90% sequential (10% random) and 50% sequential (50% random):

4KB writes, IO queue depth 1 100% Sequential/0% Random 90% Sequential/10% Random 50% Sequential/50% Random 0% Sequential/100% Random
OCZ Core (JMicron, MLC) 0.36ms 25.8ms 130ms 244ms
OCZ (Samsung, SLC) 0.16ms 1.97ms 5.19ms 9.17ms
Intel X25-M (Intel, MLC) 0.09ms 0.09ms 0.09ms 0.089ms
Seagate Momentus 7200.2 0.16ms 0.94ms 4.35ms 9.4ms

 

The average latency was higher on the OCZ Core (MLC) than the rest of the drives, but still manageable at 0.36ms when I ran the 100% sequential test, but look at what happened in the 90% sequential test. With just 10% random writes the average latency jumped to 25.8ms, that's 13x the latency of the OCZ SLC drive. Again, this isn't an MLC issue as the Intel drive does just fine. Although I left it out of the table to keep things simpler, the max latency in the 90/10 test was 983ms for the OCZ Core drive once again. The 90/10 test is particularly useful because it closely mimics a desktop write pattern, most writes are sequential in nature but a small percentage (10% or less) are random in nature. What this test shows us is that even 10% of random writes is all it takes to bring the OCZ Core to its knees.

The problem gets worse as you increase the load on the drive. Most desktop systems have less than 1 outstanding IO during normal operation, but under heavy multitasking you can see the IO queue depth hit 4 or 5 IOs for writes. Going much above that and you pretty much have to be in a multi-user environment, either by running your machine as a file server or by actually running a highly trafficked server. I ran the same 100% random, 4KB write test but varied the number of outstanding IOs from 1 all the way up to 64. Honestly, I just wanted to see how bad it would get:

This is just ridiculous. Average write latency climbs up to fifteen seconds, while max latency peaked at over thirty seconds for the JMicron based MLC drives. All this graph tells you is that you shouldn't dare use one of these drives in a server, but even at a queue depth of four the max latency is over two seconds which is completely attainable in a desktop scenario under heavy usage. I've seen this sort of behavior first hand under OS X with the SuperTalent MLC drive, the system will just freeze for anywhere from a fraction of a second to over a full second while a write completes in the background. The write that will set it off will often times be something as simple as writing to my web browser's cache or sending an IM, it's horribly frustrating.

I did look at read performance, and while max latency was a problem (peaking at 250ms) it was a fairly rare case, average latency was more than respectable and comparable to the SLC drives. This seems to be a write issue. Let's see if we can make it manifest itself in some real world tests.

Enter the Poorly Designed MLC The Generic MLC SSD Problem in the Real World
Comments Locked

96 Comments

View All Comments

  • mindless1 - Thursday, September 11, 2008 - link

    Sometimes the cure is worse than the problem.
  • Gannon - Tuesday, September 9, 2008 - link

    Don't worry derek I still heart you guys! :P

    Here's some cool software to check out (they have free trial version)

    http://www.whitesmoke.com/landing_flash/free_hotfo...">http://www.whitesmoke.com/landing_flash...otforwor...

    Maybe it will help escape complaints from the grammar nazi's, I think a lot of grammar is BS anyway. Language evolves constantly. It's a flexible tool to communicate.
  • Nihility - Monday, September 8, 2008 - link

    An excellent review. The benchmark results were always confusing in the past. No one would try to explain why an SSD with seemingly superior specs can't outperform a 7200 drive in media test. Thanks for putting the time in to resolve this issue.

    As for buying a drive like that, the price is still too steep for me to consider and you definitely made it clear that buying a jmicro SSD is out of the question.
    As for further testing, I'm very interested in seeing how a good SSD performs as an external drive over USB. The robustness and sturdiness of the drive is very important for something you lug around. We all know how bad bandwidth is over USB but I wonder how the latency will fair.

    Keep up the good work.
  • kmmatney - Monday, September 8, 2008 - link

    One of the other reviews I read said this SSD's controller will learn hard drive usage patterns, and get faster over time. Any tests of this feature?
  • leexgx - Monday, September 8, 2008 - link

    not sure how thay can lern

    i did wunder why thay never put any DRAM buffer on SSD drives as i was expecting SSD to suffer badly from lack of buffer any MLC drives basicly suck (16kb buffer per flash chip) unless its the intel MLC drive lol or an SLC drive seem mostly ok, but an intel SLC going to rock when thay get tested
  • Anand Lal Shimpi - Monday, September 8, 2008 - link

    The Intel drive will learn hard drive usage patterns however it does so over an extremely long period of time, not something I could develop a test for in my time with the drive.

    Take care,
    Anand
  • whatthehey - Monday, September 8, 2008 - link

    ...that doesn't think too much about HDD performance, particularly when we're talking about insane prices. Sure, rebooting and reloading all of your apps will feel much faster. Personally, when I reboot I walk into the kitchen or bathroom, walk back a few minutes later, and I don't notice the delays. Not to mention, I only reboot about once a month (usually when nVidia releases a new driver that I need to install).

    Another major problem I have is the tests as an indication of the "real world". Take the whole antivirus thing. I hate AV software and software firewalls, which is why I don't use Norton, AVG, Avast, McAfee or any other product that kills performance, sucks up memory, and only prevents virii/trojans after an update. AV software is just a BS excuse to pay a $60/year subscription and get nags every time your subscription expires. So there's on "real" scenario I don't ever encounter.

    Archive extraction can be pretty disk intensive as well, but how often do you need to extract a 5GB archive? Okay, so let's say you're a pirate and you do that daily... great. Now you can extract faster, but you have an SSD that can only hold 14 or so large archives. It's a nice illustration of SSDs being faster, but it's completely impractical. I have a 1TB drive just for all the movies, images, music, and disc images I have floating around.

    The tests show that SSDs can help a lot, but I for one use capacity far more. Between several games, my standard apps, and Vista I think I would use most of the 80GB. Then I think of the price and I could grab a couple VelociRaptors or even four 1TB Samsung F1 drives. I'll be truly impressed when I can get at least 320GB of SSD for less than $200. Actually, it's more like I want a good SSD with a reasonable capacity for under $100. Until then I'll just stick with my slower drives and avoid worst-case situations where HDD performance is a problem as much as possible.

    The article was good, and I appreciate the info on the MLC issues with JMicron. That confirms my suspicion that inexpensive flash drives are worse than standard mechanical drives. Intel has addressed the problem, but price is now back to where we were last year it seems. I guess the real problem is that I'm just not enough of an "enthusiast" to spend this much money on 80GB of storage... not counting stuff like that old 4GB hard drive back in the day that set me back over $200. Give it a few more cycles and I think I'll be ready for SSDs.

    PS - Also, who cares about $600 CPUs when you can buy $200 CPUs and overclock to higher performance levels? I don't think we'll ever see overclocked SSDs or HDDs.
  • DerekWilson - Tuesday, September 9, 2008 - link

    i wouldn't be so sure about not seeing overclocked SSD ...

    as this article points out, intel puts a focus on reliability ... but to do so they do sacrifice performance. the voltage applied to the transistors to store data is calibrated to write the cells quickly while maintaining a good life span. a higher voltage could be applied that would allow the cells to be written faster but would reduce the number of writes that a cell could handle.

    if intel says 100gb a day for 5 years ... i don't need that by a long shot. i would be very willing to sacrifice a lot of that for more speed.

    i actually spoke with intel about the possibility of overclocking their ssd drives at idf -- it is something that could be done as it is controlled via the firmware of the drive. if intel doesn't convolute their firmware too much or if they allow enthusiasts to have the necessary control over settings at that level we could very well see overclocked SSDs ...

    which would be very interesting indeed.
  • shabby - Monday, September 8, 2008 - link

    I was so close in buying one of those ocz drives, in fact the reason i didnt buy it was because it was a special order that took 2 weeks.
    Excellent write up, especially about the jmicron/mlc "glitch".
  • OCedHrt - Monday, September 8, 2008 - link

    Any reason why the WD GP drive does so well in the multitasking test? Even better than the VelociRaptor?

Log in

Don't have an account? Sign up now