The Cleaning Lady and Write Amplification

Imagine you’re running a cafeteria. This is the real world and your cafeteria has a finite number of plates, say 200 for the entire cafeteria. Your cafeteria is open for dinner and over the course of the night you may serve a total of 1000 people. The number of guests outnumbers the total number of plates 5-to-1, thankfully they don’t all eat at once.

You’ve got a dishwasher who cleans the dirty dishes as the tables are bussed and then puts them in a pile of clean dishes for the servers to use as new diners arrive.

Pretty basic, right? That’s how an SSD works.

Remember the rules: you can read from and write to pages, but you must erase entire blocks at a time. If a block is full of invalid pages (files that have been overwritten at the file system level for example), it must be erased before it can be written to.

All SSDs have a dishwasher of sorts, except instead of cleaning dishes, its job is to clean NAND blocks and prep them for use. The cleaning algorithms don’t really kick in when the drive is new, but put a few days, weeks or months of use on the drive and cleaning will become a regular part of its routine.

Remember this picture?

It (roughly) describes what happens when you go to write a page of data to a block that’s full of both valid and invalid pages.

In actuality the write happens more like this. A new block is allocated, valid data is copied to the new block (including the data you wish to write), the old block is sent for cleaning and emerges completely wiped. The old block is added to the pool of empty blocks. As the controller needs them, blocks are pulled from this pool, used, and the old blocks are recycled in here.

IBM's Zurich Research Laboratory actually made a wonderful diagram of how this works, but it's a bit more complicated than I need it to be for my example here today so I've remade the diagram and simplified it a bit:

The diagram explains what I just outlined above. A write request comes in, a new block is allocated and used then added to the list of used blocks. The blocks with the least amount of valid data (or the most invalid data) are scheduled for garbage collection, cleaned and added to the free block pool.

We can actually see this in action if we look at write latencies:

Average write latencies for writing to an SSD, even with random data, are extremely low. But take a look at the max latencies:

While average latencies are very low, the max latencies are around 350x higher. They are still low compared to a mechanical hard disk, but what's going on to make the max latency so high? All of the cleaning and reorganization I've been talking about. It rarely makes a noticeable impact on performance (hence the ultra low average latencies), but this is an example of happening.

And this is where write amplification comes in.

In the diagram above we see another angle on what happens when a write comes in. A free block is used (when available) for the incoming write. That's not the only write that happens however, eventually you have to perform some garbage collection so you don't run out of free blocks. The block with the most invalid data is selected for cleaning; its data is copied to another block, after which the previous block is erased and added to the free block pool. In the diagram above you'll see the size of our write request on the left, but on the very right you'll see how much data was actually written when you take into account garbage collection. This inequality is called write amplification.


Intel claims very low write amplification on its drives, although over the lifespan of your drive a < 1.1 factor seems highly unlikely

The write amplification factor is the amount of data the SSD controller has to write in relation to the amount of data that the host controller wants to write. A write amplification factor of 1 is perfect, it means you wanted to write 1MB and the SSD’s controller wrote 1MB. A write amplification factor greater than 1 isn't desirable, but an unfortunate fact of life. The higher your write amplification, the quicker your drive will die and the lower its performance will be. Write amplification, bad.

Live Long and Prosper: The Logical Page Why SSDs Care About What You Write: Fragmentation & Write Combining
POST A COMMENT

296 Comments

View All Comments

  • jasperjones - Monday, September 07, 2009 - link

    Bit of a late reply to your question but a single overwrite with random data is fully secure. At least for HDDs there have been tests from academics that tried to recover data from a HDD after wiping it via "dd if=/dev/urandom bs=1M" They weren't able to recover anything. Reply
  • smjohns - Thursday, September 03, 2009 - link

    Great 3rd installment and I have learnt more about SSD's from this site than any other !!

    Whilst there is no doubt that Intel G2 definitely remains the SSD drive of choice (assuming you have the cash). Why did Intel choose not to address the poor sequential write speeds? In the above tests it seems no better than a standard 5400 hard disc....which is a little poor. I accept it is blisteringly fast for everything else but not sure why this was ignored / shelved?

    Is it that it is currently impossible to build a drive that can be fast at both large sequential and small random file writes? Or is it that the G2 was always intended to be an incremental improvement over the G1 (fixing some of its short comings) rather than a complete top to bottom redesign of the unit, which may have lead to this being addressed? As such could a future firmware release improve these speeds....or is it definitely a hardware restriction?

    I have to say I am personally torn between the OCZ Vertex and Intel G2 at the moment. Whilst I accept the G2 seems to be the quicker drive in the real world, I was disappointed that they did not improve the sequential write speeds and in addition to this, they do seem a little slow with support. The OCZ on the other hand seems a bit of an all rounder and not that much slower than the G2. In addition to this I REALLY like OCZ's approach to supporting these drives and they really seem to listen to their customers feedback.

    One final question....when installing an SSD into a laptop with a fresh Windows 7 install, is there now any need for special formatting / OS settings to ensure best drive performance / life? There is a lot of stuff on the web but it all seems particularly relevant for XP and partially Vista but I was under the impression that Win7 was designed to work with SSDs out of the box?
    Reply
  • derkurt - Thursday, September 03, 2009 - link

    Then, there is another reason why SSDs are not covered extensively by the mainstream press: They are too complicated.

    Let's say you want to buy a hard disk. You could just buy any hard disk, since the difference between good and bad ones is fairly small. If you buy an ExcelStor, for example, you will still get something which works and delivers sufficient performance compared to faster models. Unless you are looking at the server market, there is not that much difference at all. Some models have larger caches, faster seek times and higher transfer rates due to higher rotational speeds, but the main difference is capacity, so the market is transparent.

    Now look at the SSD market: The difference between good and bad ones is huge, incredibly huge. The Intel G2 is lightning fast while some old JMicron-based drives are much worse than a 5-years-old hard disk. You can't just go and buy "an SSD". You need to be informed:

    What controller is the SSD using? Do I have to align my partitions, or is my operating system detecting the SSD and doing that for me? Does my OS support TRIM? Does my AHCI driver support TRIM? Does my SSD support TRIM? Does my current firmware revision support TRIM, and if so, do I need to flash a beta firmware which still has some serious flaws in it? How is the performance degradation after heavy use? What about random write access times (very big differences here which strongly affect real world performance)?

    If you don't care about the above, chances are you will get a crappy drive. And even if you do, you'll have a hard time finding out some essential facts (thanks Anand!), since the manufacturers aren't exactly putting them on their webpages. They will tell you the capacity and the maximum linear transfer rates. That's all, basically. You will have to do some exhaustive googling to investigate what controller the drive is using, whether the firmware supports TRIM, and so on. Even Intel is holding back with detailed information, though they wouldn't have to, since they have nothing to hide as their drives are the fastest in nearly all aspects.

    I don't know for sure why the manufacturers are making a secret out of essential information, even if they can shine there. But there's one thing I do know: Only when people don't need to care about controllers, OS support, firmwares etc. anymore, SSDs are ready to hit the mainstream.
    Reply
  • smjohns - Thursday, September 03, 2009 - link

    I fully agree with you here and it is one of the reasons why I have not taken the plunge yet. I am definitely holding out for Win7 and then upgrade my laptop with both that and an SSD.

    Even after reading these great articles, whilst I now know which drives support Trim and the fact that none of them have this functionality fully enabled and will require a future firmware update ("shudders"), the SSD market is indeed a confusing place to be. And thats before you consider having to align partitions (what the heck is this) and the various settings in the BIOS / OS you need to enable / disable to ensure your lovely new drive does not die within a few weeks / months / years.

    If the industry really does want widespread adoption of these new drives, it needs to resolve these issues and come up with some easy and readily available standards we can all follow. I just hope Win7 is as SSD friendly as we are led to believe.
    Reply
  • derkurt - Thursday, September 03, 2009 - link

    quote:

    and thats before you consider having to align partitions (what the heck is this)


    AFAIK, "aligning" partitions means that the logical layout of blocks has to match the physical block assigment on the SSD in a certain way, otherwise writing one logical block on the filesystem level may result in an unnecessary I/O operation covering two blocks on the SSD (because the logical block spans the boundaries between two physical blocks). But don't ask me for details, I haven't dug into that yet.

    According to MS, Windows 7 detects SSDs and applies a proper alignment scheme automatically during installation of the OS. If you'd like to install a Linux distribution or an older version of Windows, you'll probably have to take care of that by yourself, unfortunately.

    quote:

    and the various settings in the BIOS / OS you need to enable / disable to ensure your lovely new drive does not die within a few weeks / months / years.


    I guess there aren't that many, you should just turn on AHCI support - the drive will work without it, but you need it for enabling NCQ, which can give you a 5-10% performance boost. However, you need to do this before the OS installation, otherwise your OS might cease to boot. Oh, and also you may have to temporarily turn off AHCI support when flashing a new firmware, because some flashing tools are struggling of AHCI is turned on.

    I hope that with the advent of Windows 7 going into public sale, SSD manufacturers will start to ship reliable, TRIM-enabled firmware revisions. If so, you shouldn't have to think about all these issues anymore as long as you are using Windows 7.
    Reply
  • derkurt - Thursday, September 03, 2009 - link

    I was one of the lucky guys to get an Intel G2 drive before they stopped shipping it for a while, and I can absolutely confirm everything Anand states about performance.

    However, I still wonder why there is relatively few competition out there. At least in theory, it takes far less know-how to produce a good SSD than is required to manufacture reliable hard disk drives - think about the expansive and complicated fine mechanics involved. Actually, there are some Chinese manufacturers most of us have never heard of, such as RunCore, which manage to deliver SSDs of at least usable quality.

    Where is the Samsung drive that blows the competition away? What about Seagate, Western Digital, Hitachi? Are they just watching from the sideways while SSDs from some young and small companies are cannibalizing their markets?

    At the time the shift from CRTs to LCDs was taking place, German premium TV manufacturer Loewe estimated that it would take many years until CRTs became obsolete. But the change happened so fast it nearly blew off their business before they finally started to ship high-quality LCDs in response to market demand. It seems to me that the very same thing is happening again now.

    The G2 is gorgeous, no doubt about it. But the price point is still way above being ready to hit the mainstream. Computers are simply not important enough to Joe Sixpack to spend 200+ USD for storage solutions only, even if it _really_ accelerates the machine (something most people won't believe until they experienced it themselves), and especially considering the low capacities offered by SSDs so far.

    If something as great as the G2 can be offered for 240 USD while being sold to a relatively small audience, what prices can we expect to see if the mainstream is hit? If USB sticks can be sold for less than 5 USD, what is the fundamental problem at reaching a price point of 60 USD for high-quality SSDs? Of course, SSDs contain much more intelligence than USB pen drives: Multi-channel controllers with sophisticated strategies, caches, and so on, but the main difference should be the effort required to engineer these devices, rather than the cost for building them.

    I am a bit frustrated that while there are SSDs available now which deliver superior performance, they still cover a small niche of enthusiasts (and there are probably a lot more people who would want to buy one if they only knew that these things exist), and the traditional hard drive manufacturers cease to join the game. The most important reason why Intel priced the G2 at a more affordable level is probably not the competition by Indilinx drives, but rather the idea that they can gain more profit by selling much more drives, even if they are sold at a lower price, as long as production costs are fairly small.

    Is Samsung sleeping, or are they just fearing that the shift to SSDs might destroy their mechanical hard drive business? I doubt that they don't have engineers capable of creating SSDs which deliver a performance comparable to Intel's drives. Maybe the mediocre performance of their SSDs is part of a strategy, which says that SSD development shouldn't be pushed too fast until the rest of the market is really forcing them to do so.

    Companies such as Apple need to sell good SSDs with their computers, by default. Why can't premium PC manufacturers like Apple sell their hardware with a G2 drive, while they are offering similarly expansive CPUs? If you are spending the 240 USD for a CPU upgrade instead, I'd take every bet that you were unable to feel a comparable performance gain. It's a shame that PC sellers are neglecting hard drive performance while at the same time stressing the CPU power of their systems in their advertisements. Only if Seagate & Co. realize that they are losing a large and growing market share by not joining the SSD race, prices will drop. So far, they just don't care about some hardware geeks like us.
    Reply
  • pepito - Monday, November 16, 2009 - link

    There are a bunch of companies selling SSD already, its just that you don't know where to find them, and most reviewers only care about big players, such as Intel or Samsung.

    If you check, for example, http://kakaku.com/pc/ssd/">http://kakaku.com/pc/ssd/ you can see there are currently 24 manufacturers listed there (use google translate, as its in japanese).

    Some you probably never heard of: MTRON, Greenhouse, Buffalo, CFD, Wintec, PhotoFast, etc.
    Reply
  • iwodo - Thursday, September 03, 2009 - link

    I have trouble understanding WHY Apple, uses Samsung CRAPPY SSD like everyone else when they could easily make their own.

    And SSD drive, like all Indlinx drive, are nothing more then Flash Chip soldered on to PCB with Indilinx Core. Apple is already the largest Flash buyer in the world, they properly buy the cheapest Flash memory in the market. ( Intel and Samsung of coz don't count since they make the flash themselfs. ) Building an SSD themself would be adding $20 dollars on top of 8 Chips 64Gb Flash.

    Why they dont build one and use it accross its Mac is beyond me. Since even the firmware is the same as everyone else.
    Reply
  • pepito - Monday, November 16, 2009 - link

    For the same reason that Dell doesn't make their own batteries, its not their business. Reply
  • Borski - Thursday, September 03, 2009 - link

    How does G.skill Falcon compare with the reviewed units? I've seen very good reviews (close to Vertex) elsewhere but they don't mention things like used vs new performance, or power consumption.

    I'm considering buying the G.Skill Falcon 64G, which is cheaper than Agility in some places.
    Reply

Log in

Don't have an account? Sign up now