A Month with Apple's Fusion Drive
by Anand Lal Shimpi on January 18, 2013 9:30 AM EST- Posted in
- Storage
- Mac
- SSDs
- Apple
- SSD Caching
- Fusion Drive
Fusion Drive: Under the Hood
I took the 27-inch iMac out of the box and immediately went to work on Fusion Drive testing. I started filling the drive with a 128KB sequential write pass (queue depth of 1). Using iStat Menus 4 to actively monitor the state of both drives I noticed that only the SSD was receiving this initial write pass. The SSD was being written to at 322MB/s with no activity on the HDD.
After 117GB of writes the HDD took over, at speeds of roughly 133 - 175MB/s to begin with.
The initial test just confirmed that Fusion Drive is indeed spanning the capacity of both drives. The first 117GB ended up on the SSD and the remaining 1TB of writes went directly to the HDD. It also gave me the first indication of priority: Fusion Drive will try to write to the SSD first, assuming there's sufficient free space (more on this later).
Next up, I wanted to test random IO as this is ultimately where SSDs trump hard drives in performance and typically where SSD caching or hybrid hard drives fall short. I first tried the worst case scenario, a random write test that would span all logical block addresses. Given that the total capacity of the Fusion Drive is 1.1TB, how this test was handled would tell me a lot about how Apple maps LBAs (Logical Block Addresses) between the two drives.
The results were interesting and not unexpected. Both the SSD and HDD saw write activity, with more IOs obviously hitting the hard drive (which consumes a larger percentage of all available LBAs). The average 4KB (QD16) random write performance was around 0.51MB/s, it was constrained by the hard drive portion of the Fusion Drive setup.
After stopping the random write task however, there was immediate moving of data between the HDD and SSD. Since the LBAs were chosen at random, it's possible that some (identical or just spatially similar) addresses were picked more than once and those blocks were immediately marked for promotion to the SSD. This was my first experience with the Fusion Drive actively moving data between drives.
A full span random write test is a bit unfair for a consumer SSD, much less a hybrid SSD/HDD setup with roughly an 1:8 ratio of LBAs. To get an idea of how good Fusion Drive is at dealing with random IO I constrained the random write test to the first 8GB of LBAs.
The resulting performance was quite different. For the first pass, average performance was roughly 7 - 9MB/s, with most of the IO hitting the SSD and a smaller portion hitting the hard drive. After the 3 minute test, I waited while the Fusion Drive moved data around, then repeated it. For the second run, total performance jumped up to 21.9MB/s with more of the IO being moved to the SSD although the hard drive was still seeing writes.
In the shot to the left, most random writes are hitting the SSD but some are still going to the HDD, after some moving of data and remapping of LBAs nearly all random writes go to the SSD and performance is much higher
On the third attempt, nearly all random writes went to the SSD with performance peaking at 98MB/s and dropping to a minimum of 35MB/s as the SSD got more fragmented. This told me that Apple seems to dynamically map LBAs to the SSD based on frequency of access, a very pro-active approach to ensuring high performance. Ultimately this is a big difference between standard SSD caches and what Fusion Drive appears to be doing. Most SSD caches seem to work based on frequency of read access, whereas Fusion Drive appears to (at least partially) take into account what LBAs are frequently targeted for writes and mapping those to the SSD.
Note that subsequent random write tests produced very different results. As I filled up the Fusion Drive with more data and applications (~80% full of real data and applications), I never saw random write performance reach these levels again. After each run I'd see short periods where data would move around, but random IO hit the Fusion Drive in around an 7:1 ratio of HDD to SSD accesses. Given the capacity difference between the drives, this ratio makes a lot of sense. If you have a workload that is composed of a lot of random writes that span all available space, Fusion Drive isn't for you. Given that most such workloads are confined to the enterprise space, that shouldn't really be a concern here.
Facebook
Twitter
RSS
127 Comments
View All Comments
Richard Fairbanks - Saturday, January 19, 2013 - link
Thanks, Anand, for yet another timely article!I do almost all my work in code (i.e. text) with few graphics. I want to ensure reliability in case of disk failure.
Thus I am considering getting a 2012 Mac mini, opening it up, and adding a 256GB Samsung 840 Pro, in addition to the default 1TB HDD. (The 256GB capacity would allow me a 25+% spare area.) This is my ideal configuration for many reasons.
If I partition the HDD to match the 256GB SSD (leaving ~750MB for random, non-critical data), is it possible to create a RAID 1 array between the SSD and the 256GB HDD partition? (Full backups are made daily.)
In theory, this would allow all the array reads to come from the SSD for fastest response, and still maintain a mirrored HDD that could be booted from should the SSD fail. (If only the HDD partition could be a ZEVO ZFS format! ;-) )
Thoughts? Thanks!!
NCM - Saturday, January 19, 2013 - link
Richard asks: "If I partition the HDD to match the 256GB SSD (leaving ~750MB for random, non-critical data), is it possible to create a RAID 1 array between the SSD and the 256GB HDD partition?"That's an interesting question. I think the problem would be that there is no "master" disk in a RAID 1 array. Each slice is treated equally. You're hoping that read/write activity would be first served by the faster SSD, with the HD slice catching up in the background on its own time. I don't know that there's any evidence it would work like that, or, putting it another way, that anyone has written a RAID controller to make it happen that way.
It would be interesting to try it out.
We have some Mac Pro towers that I've set up SSD boot/application drives, but we rely on conventional Time Machine backups to an internal HD rather than a RAID mirror.
name99 - Saturday, January 19, 2013 - link
It is possible to create a RAID 1 in the way you are thinking using AppleRAID.What you want to do is simple enough that you can do it in DIsk Utility using the GUI.
If you really insist on going hardcore, hit Terminal and look at diskutil.
And you can boot off such an AppleRAID system.
HOWEVER I suspect you will be very unhappy with the results. A system like that can deliver snappy reads (because they'll mostly come from the SSD) but writes will be gated by the HD, and the system will frequently feel an HD system.
It is ALSO possible that you won't even get the read speeds you imagine.
When I used AppleRAID in this way (mirroring two HDs) a few years ago, it seemed to me that reads were also slower, and my assumption was that the system, assuming you cared primarily about data correctness (that's why you were mirroring rather than striping), performed both reads and compared the results before passing them up to the file system. Which suggests that your reads will ALSO be gated by the HD performance.
I'm also not sure what problem you believe you are solving with this. SSD failures are simply not that common. You can protect against them using Time Machine. If you REALLY are scared, you can have Time Machine alternate between two (or more) different backup drives.
It seems like a huge amount of pain to solve a problem that barely exists and that can be protected against much better in other ways.
name99 - Saturday, January 19, 2013 - link
To add to what I said, the AppleRAID mirroring stuff DOES work in terms of reliability, in that if one disk dies, you can just pop it out, replace it, and have the other disk copy to it. But, as I said, you pay a substantial hit in performance for this privilege.cjb110 - Saturday, January 19, 2013 - link
Gaming would have been an interesting 'use' case for the Fusion. When your playing you obviously want the fast access of SSD, but unless its your favourite game, it might not get used much and moved to the HDD.Also Games being much larger 'applications' would quickly fill the SSD if the Fusion just had a simple 'If App = On SSD" rule.
klaudyuxxx - Saturday, January 19, 2013 - link
They reinvented the wheel. 128 GB flash + 1-3TB HDD fused into a single volume?! AKA HYBRID SAMSUNG HARD DRIVESNCM - Saturday, January 19, 2013 - link
You really haven't bothered to read the article, have you? Or perhaps it's a reading comprehension issue.nerd1 - Saturday, January 19, 2013 - link
Typical apple - charging $$$$ for non-tech-savy people.It's way better to have a proper SSD (most laptops and desktops now have mSATA port) in terms of both performance and cost. Yes, I know that swapping the HDD of any apple device kills the warranty and most apple customers don't know how to upgrade a single component.....
Andhaka - Monday, January 21, 2013 - link
Nope, swapping the HDD with a SSD does not kill the warranty and many Apple users do that (I have done it on a 4 years old Macbook).But if other people find it better to pay for the Fusion solution (and a good solutions it seems to be) good for them.
Cheers
pichemanu - Saturday, January 19, 2013 - link
Hi Anand,i saw that in order to test a "pure ssd" setup you connected a 830 ssd to the imac over USB 3. As far as i know the best transfer rate over USB 3 is around 250 MB/s and the worst is well... terrible.
Considering the best case scenario for the iMac:
-USB 3 connected SSD would do 250 MB/s
-SATA 3 connected SSD would do 322 MB/s (taken from your article)
The performance would be 6.94 for fusion drive and 10.19 for a "pure SSD". This is an increase from 114% advantage for the "pure SSD" to 147% advantage for the "pure SSD".
If on the other hand your USB connected SSD did not write at max and a SATA 3 connected SSD would (that is 350 for the samsung 830 on an intel Z77 SATA 3 port) that difference would skyrocket.
Did you check that on your particular workload the USB 3 connection was not a bottleneck?
Thank you.