A Month with Apple's Fusion Drive
by Anand Lal Shimpi on January 18, 2013 9:30 AM EST- Posted in
- Storage
- Mac
- SSDs
- Apple
- SSD Caching
- Fusion Drive
Management Granularity
Much of Apple’s marketing on Fusion Drive talks about moving data at the file and application level, but in reality data can be moved between the SSD and HDD portions in 128KB blocks.
Ars actually confirmed this a while ago, but I wanted to see for myself. Using fs_usage I got to see the inner workings of Apple's Fusion Drive. Data is moved between drives in 128KB blocks, likely determined by frequency of use of those blocks. Since client workloads tend to be fairly sequential (or pseudo-random at worst) in nature, it's a safe bet that if you're accessing a single LBA within a 128KB block that you're actually going to be accessing more LBAs in the same space. The migration process seems to happen mostly during idle periods, although I have seen some movement between drives during light IO.
What’s very interesting is just how quickly the migration is triggered after a transfer occurs. As soon as file copy/creation, application launch or other IO activity completes, there’s immediate back and forth between the SSD and HDD. As you fill up the Fusion Drive, the amount of data moved between the SSD and HDD shrinks considerably. Over time I suspect this is what should happen. Infrequently accessed data should settle on the hard drive and what really matters will stay on the SSD. Apple being less aggressive about evicting data from the SSD as the Fusion Drive fills up makes sense.
The migration process itself is pretty simple with data being marked for promotion/demotion, it being physically copied to the new storage device and only then is it moved. In the event of a power failure during migration there shouldn't be any data loss caused by the Fusion Drive, it looks like only after two copies of the 128KB block are in place is the source block removed. Apple told me as much last year, but it's good to see it for myself.
By moving data in 128KB blocks between the HDD and SSD, Apple enjoys the side benefit of partially defragmenting the SSD with all writes to it. Even though the Fusion Drive will prefer the SSD for all incoming writes (which can include smaller than 128KB, potentially random/pseudo-random writes), any migration from the HDD to the SSD happens as large block sequential writes, which will trigger a garbage collection/block recycling routine in cases of a heavily fragmented drive. Performance of the SSD can definitely degrade over time, but this helps keep it higher than it would otherwise given that the SSD is almost always running at full capacity and the recipient of all sorts of unrelated writes. As I mentioned earlier, I would’ve preferred a controller with more consistent IO latency or for Apple to set aside even more of the PM830’s NAND as spare area. I suspect cost was the deciding factor in sticking with the standard amount of overprovisioning.
Facebook
Twitter
RSS
127 Comments
View All Comments
mutatio - Saturday, January 19, 2013 - link
Thanks, Anand, for the review. It would be interesting to see some comparisons to aftermarket Fusion setups as have been discussed in the comments here. e.g., How does the Fusion arrangement pan out with a 256GB or 512GB SSD on a Mac Mini? If my Mac were not a laptop I'd likely go for a Fusion setup but I'm not willing to take the hit in battery life while running an SSD and HDD (in the optical bay).philipma1957 - Sunday, January 20, 2013 - link
I have a 2012 quad 2.3 mini using a 512gb ssd and the oem 1tb hdd. It works very well. I used to run this mini with 2x 512gb ssds in raid0. They were in a pegasus r6 case hooked up via t-bolt. I now run this with the 1.5 tb fusion and a tm plus a clone backup in the thunderbolt case. I prefer the setup this way. I have been doing this for a week I have about 480gb on the 1.5tb fusion . It will take me a while but I will put about 1tb on the fusion I will then see how much it slows.pichemanu - Saturday, January 19, 2013 - link
A chart illustrating the possible impact of a USB 3 bottleneck on your test:http://tinypic.com/r/34fir7m/6
Suddenly the fusion drive no longer looks that good ;).
mrbreaker101 - Saturday, January 19, 2013 - link
I have a 2011 Macbook Pro with a 128GB Corsair Force 3 SSD and a 500GB hard disk. I decided to run the commands to make a fusion drive and so far, it's working, although the Force 3 isn't running that fast, despite being set to 6Gbps SATA.I'm going to install some new hardware in my iMac next. A 256GB Kingston Hyper X 3K SSD and 2TB hard disk (with thermal sensing cable from OWC, essential for iMac custom hard disk installs).
I'm not entirely sure I want to make a Fusion drive on the iMac as well. My current usage is about 500GB, with most of that being music files. On one hand, I have no need for mp3s using up my precious SSD space, even if I listen to them 1000 times a day. On the other, the simplicity of no longer having to worry about the locations of files is very appealing.
I also wonder, if I do a fresh install of Mountain Lion and then re-import my photos, media, etc, will Fusion Drive work better than by migrating over a Time Machine backup, which may fill the SSD with music, but leave apps on the hard disk?
ThomasA - Sunday, January 20, 2013 - link
I have no experience with the 'fusion drive' but wonder about a clone/backup method, given the circumstances.I do have a MBPro with a 256GB SSD paired with a 'spinning' HD (replacing the DVD).
I followed instructions offered by a Matt Gemmell, re: moving the 'home folder' to the HD. This allowed a backup clone of the SSD & a separate copy of the HD files.
http://mattgemmell.com/2011/06/21/using-os-x-with-...
twotwotwo - Sunday, January 20, 2013 - link
Awesome that Anand can't resist throwing in some sort of enterprise-y benchmarks in a review of a consumer thing. :)I'm sort of curious how the enterprise caching solutions work on the real-world database-y workloads I've seen AnandTech do measurements of before. That's super workload- and software-dependent (how big is your working set? how fat is your long tail of requests outside it?) so AT's results wouldn't directly apply to the world at large--still, one data point is a big step ahead of zero. :)
(It also seems to be increasingly a thing to design enterprise stuff with data 'intelligently' routed by to SSD or HDD by application-aware software [Evernote has done this, say, and Amazon offers SSDs and HDDs in separate instance types], rather than routing with magic at the block device layer. Still, would be awesome to know anything at all about how effective the automagical caching is in databases.)
And, unrelatedly, I'm sure Apple will get rid of the Pros with HDDs when they can, and I hope most other manufacturers do, too. You don't need TBs of internal storage now (esp. if videos are streamed, cloud storage an cheap externals exist, etc.) and Flash is good for speed, size and weight, durability (no head crashes!), etc.
carraj - Saturday, January 26, 2013 - link
Machine: 2010 15" MacBook Pro, 8 GB RAM. Disk controller is SATA II, so pointless using SATA III devices.Installed 750 GB WD Scorpio Black HDD in optical drive bay using Data Doubler bracket (/dev/disk1).
Installed 240 GB OCZ Vertex Plus R2 SSD in HDD bay (/dev/disk0).
Both were formatted as JHFS+ using Disk Utility. A Recovery HD partition was created on the HDD using Carbon Copy Cloner as /dev/diak1s3.
Fused /dev/disk0s2 and /dev/disk1s2 together giving ~1 TB Fusion Drive.
Performance is incredible, though not quite as fast as my mid-2012 13" MacBook Air with 256 GB SATA III SSD in tests. Here are the results from Xbench 1.3 (the Fusion Drive clocks in at about 2/3 of the performance of the pure SSD, despite having a 3 Gb/s SATA II interface rather than a 6 Gb/s SATA III interface):
15" MBP with 240/750 GB SATA II Fusion Drive:
Disk Test 277.85
Sequential 180.25
Uncached Write 412.52 253.28 MB/sec [4K blocks]
Uncached Write 278.11 157.36 MB/sec [256K blocks]
Uncached Read 89.26 26.12 MB/sec [4K blocks]
Uncached Read 201.26 101.15 MB/sec [256K blocks]
Random 606.00
Uncached Write 555.66 58.82 MB/sec [4K blocks]
Uncached Write 531.52 170.16 MB/sec [256K blocks]
Uncached Read 1566.27 11.10 MB/sec [4K blocks]
Uncached Read 438.38 81.34 MB/sec [256K blocks]
13" MBA with 256 GB SATA III SSD:
Disk Test 431.86
Sequential 278.39
Uncached Write 720.45 442.35 MB/sec [4K blocks]
Uncached Write 479.45 271.27 MB/sec [256K blocks]
Uncached Read 106.55 31.18 MB/sec [4K blocks]
Uncached Read 662.54 332.99 MB/sec [256K blocks]
Random 962.48
Uncached Write 751.67 79.57 MB/sec [4K blocks]
Uncached Write 748.11 239.50 MB/sec [256K blocks]
Uncached Read 2147.57 15.22 MB/sec [4K blocks]
Uncached Read 977.31 181.35 MB/sec [256K blocks]
kamaaina - Tuesday, February 12, 2013 - link
I am very happy after 6 weeks with my DIY Fusion Drive in my MBP. Replaced the optical drive as well and moved the HDD 500GB Momentus XT into the DVD HDD tray, and put an Intel330 240GB SSD in. Fused together, it runs really well. I followed the video on Youtube from Tomas Villegas, took me a only a few minutes. Awesome improvement!qzyxya - Saturday, February 16, 2013 - link
Is there a way to get this for windows? I'd love to have this. Like buy a 128gb ssd and use it for caching with my 3tb samsung 7200.14 hard drivenavss - Friday, April 26, 2013 - link
Great read, personally I like the thought of getting the Mini with the 256GB SSD + adding another 2-3TB HDD for relatively cheap and using that terminal command to set up the Fusion Drive.Also Anand, if you're going to be taking screenshots of programs on the Mac: Cmd+Shift+4+(hover over the program you want a screenshot of) then hit the spacebar, you'll get much nicer screenshots.