SSD caching has been around for a long time, as a way to reap many of the performance benefits of fast storage without completely abandoning the high capacity and lower prices of slower storage options. In recent years, the fast, small, expensive niche has been ruled by Intel's Optane products using their 3D XPoint non-volatile memory. Intel's third generation of Optane Memory SSD caching products has arrived, bringing the promise of Optane performance to a new product segment. The first Optane Memory products were tiny NVMe SSDs intended to accelerate access to larger slower SATA drives, especially mechanical hard drives. Intel is now supporting using Optane Memory SSDs to cache other NVMe SSDs, with an eye toward the combination of Optane and QLC NAND flash. They've put both types of SSD onto a single M.2 module to create the new Optane Memory H10.

The Intel Optane Memory H10 allows Intel for the first time to put their Optane Memory caching solution into ultrabooks that only have room for one SSD, and have left SATA behind entirely. Squeezing two drives onto a single-sided 80mm long M.2 module is made possible in part by the high density of Intel's four bit per cell 3D QLC NAND flash memory. Intel's 660p QLC SSD has plenty of unused space on the 1TB and 512GB versions, and an Optane cache has great potential to offset the performance and endurance shortcomings of QLC NAND. Putting the two onto one module has some tradeoffs, but for the most part the design of the H10 is very straightforward.

The Optane Memory H10 does not introduce any new ASICs or any hardware to make the Optane and QLC portions of the drive appear as a single device. The caching is managed entirely in software, and the host system accesses the Optane and QLC sides of the H10 independently. Each half of the drive has two PCIe lanes dedicated to it. Earlier Optane Memory SSDs have all been PCIe x2 devices so they aren't losing anything, but the Intel 660p uses a 4-lane Silicon Motion NVMe controller, which is now restricted to just two lanes. In practice, the 660p almost never needed more bandwidth than an x2 link can provide, so this isn't a significant bottleneck.

Intel Optane Memory H10 Specifications
Advertised Capacity 256 GB 512 GB 1TB
Form Factor single-sided M.2 2280
NAND Controller Silicon Motion SM2263
NAND Flash Intel 64L 3D QLC
Optane Controller Intel SLL3D
Optane Media Intel 128Gb 3D XPoint
QLC NAND Capacity 256 GB 512 GB 1024 GB
Optane Capacity 16 GB 32 GB 32 GB
Sequential Read 1450 MB/s 2300 MB/s 2400 MB/s
Sequential Write 650 MB/s 1300 MB/s 1800 MB/s
Random Read IOPS 230k 320k 330k
Random Write IOPS 150k 250k 250k
L1.2 Idle Power < 15 mW
Warranty 5 years
Write Endurance 75 TB
0.16 DWPD
150 TB
0.16 DWPD
300 TB
0.16 DWPD

With a slow QLC SSD and a fast Optane SSD on one device, Intel had to make judgement calls in determining the rated performance specifications. The larger two capacities of H10 are rated for sequential read speeds in excess of 2GB/s, reflecting how Intel's Optane Memory caching software can fetch data from both QLC and Optane portions of the H10 simultaneously. Writes can also be striped, but the maximum rating doesn't exceed any obvious limit for single-device performance. The random IO specs for the H10 fall between the performance of the existing Optane Memory and 660p SSDs, but are much closer to Optane performance. Intel's not trying to advertise a perfect cache hit rate, but they expect it to be pretty good for ordinary real-world usage.

The Optane cache should help reduce the write burden that the QLC portion of the H10 has to bear, but Intel still rates the whole device for the same 0.16 drive writes per day that their 660p QLC SSDs are rated for.

Intel's marketing photos of the Optane Memory H10 show it with a two-tone PCB to emphasize the dual nature of the drive, but in reality it's a solid color. The PCB layout is unique with two controllers and three kinds of memory, but it is also obviously reminiscent of the two discrete products it is based on. The QLC NAND half of the drive is closer to the M.2 connector and features the SM2263 controller and one package each of DRAM and NAND. The familiar Silicon Motion test/debug connections are placed at the boundary between the NAND half and the Optane half. That Optane half contains Intel's small Optane controller, a single package of 3D XPoint memory, and most of the power management components. Both the Intel SSD 660p and the earlier Optane Memory SSDs had very sparse PCBs; the Optane Memory H10 is crowded and may have the highest part count of any M.2 SSD on the market.

On the surface, little has changed with the Optane Memory software; there's just more flexibility now in which devices can be selected to be cached. (Intel has also opened extended Optane Memory support to Pentium and Celeron branded processors on platforms that were already supported with Core processors.) When the boot volume is cached, Intel's software allows the user to specify files and applications that should be pinned to the cache and be immune from eviction. Other than this, there's no room for tweaking of the cache behavior.

Some OEMs that sell systems equipped with Optane Memory have been advertising memory capacities as the sum of DRAM and Optane capacities, which might be reasonable if we were talking about Optane DC Persistent Memory modules that connect to the CPU's memory controller, but is very misleading when the Optane product in question is an SSD. Intel says to blame the OEMs for this misleading branding, but Intel's own Optane Memory software does the same thing.

Initially, the Optane Memory H10 will be an OEM-only part, available to consumers only pre-installed in new systems—primarily notebooks. Intel is considering bringing the H10 to retail both as a standalone product and as part of a NUC kit, but they have not committed to plans for either. Their motherboard partners have been laying the groundwork for H10 support for almost a year, and many desktop 300-series motherboards already support the H10 with the latest publicly available firmware.

Platform Compatibility

Putting two PCIe devices on one M.2 card is novel to say the least. Intel has put two SSD controllers on one PCB before with high-end enterprise drives like the P3608 and P4608, but those drives use PCIe switch chips to split an x8 host connection into x4 for each of the two NVMe controllers on board. That approach leads to a 40W TDP for the entire card, which is not at all useful when trying to work within the constraints of a M.2 card.

There are also several PCIe add-in cards that allow four M.2 PCIe SSDs to be connected through one PCIe x16 slot. A few of these cards also include PCIe switches, but most rely on the host system supporting PCIe port bifurcation to split a single x16 port into four independent x4 ports. Mainstream consumer CPUs usually don't support this, and are generally limited to x8+x4+x4 or just x8+x8 bifurcation, and only when the lanes are being re-routed to different slots to support multi-GPU use cases. Recent server and workstation CPUs are more likely to support bifurcation down to x4 ports, but motherboard support for enabling this functionality isn't universal.

Even on CPUs where an x16 slot can be split into four x4 ports, further bifurcation down to x2 ports is seldom or never possible. The chips that do support operating a lot of PCIe lanes as narrow x2 or x1 ports are the southbridge/PCH chips on most motherboards. These tend to not support ports any wider than x4, because that's the normal width of the connection upstream to the CPU.

Based on the above, we put theory to the test and tried the Optane Memory H10 with almost every PCIe 3.0 port we had on hand, using whatever adapters were necessary. Our results are summarized below:

Intel Optane Memory H10
Platform Compatibility
Platform PCIe
Source
NAND
Usable
Optane
Usable
Optane Memory
Caching
Whiskey Lake PCH Yes Yes Yes
Coffee Lake CPU Yes No No
PCH Yes Yes No*
Kaby Lake CPU Yes No No
PCH Yes No No
Skylake CPU Yes No No
PCH Yes No No
Skylake-SP
(Purley)
CPU Yes No No
PCH Yes No No
Threadripper CPU No Yes No
Avago PLX Switch Yes No No
Microsemi PFX Switch No Yes No

The Whiskey Lake notebook Intel provided for this review is of course fully compatible with the Optane Memory H10, and will be available for purchase in this configuration soon. Compatibility with older platforms and non-Intel platforms is mostly as expected, with only the NAND side of the H10 accessible—those motherboards don't expect to find two PCIe devices sharing a physical M.2 x4 slot, and aren't configured to detect and initialize both devices. There are a few notable exceptions:

First, the H370 motherboard in our Coffee Lake system is supposed to fully support the H10, but GIGABYTE botched the firmware update that claims to have added H10 support: both the NAND and Optane portions of the H10 are accessible when using a M.2 slot that connects to the PCH, but it isn't possible to enable caching. There are plenty of 300-series motherboards that have successfully added H10 support, and I'm sure GIGABYTE will release a fixed firmware update for this particular board soon. Putting the H10 into a PCIe x16 slot that connects directly to the CPU does not provide access to the Optane side, reflecting the CPU's lack of support for PCIe port bifurcation down to x2+x2.

The only modern AMD system we had on hand was a Threadripper/X399 motherboard. All of the PCIe and M.2 slots we tried led to the Optane side of the H10 being visible instead of the NAND side.

We also connected the H10 through two different brands of PCIe 3.0 switch. Avago's PLX PEX8747 switch only provided access to the NAND side, which is to be expected since it only supports PCIe port bifurcation down to x4 ports. The Microsemi PFX PM8533 switch does claim to support bifurcation down to x2 and we were hoping it would enable access to both sides of the H10, but instead we only got access to the Optane half. The Microsemi switch and Threadripper motherboard may both be just a firmware update away from working with both halves of the H10, and earlier Intel PCH generations might also have that potential, but Intel won't be providing any such updates. Even if these platforms were able to access both halves of the H10, they would not be supported by Intel's Optane Memory caching drivers, but third-party caching software exists.

Test Setup
POST A COMMENT

67 Comments

View All Comments

  • Alexvrb - Monday, April 22, 2019 - link

    "The caching is managed entirely in software, and the host system accesses the Optane and QLC sides of the H10 independently. "

    So, it's already got serious baggage. But wait, there's more!

    "In practice, the 660p almost never needed more bandwidth than an x2 link can provide, so this isn't a significant bottleneck."

    Yeah OK, what about the Optane side of things?
    Reply
  • Samus - Tuesday, April 23, 2019 - link

    They totally nerf'd this thing with 2x PCIe. Reply
  • PeachNCream - Tuesday, April 23, 2019 - link

    Linux handles Optane pretty easily without any Intel software through bcache. I'm not sure why Anandtech can't test that, but maybe just a lack of awareness.

    https://www.phoronix.com/scan.php?page=article&...
    Reply
  • Billy Tallis - Tuesday, April 23, 2019 - link

    Testing bcache performance won't tell us anything about how Intel's caching software behaves, only how bcache behaves. I'm not particularly interested in doing a review that would have such a narrow audience. And bcache is pretty thoroughly documented so it's easier to predict how it will handle different workloads without actually testing. Reply
  • PeachNCream - Wednesday, April 24, 2019 - link

    Hmm interesting - message received. Reply
  • PeachNCream - Friday, May 03, 2019 - link

    I think the basis of this article is to analyze how the hardware behaves rather than specifically focusing on Intel's caching software. After all, Anandtech touts "hardware reviews since 1997" or somesuch thing and you specifically cited a problem with Linux testing in the article's message body. bcache will behave differently and offer differing performance based on the underlying hardware and thus, may provide useful insight into said performance. Reply
  • easy_rider - Wednesday, April 24, 2019 - link

    Is there a reliable review of 118gb intel optane ssd in M2 form factor? Does it make sense to hunt it down and put as a system drive in the dual-m2 laptop? Reply
  • name99 - Thursday, April 25, 2019 - link

    "QLC NAND needs a performance boost to be competitive against mainstream TLC-based SSDs"

    The real question is what dimension, if any, does this thing win on?
    OK, it may not be the fastest out there? But does it, say, provide approximately leading edge TLC speed at QLC prices, so it wins by being cheap?
    Because just having a cache is meaningless. Any QLC drive that isn't complete garbage will have a controller-managed cache created by using the QLC flash as SLC; and the better controllers will slowly degrade across the entire drive, maintaining always an SLC cache, but also using the entire drive (till its filled up) as SLC, then switching blocks to MLC, then to TLC, and only when the drive is approaching capacity, using blocks as QLC.

    So the question is not "does it give cached performance to a QLC drive", the question is does it give better performance or better price than other QLC solutions?
    Reply
  • albert89 - Saturday, April 27, 2019 - link

    Didn't I tell ya ? Optane's capacity was too small for many yrs and compatible with a very tiny number devices/hardware/OS. She played the game of hard to get and now no guy wants her. Reply
  • peevee - Monday, April 29, 2019 - link

    "The caching is managed entirely in software, and the host system accesses the Optane and QLC sides of the H10 independently. Each half of the drive has two PCIe lanes dedicated to it."

    Fail.
    Reply

Log in

Don't have an account? Sign up now