Exploring the NVMe Host Memory Buffer Feature

Most modern SSDs include onboard DRAM, typically in a ratio of 1GB RAM per 1TB of NAND flash memory. This RAM is usually dedicated to tracking where each logical block address is physically stored on the NAND flash—information that changes with every write operation due to the wear leveling that flash memory requires. This information must also be consulted in order to complete any read operation. The standard DRAM to NAND ratio provides enough RAM for the SSD controller to use a simple and fast lookup table instead of more complicated data structures. This greatly reduces the work the SSD controller needs to do to handle IO operations, and is key to offering consistent performance.

SSDs that omit this DRAM can be cheaper and smaller, but because they can only store their mapping tables in the flash memory instead of much faster DRAM, there's a substantial performance penalty. In the worst case, read latency is doubled as potentially every read request from the host first requires a NAND flash read to look up the logical to physical address mapping, then a second read to actually fetch the requested data.

The NVMe version 1.2 specification introduced an in-between option for SSDs. The Host Memory Buffer (HMB) feature takes advantage of the DMA capabilities of PCI Express to allow SSDs to use some of the DRAM attached to the CPU, instead of requiring the SSD to bring its own DRAM. Accessing host memory over PCIe is slower than accessing onboard DRAM, but still much faster than reading from flash. The HMB is not intended to be a full-sized replacement for the onboard DRAM that mainstream SSDs use. Instead, all SSDs using the HMB feature so far have targeted buffer sizes in the tens of megabytes. This is sufficient for the drive to cache mapping information for tens of gigabytes of flash, which is adequate for many consumer workloads. (Our ATSB Light test only touches 26GB of the drive, and only 8GB of the drive is accessed more than once.)

Caching is of course one of the most famously difficult problems in computer science and none of the SSD controller vendors are eager to share exactly how their HMB-enabled controllers and firmware use the host DRAM they are given, but it's safe to assume the caching strategies focus on retaining the most recently and heavily used mapping information. Areas of the drive that are accessed repeatedly will have read latencies similar to that of mainstream drives, while data that hasn't been touched in a while will be accessed with performance resembling that of traditional DRAMless SSDs.

SSD controllers do have some memory built in to the controller itself, but usually not enough to allow a significant portion of NAND mapping tables to be cached. For example, the Marvell 88SS1093 Eldora high-end NVMe SSD controller has numerous on-chip buffers with capacities in the kilobyte range and aggregate capacity of less than 1MB. Some SSD vendors have hinted that their controllers have significantly more on-board memory—Western Digital says this is why their SN520 NVMe SSD doesn't use HMB, but they declined to say how much memory is on that controller. We've also seen some other drives in recent years that don't fall clearly into the DRAMless category or the 1GB per TB ratio. The Toshiba OCZ VX500 uses a 256MB DRAM part for the 1TB model, but the smaller capacity drives rely just on the memory built in to the controller (and of course, Toshiba didn't disclose the details of that controller architecture).

The Toshiba RC100 requests a block of 38 MB of host DRAM from the operating system. The OS could provide more or less than the drive's preferred amount, and if the RC100 gets less than 10MB it will give up on trying to use HMB at all. Both the Linux and Windows NMVe drivers expose some settings for the HMB feature, allowing us to test the RC100 with HMB enabled and disabled. In theory, we could also test with varying amounts of host memory allocated to the SSD, but that would be a fairly time-consuming exercise and would not reflect any real-world use cases, because the driver settings in question are so obscure and not worth changing from their defaults.

Working Set Size

We can see the effects of the HMB cache quite clearly by measuring random read performance while increasing the test's working set—the amount of data that's actively being accessed. When all of the random reads are coming from the same 1GB range, the RC100 performs much better than when the random reads span the entire drive. There's a sharp drop in performance when the working set approaches 32GB. When the RC100 is tested with HMB off, performance is just as good for a 1GB working set (and actually substantially better on the 480GB model), but larger working sets are almost as slow as the full-span random reads. It looks like the RC100's controller may have about 1MB of built-in memory that is much faster than accessing host DRAM over the PCIe link.

Most mainstream SSDs offer nearly the same random read performance regardless of the working set size, though performance through this test varies some due to other factors (eg. thermal throttling). The drives using the Phison E7 and E8 NVMe controllers are a notable exception, with significant performance falloff as the working set grows, despite these drives being equipped with ample onboard DRAM.

Introduction AnandTech Storage Bench - The Destroyer
POST A COMMENT

62 Comments

View All Comments

  • MadAd - Saturday, June 23, 2018 - link

    +1

    Anyone can review a flagship release, this is real tech reporting.
    Reply
  • Calin - Thursday, June 14, 2018 - link

    Yes, more people care about the low-mid end of the market than about the very high end - the M.2 2242 has multiple advantages (compact, doesn't need a 2.5 or 3.5 inch case location, doesn't need power cable, low power. Plenty of people will pay more for that, if the performance is equivalent (just as plenty of people pay more for pretty lights when the performance is equivalent, or pay more for a quiter component, or ...).
    So, while I'm not interested in M2 drives, thank you for the article :)
    Reply
  • gnufied - Thursday, June 14, 2018 - link

    Actually I have been looking for a decent 2242 SSD since forever. My thinkpad T450s has a 2242 slot and we have very little options.

    Having said all this, I dunno if this is comptaible with thinkpad's 2242 slot.
    Reply
  • timecop1818 - Thursday, June 14, 2018 - link

    most likely not, as that slot is probably for USB-based LTE modem or similar device. M.2 spec does provide for USB connectivity on A, B and E keyed cards/sockets. Reply
  • gnufied - Thursday, June 14, 2018 - link

    The 2242 slot defenitely can be used for M.2 SSDs. I am currently using https://www.amazon.com/gp/product/B012ASBZEO/ref=o... SSD with my thinkpad and it works.

    Reading through R100 review does not fill me with joy though. This NVMe disk does not even look better than M500.. :(
    Reply
  • Targon - Thursday, June 14, 2018 - link

    M.2 has different flavors, and depending on the slot and device you plug into it, may use a SATA connection rather than NVMe with the dedicated PCI Express lanes. The 2242 is a measure of width and length, NOT the interface. An obvious way to tell is that if you have two notches on the connector, you are not looking at a NVMe connection, while the single notch will support the PCI Express lanes. At least, that is typically what is going on. With only two PCI Express lanes, without investigating, I suppose the two notches might still give you some NVMe, but I wouldn't be sure about that. Reply
  • Kwarkon - Friday, June 15, 2018 - link

    There are NVME drives with two notches (M+B), but this keying is limited to only 2 PCIe lines. Reply
  • Jorgp2 - Thursday, June 14, 2018 - link

    I already ordered one, I'll try to send you a PM if it works Reply
  • gnufied - Thursday, June 14, 2018 - link

    Nice. thank you. also do a crystal mark or something while you are at it. :-) Reply
  • close - Thursday, June 14, 2018 - link

    Because it's less than half the price so it has a better chance of ending up in a "regular machine" than the 970 Pro. Reviews for halo products are great but what do you think most people will have a $350 SSD or a $150 one? Reply

Log in

Don't have an account? Sign up now