Exploring the NVMe Host Memory Buffer Feature

Most modern SSDs include onboard DRAM, typically in a ratio of 1GB RAM per 1TB of NAND flash memory. This RAM is usually dedicated to tracking where each logical block address is physically stored on the NAND flash—information that changes with every write operation due to the wear leveling that flash memory requires. This information must also be consulted in order to complete any read operation. The standard DRAM to NAND ratio provides enough RAM for the SSD controller to use a simple and fast lookup table instead of more complicated data structures. This greatly reduces the work the SSD controller needs to do to handle IO operations, and is key to offering consistent performance.

SSDs that omit this DRAM can be cheaper and smaller, but because they can only store their mapping tables in the flash memory instead of much faster DRAM, there's a substantial performance penalty. In the worst case, read latency is doubled as potentially every read request from the host first requires a NAND flash read to look up the logical to physical address mapping, then a second read to actually fetch the requested data.

The NVMe version 1.2 specification introduced an in-between option for SSDs. The Host Memory Buffer (HMB) feature takes advantage of the DMA capabilities of PCI Express to allow SSDs to use some of the DRAM attached to the CPU, instead of requiring the SSD to bring its own DRAM. Accessing host memory over PCIe is slower than accessing onboard DRAM, but still much faster than reading from flash. The HMB is not intended to be a full-sized replacement for the onboard DRAM that mainstream SSDs use. Instead, all SSDs using the HMB feature so far have targeted buffer sizes in the tens of megabytes. This is sufficient for the drive to cache mapping information for tens of gigabytes of flash, which is adequate for many consumer workloads. (Our ATSB Light test only touches 26GB of the drive, and only 8GB of the drive is accessed more than once.)

Caching is of course one of the most famously difficult problems in computer science and none of the SSD controller vendors are eager to share exactly how their HMB-enabled controllers and firmware use the host DRAM they are given, but it's safe to assume the caching strategies focus on retaining the most recently and heavily used mapping information. Areas of the drive that are accessed repeatedly will have read latencies similar to that of mainstream drives, while data that hasn't been touched in a while will be accessed with performance resembling that of traditional DRAMless SSDs.

SSD controllers do have some memory built in to the controller itself, but usually not enough to allow a significant portion of NAND mapping tables to be cached. For example, the Marvell 88SS1093 Eldora high-end NVMe SSD controller has numerous on-chip buffers with capacities in the kilobyte range and aggregate capacity of less than 1MB. Some SSD vendors have hinted that their controllers have significantly more on-board memory—Western Digital says this is why their SN520 NVMe SSD doesn't use HMB, but they declined to say how much memory is on that controller. We've also seen some other drives in recent years that don't fall clearly into the DRAMless category or the 1GB per TB ratio. The Toshiba OCZ VX500 uses a 256MB DRAM part for the 1TB model, but the smaller capacity drives rely just on the memory built in to the controller (and of course, Toshiba didn't disclose the details of that controller architecture).

The Toshiba RC100 requests a block of 38 MB of host DRAM from the operating system. The OS could provide more or less than the drive's preferred amount, and if the RC100 gets less than 10MB it will give up on trying to use HMB at all. Both the Linux and Windows NMVe drivers expose some settings for the HMB feature, allowing us to test the RC100 with HMB enabled and disabled. In theory, we could also test with varying amounts of host memory allocated to the SSD, but that would be a fairly time-consuming exercise and would not reflect any real-world use cases, because the driver settings in question are so obscure and not worth changing from their defaults.

Working Set Size

We can see the effects of the HMB cache quite clearly by measuring random read performance while increasing the test's working set—the amount of data that's actively being accessed. When all of the random reads are coming from the same 1GB range, the RC100 performs much better than when the random reads span the entire drive. There's a sharp drop in performance when the working set approaches 32GB. When the RC100 is tested with HMB off, performance is just as good for a 1GB working set (and actually substantially better on the 480GB model), but larger working sets are almost as slow as the full-span random reads. It looks like the RC100's controller may have about 1MB of built-in memory that is much faster than accessing host DRAM over the PCIe link.

Most mainstream SSDs offer nearly the same random read performance regardless of the working set size, though performance through this test varies some due to other factors (eg. thermal throttling). The drives using the Phison E7 and E8 NVMe controllers are a notable exception, with significant performance falloff as the working set grows, despite these drives being equipped with ample onboard DRAM.

Introduction AnandTech Storage Bench - The Destroyer
POST A COMMENT

62 Comments

View All Comments

  • bug77 - Thursday, June 14, 2018 - link

    I'm talking about what is, you're talking wishful thinking.
    PCIe is supposed to cater to a lot of devices, it can't change its sleep current just because of one type of devices in particular. Not saying it's impossible, just that it's highly unlikely.
    Reply
  • PeachNCream - Monday, June 18, 2018 - link

    Since SATA has not been entirely replaced by NVMe yet, there is still time (and lots of it really) for changes. It's simply a matter of a drive identifying itself to the PCIe bus and then making on-the-fly sleep state changes. Yes, that's non-trivial, but far from wishful thinking. Reply
  • Gasaraki88 - Thursday, June 14, 2018 - link

    SATA needs to go away. That is old technology for old drives. NVMe should be the new standard for hard drives, just like SAS was a better protocol than SATA, NVMe has less overhead and is designed for NAND storage. Reply
  • Targon - Thursday, June 14, 2018 - link

    Space, and because people like these super-thin machines. Also, without the extra packaging, it may be less expensive to make a card based SSD compared to a 2.5 inch SSD drive. Smaller=cheaper when it comes to shipping/packaging as well.

    SATA hasn't really had any evolution over the past few years as well, so without something big to hype, SATA isn't a buzz word that attracts buyers. No SATA 4 standard, so they can't say it is the latest and greatest, while card based SSDs have an appeal as seeming to be a newer technology.
    Reply
  • HStewart - Thursday, June 14, 2018 - link

    One thing I am curious about is what performance do you need for SSD in external USB drive - I have a couple of them. These cheaper drivers are probably good for that purpose Reply
  • timecop1818 - Thursday, June 14, 2018 - link

    Except cheap USB to M.2 adapters ONLY support SATA drives. The review unit is NVMe. Reply
  • Targon - Thursday, June 14, 2018 - link

    USB 3.1 at the minimum if you want an external SSD in my opinion. Reply
  • HStewart - Thursday, June 14, 2018 - link

    The one I am using ( actually two of them ) is WavLink USB 3.1 Gen 2 that actually does 10gbs '

    https://www.amazon.com/gp/product/B06Y5XJG8J/ref=o...

    It is not intended be primary storage - but works quite nice for my needs.

    One thing some one should come out with lower cost TB3 drive case - right now they are at premium.
    Reply
  • peevee - Thursday, June 14, 2018 - link

    I wonder who would possibly buy the 120GB version given that only extra #20 will bring it to useful capacity and performance? Reply
  • Jorgp2 - Thursday, June 14, 2018 - link

    Could you elaborate on how to configure the Host Memory Buffer Size? Reply

Log in

Don't have an account? Sign up now