A week later than originally planned, AMD is launching NVMe RAID support for their Ryzen Threadripper platform. Grab the Windows 10 drivers and release notes here. This updated RAID capability should work with any X399/Threadripper motherboard and any NVMe SSDs, enabling RAID modes 0, 1 and 10 for up to 10 NVMe SSD connected to the CPU's PCIe lanes, but a motherboard firmware update may be required first.

AMD's Ryzen Threadripper launched with support for RAID modes 0, 1 and 10 for arrays of SATA drives connected through the X399 chipset's eight SATA ports. The X399 chipset doesn't offer RAID capability for NVMe SSDs, but given its limited PCIe lane counts, this isn't much of an inconvenience. The Ryzen Threadripper processors themselves have plenty of PCIe lanes to go around. In a typical configuration, the motherboard will have up to three M.2 slots providing PCIe x4 connections to the Threadripper CPU (leaving 48 PCIe lanes for GPUs and four lanes for the connection to the X399 chipset). These PCIe x4 ports can also operate as SATA ports to support M.2 SATA SSDs. However, since these ports are provided by the CPU and not the chipset, they're not covered by the RAID capabilities of the chipset.

To allow for RAID arrays of NVMe SSDs, the Ryzen Threadripper platform needed another RAID implementation. Nobody has manufactured a true NVMe hardware RAID controller—one that completely abstracts the individual drives in the array and presents a single NVMe device to the host. Like all previous NVMe RAID implementations and almost all motherboard RAID implementations for SATA drives, AMD's NVMe RAID for Ryzen Threadripper is at heart a software RAID system. That means it needs new drivers for the OS, and matching support in the motherboard's UEFI to enable booting from a RAID array.

Chipset RAID implementations for SATA typically rely on switching the SATA controller out of AHCI mode and into a proprietary RAID mode. This ensures that the operating system's standard storage drivers won't be able to access the drives, so there's no danger of the array being accidentally corrupted by an OS that isn't aware the drive is part of an array. When Intel added NVMe RAID to their consumer platform with the Sunrise Point 100-series chipsets accompanying Skylake processors, they included a similar mechanism that hides NVMe devices that are connected through the chipset and instead makes them accessible through non-standard interfaces on the SATA controller.

For Intel's X299 HEDT platform for Skylake-X, the same chipset RAID capabilities are available, but the more interesting feature is Intel's Virtual RAID on CPU (VROC) for NVMe SSDs attached to the CPU's PCIe lanes. VROC is a software RAID system that builds atop the Intel Volume Management Device (VMD) feature of Skylake-X and Skylake-SP processors. VMD allows some of the CPU's PCIe ports to be partitioned off into a separate PCI domain behind a virtual PCI bridge. VMD has largely been described by Intel as a feature intended to simplify support for hotplug and enclosure management (status LEDs) of NVMe SSDs, but it apparently also helps build a robust software RAID stack. Intel has not shared much technical information about how VROC works, but they have announced that its features will be segmented by requiring the purchase of a hardware key to unlock RAID capability, and a premium key to unlock RAID-5.

AMD has not described any features of their Zen architecture that sound particularly similar to Intel's VMD or the NVMe remapping feature of Intel's consumer platform chipsets, so it's not clear what hardware features (if any) are helping to enable their NVMe RAID implementation. From a user perspective, setting up NVMe RAID is very similar to setting up chipset SATA RAID: configure the array through the motherboard firmware and load an extra driver when installing Windows. Unlike Intel's plans for VROC, AMD isn't trying for any extra product segmentation and isn't charging extra for any of their NVMe RAID functionality.

One important warning: the new RAID system is incompatible with the previous SATA RAID system for X399. Users with an existing SATA RAID array will not be able to access it using the new RAID drivers. The only migration path AMD currently offers is to re-create the array using the new drivers and firmware, and restore the data from a back-up. For users booting off a RAID array, this means re-installing Windows.

Source: AMD

POST A COMMENT

47 Comments

View All Comments

  • Hul8 - Monday, October 02, 2017 - link

    First of all, EPYC has 128 lanes, Threadripper only has 64, of which 4 go to the chipset.

    Secondly, the sentence was about the PCIe lanes from the chipset. Those are limited.
    Reply
  • IGTrading - Tuesday, October 03, 2017 - link

    Understood.

    Thank you!
    Reply
  • msroadkill612 - Friday, October 06, 2017 - link

    " Intel's platforms where the chipsets have more lanes, and do offer NVMe RAID."

    This is sadly the perception many have. Intel can plausibly say the latter part is true, but it isn't in reality.

    Yes intel offer native nvme raid on the chipset, but no they dont provide more than the usual 4 lanes for the chipset.

    Meaning one good nvme almost saturates the entire bandwidth of the chipset, making multi drive raid acceleration pointless.

    It gets worse.

    All that processing required to share that bandwidth takes time and introduces lag.

    Intels most popular current processor, the 7700k, has no spare lanes after adding a normal 16 lane gpu. There is no way (except an 8 lane gpu) that you can add a true nvme drive at all, let alone an array.

    There is no way you will get anything like comparable speeds from an intel chipset raid0 array.

    I feel sad knowing how disappointed folks will be so soon after purchase if they are ~excluded from the still evolving benefits of nvme.

    In the known universe, name me something that has increased by factor of 35 in a short space of time? Even in the IT world, thats astonishing. Yet in the time SSDs have been affordable, thats what ~mainstream desktop workdisks have become, speedwise.

    Our ~mainstream such work drive device went from a 100MB/s sata hdd, to 3500MB/s samsung nvme. Its not a tick or a tock, or several of each, its seismic. A vacuum that is a long way from filled.

    That Amd now have provided TR users with the native means and lanes to multiply that 35x factor, by further ~6x, is truly head spinning.

    It blurs the traditional boundaries between storage and memory, which has interesting possibilities in a seemingly perpetually dram starved IT market.

    FYI, an instructive thread on what a mess intel raid is.

    http://forum.asrock.com/forum_posts.asp?TID=5587&a...
    Reply
  • msroadkill612 - Thursday, October 12, 2017 - link

    The chipsets provide the illusion of pcie3 lanes, but they all share the same four real lanes. They are a joke. Reply
  • The Benjamins - Monday, October 02, 2017 - link

    By using 4 drives. The up to 3 NVME is up to 3 M.2 NVME compatible slots, but you can use PCIe cards and use up to 4 M.2 Drives per x16 PCIe slot. Der8auers Video (was released early and is still currently taken down.) has him using 2 4xM.2 cards for 8 960 pros in a RAID 0 setup. Reply
  • Gothmoth - Monday, October 02, 2017 - link

    becaseu our infos are wrong.. it suppiots 6 drives.

    AMD: "With a single GPU in the system, arrays containing up to six NVMe SSDs can be supported without adapters. NVMe RAID support on the AMD Ryzen Threadripper platform does not require specific NVMe disks or hardware activation keys."
    Reply
  • Gothmoth - Monday, October 02, 2017 - link

    stupid comment system.. i am typing on my phone... sorry for the typos. but no editing feature with this stupid comment system. Reply
  • Ej24 - Monday, October 02, 2017 - link

    Threadripper allows up to 10 nvme drives. Not sure where you got the limit of 3 from. Three is just the typical number of onboard m.2 connections, but you can just add more via the normal pcie slots and a riser card. Reply
  • peevee - Monday, October 02, 2017 - link

    I quoted the article. "In a typical configuration, the motherboard will have up to three M.2 slots providing PCIe x4 connections to the Threadripper CPU" Reply
  • Billy Tallis - Monday, October 02, 2017 - link

    M.2 slots on the motherboard are far from being the only way to attach a NVMe SSD to the CPU's PCIe lanes. Reply

Log in

Don't have an account? Sign up now