Sequential Read Performance

Sequential operations still make up a substantial portion of enterprise storage workloads. One of the most IO heavy workloads we run in AnandTech's own infrastructure are our traffic and ad stats processing routines. These tasks run daily as well as weekly and both create tremendous IO load for our database servers. Profiling the workload reveals an access pattern that's largely sequential in nature.

We'll start with a look at sequential read performance. Here we fill the drive multiple times with sequential data and then read it back for a period of 3 minutes. I'm reporting performance in MB/s as well as latency over a range of queue depths. First up, bandwidth figures:

The biggest takeaway from this graph is just how much parallelism Intel manages to extract from each transfer even at a queue depth of 1. The P3700 delivers more than 1GB/s of bandwidth at QD1. That's more than double any of the other competitors here, and equal to the performance of 3.7x SATA Intel SSD DC S3700s. Note that if you force the P3700 into a higher power, 25W operating mode, Intel claims peak performance hits 2.8GB/s compared to the 2.2GB/s we show here.

With NVMe you get a direct path to the PCIe controller, and in the case of any well designed system the storage will communicate directly to a PCIe controller on the CPU's die. With a much lower overhead interface and protocol stack, the result should be substantially lower latency. The graph below looks at average

The P3700 also holds a nice latency advantage here. You'll be able to see just how low the absolute latencies are in a moment, but for now we can look at the behavior of the drives vs. queue depth. The P3700's latencies stay mostly flat up to a queue depth of 16, it's only after QD32 that we see further increased latencies. The comparison to the SATA based S3700 is hilarious. The P3700's IO latency at QD32 is lower than the S3700 at QD8.

The next graph removes the sole SATA option and looks at PCIe comparisons alone, including the native PCIe (non-NVMe) Micron P420m:

Micron definitely holds the latency advantage over Intel's design at higher queue depths. Remember that the P420m also happens to be a native PCIe SSD controller, it's just using a proprietary host controller interface.

Sequential Write Performance

Similar to our discussion around sequential read performance, sequential write performance is still a very valuable metric in the enterprise space. Large log processing can stress a drive's sequential write performance, and once again it's something we see in our own server environment.

Here we fill the drive multiple times with sequential data and then write it back for a period of 3 minutes. I'm reporting performance in MB/s as well as latency over a range of queue depths.

Once again we see tremendous performance at very low queue depths. At a queue depth of 1 the P3700 already performs better than any of the other drives here, and delivers 1.3GB/s of sequential write performance. That's just insane performance at such a low queue depth. By QD4, the P3700 reaches peak performance at roughly 1.9GB/s regardless of what power mode you operate it in.

The chart below shows average latency across the QD sweep:

The P3700 continues to do extremely well in the latency tests, although Intel's original PCIe SSD didn't do so badly here either - its bandwidth was simply nowhere as good. Another way to look at it is that Intel now delivers better latency than the original 910, at substantially higher bandwidths. Micron's P420m manages to land somewhere between a good SATA drive and the P3700.

The next chart just removes the SATA drive so we get a better look at the PCIe comparison:

 

Introduction & Consistency Random Read/Write Performance & Latency Analysis
POST A COMMENT

85 Comments

View All Comments

  • extide - Tuesday, June 3, 2014 - link

    Yeah, this except more correctly it is SATA vs PCIe as the interface and AHCI vs NVMe as the protocol.

    Connectors:
    M.2 --> Supports AHCI over SATA, AHCI over PCIe, and NVMe over PCIe
    SFF-8639 --> Supports AHCI over PCIe and NVMe over PCIe
    PCIe card --> AHCI over PCIe, and NVMe over PCIe

    Now the latter 2 (and even the first one if you really wanted to...) could have a PCIe based SATA controller on it which would go PCIe --> SATA/SATA RAID Controller -> SATA SSD Controller(s), (For example this is how the OCZ Revo Drive works)
    Reply
  • Galatian - Wednesday, June 4, 2014 - link

    That's not what I meant with my comment. I'm upset that besides ASRock on the Extreme 6 and 9 and ASUS on their Impact no other manufacture included a higher bandwidth M.2 connector. I guess all upcoming PCIe M.2 drives will already be bottlenecked by the lackluster M.2 speeds most mainboard manufactures are building into their products, Reply
  • hpvd - Tuesday, June 3, 2014 - link

    hmmm are you sure? no new mainboard needed? No new Bios? Should it work in all boards which could boot existing PCIe SSDs? Reply
  • hpvd - Tuesday, June 3, 2014 - link

    I would really appreciate a short test of this. How should this work when AHCI is the standard on todays Mainboards/Bios/UEFI? There is alreday some work done until the Windows-/Linux driver take over the helm
    (which is of course already available: http://www.nvmexpress.org/blog/open-fabrics-allian...
    Reply
  • TelstarTOS - Tuesday, June 3, 2014 - link

    404 Reply
  • j00d - Friday, June 6, 2014 - link

    just take off the trailing ) in the url Reply
  • Ryan Smith - Tuesday, June 3, 2014 - link

    Since a couple of you asked, I threw it in our X79 testbed.

    Windows 8.1U1 sees the drive without issue, however it is not bootable as our motherboard cannot see the drive as a bootable devices. I preface that with the fact that our X79 testbed is a consumer platform (ASRock X79 Professional) and X79 is a rather old chipset. So I can't speak for how this would behave on a brand spanking new Z97 board, or a server board for that matter.
    Reply
  • hpvd - Wednesday, June 4, 2014 - link

    many thanks for giving this a try! Should be further investigated... Reply
  • hpvd - Wednesday, June 4, 2014 - link

    PCIe booting may be a general prob with standard bios setting on these boards. I found a tiny bios setting guide how to fix this (on a similar Asrock X97 board). Would be awesome If you could try this:
    http://www.oczforum.com/forum/showthread.php?10114...
    You would be the very first in web booting from an NVMe device :-)
    Reply
  • hpvd - Wednesday, June 4, 2014 - link

    the other way around the question is:
    does
    - your board
    - with this bios version
    - with this bios settings
    - in this PCIe slot
    see other bootable PCIe SSD devices?

    if so this new Intel PCIe NVMe SSD behave somehow different
    If others couldn't be seen either - there is still hope for "normal" boot support :-)
    You just have the right board, bios settings...
    Reply

Log in

Don't have an account? Sign up now