Test System

Along with the Optane SSD sample, Intel provided a new server based on their latest Xeon Scalable platform for our use as a testbed. The 2U server is equipped with two 18-core Xeon Gold 6154 processors, 192GB of DRAM, and an abundance of PCIe lanes.

Because this is a dual-socket NUMA system, care needs to be taken to avoid unnecessary data transfers between sockets. For this review, the tests were setup to largely emulate a single-socket configuration: All SSDs tested for this review were connected to PCIe ports provided by CPU #2. All of the benchmarks using the FIO tool were configured to only use cores on CPU #2 and only allocate memory connected to CPU #2, so inter-socket latency is not a factor. This setup would be quite limiting when doing full enterprise application testing, but for synthetic storage benchmarks one CPU is far more than necessary to stress any single SSD.

For this review, the test system was mostly configured to offer the highest and most predictable storage performance. HyperThreading was disabled, SpeedStep and processor C-states were off, and other motherboard settings were set for maximum performance. The one notable exception is fan speeds: since this test server was installed in a home office environment instead of a datacenter, the "acoustic" fan profile had to be used instead of "performance".

Enterprise SSD Test System
System Model Intel Server R2208WFTZS
CPU 2x Intel Xeon Gold 6154 (18C, 3.0GHz)
Motherboard Intel S2600WFT
Chipset Intel C624
Memory 192GB total, Micron DDR4-2666 16GB modules
Software Linux kernel 4.13.11
FIO 3.1

The Linux kernel's NVMe driver is constantly evolving, and several new features have been added this year that are relevant to a drive like the Optane SSD. Rather than use an enterprise-oriented Linux distribution with a long-term support cycle for an older kernel version, this review was conducted using the very fresh 4.13 kernel series.

Earlier this year, the FIO storage benchmarking tool hit a major milestone with version 3.0 that switches timing measurements from microsecond precision to nanosecond precision. This makes it much easier to analyze the performance of Optane devices, where latency can be down in the single-digit microsecond territory.

The Competition

Intel SSD DC P3700 1.6TB

The Intel SSD DC P3700 was the flagship of their first generation of NVMe SSDs, and one of the first widely available NVMe SSDs. It was a great showcase for the advantages of NVMe, but is now outdated with its use of 20nm planar MLC NAND and capacities that top out at 2TB.

Intel launched a new P4x00 generation of flash-based enterprise NVMe SSDs this year using a new controller and 3D TLC NAND. However, due to the existence of the Optane SSD DC P4800X as the new flagship drive, the P3700 didn't get a direct successor: the P4600 is currently Intel's top flash-based enterprise SSD, and while it offers higher performance and capacities than the P3700 it does not match the rated write endurance of the P3700.

Intel SSD DC P3608 1.6TB

Intel's first-generation NVMe controller only supports drive capacities up to 2TB. To get around that limitation and to offer higher performance in some respects, Intel created the SSD DC P3608. Roughly equivalent to two P3600s on one card behind a PLX PCIe switch, the P3608 appears to the system as two SSDs but can be used with software RAID to create a large, high-performance volume. Our P3608 sample has a total of 1.6TB of accessible storage (800GB per controller) and has the highest built-in overprovisioning ratios in the P3608 family. This gives it the best aggregate random write performance, which is rated to match a single P3700 while random reads and sequential transfers can exceed the P3700.

For this review, one controller on the P3608 was tested as a stand-in for the P3600, and a software RAID 0 array spanning both controllers was also tested.

Intel has not officially announced the successor P4608, but they have submitted it for testing and inclusion on the NVMe Integrators List, so it will probably launch within the next few months.

Micron 9100 MAX 2.4TB

Launched in mid 2016, the Micron 9100 series was part of their first generation of NVMe SSDs. Micron wasn't new to the PCIe SSD market, but their early products predated the NVMe standard and instead used a proprietary protocol. The 9100 series uses Micron 16nm MLC NAND flash and a Microsemi Flashtec NVMe1032 controller.

Our Micron 9100 MAX 2.4TB sample is the fastest model from the 9100 series, and the second-largest. Micron recently announced the 9200 series that switched to 3D TLC NAND for a huge capacity boost and adopted the latest generation of Microsemi controllers to allow for a PCIe x8 connection, but we don't have a sample.

Intel Optane SSD 900p 280GB

Last month, Intel launched their consumer Optane SSD based on the same controller platform as the P4800X. The Optane SSD 900p has some enterprise-oriented features disabled and is rated for only a third the write endurance, but offers essentially the same performance as the P4800X for about a third the price.

This makes the 900p a very attractive option for users who need Optane level of performance but don't want to pay for the absolute highest write endurance. It isn't intended as an enterprise SSD, but the 900p can compete in this space.

Introduction Fine Tuning Performance
POST A COMMENT

60 Comments

View All Comments

  • "Bullwinkle J Moose" - Thursday, November 09, 2017 - link

    Humor me.....

    How fast can you copy and paste a 100GB file from and to the same Optane SSD

    I don't believe your mixed mode results adequately demonstrate the internal throughput

    At least not until you demonstrate a direct comparison
    Reply
  • Billy Tallis - Thursday, November 09, 2017 - link

    Your concept of "internal throughput" has no basis in reality. File copies (on a filesystem that does not do copy-on-write) require the file data to be read from the SSD into system DRAM, then written back to the SSD. There are no "copy" commands in the NVMe command set. Reply
  • "Bullwinkle J Moose" - Thursday, November 09, 2017 - link

    "There are no "copy" commands in the NVMe command set."
    ---------------------------------------------------------------------------------
    That might be fixed with a few more onboard processors in the future but does not answer my question

    How fast can you copy/paste 100GB on THAT specific drive?
    Reply
  • "Bullwinkle J Moose" - Thursday, November 09, 2017 - link

    Better yet, I'd like you to GUESS how fast it can copy and paste based on your mixed mode analysis and then go measure it Reply
  • Lord of the Bored - Friday, November 10, 2017 - link

    How will a new processor change that there is no way to tell the drive to do what you want? We don't trust storage devices to "do what I mean", because the cost of a mistake is too high. No device anyone should be using will say "it looks like they're writing back the data they just read in, I'mma ignore the input and duplicate it from the cache to save time." Especially since they can't know if the data is changed in advance.

    Barring a new interface standard, it will take exactly as long to copy a file to another location on the same drive as it will to read the file and then write the file, because that is the only provision within the NVMe command set.
    Reply
  • "Bullwinkle J Moose" - Friday, November 10, 2017 - link

    "How will a new processor change that there is no way to tell the drive to do what you want?"
    --------------------------------------------------------------------------------------------------------------------------
    I have no idea Mr Bored, the "problem" as outlined by Billy Tallis could be ignored COMPLETELY and a fix is not even needed if he would simply do the copy/paste test that I originally asked

    He won't of course, and continue his downward spiral into depression and resentment lashing out at anyone who dares say a bad word about the Spyware/Adware/Malware/Extortionware Platform that is Windows 10

    Check out the interview between Eli the Computer Guy and Baracules Nerdgasm from a year ago

    Poor depressed Eli is still trying to figure out how he can still have a future in a Microsoft World and Baracules is the happiest Guy on Earth

    Check out the Barnacules Videos on Windows Spyware and you can understand his happiness
    He does not let Microsoft dictate the framework of his business, life and future
    Youtube Search: Windows is Spyware (you will find him)

    Poor Billy is on that same sad downward slope that only leads to Suicide or Mass Murder

    Just pull yer head out of Nadella's Ass long enough to tell us how fast this drive can copy/paste 100GB

    It's not hard at all
    Even I could do it (just give me the drive)

    No need to re-invent the drive or come up with what ifs

    Easy-Peasy
    Now SMILE and repeat 3 times, Microsoft is the problem / not the solution
    Deep breath annnnnnd Relax

    There is a future if you make one!
    Reply
  • Lord of the Bored - Friday, November 10, 2017 - link

    Your test is dumb, and is attempting to measure something that can't actually be measured that way. Reply
  • "Bullwinkle J Moose" - Thursday, November 09, 2017 - link

    What would happen if Intel Colludes with AMD to implement this technology into onboard graphics instead of AMD's plan to use Flash in their graphics cards ?

    Seems to me like Internal throughput would be very important to the design
    Reply
  • Samus - Thursday, November 09, 2017 - link

    That is also file system dependent. For example, in Mac OS High Sierra, you can copy and paste (duplicate) any size file instantly on any drive formatted with APFS.

    But your question of a block by block transfer of a file internally for a 100GB file would likely take 50 seconds if not factoring in file system efficiency.
    Reply
  • cygnus1 - Thursday, November 09, 2017 - link

    That's not a copy of the file though. It's just a duplicate file entry referencing the same blocks. That and things like snapshots are possible thanks to the copy on write nature of that file system. But, if any of those blocks were to become corrupted, both 'copies' of the file are corrupt. Reply

Log in

Don't have an account? Sign up now