Application Benchmarks

With a complex multi-layer storage system like the Intel Optane Memory H10, the most accurate benchmarks will be tests that use real-world applications. BAPCo's SYSmark 2018 and UL's PCMark 10 are two competing suites of automated application benchmarks. Both share the general goal of assigning a score to represent total system performance, plus several subscores covering different common use cases. PCMark 10 is the shorter test to run and it provides a more detailed breakdown of subscores. It is also much more GPU-heavy with 3D rendering included in the standard test suite and some 3DMark tests included in the Extended test. SYSmark 2018 has the advantage of using the full commercial versions of popular applications including Microsoft Office and Adobe Creative Suite, and it integrates with a power meter to record total system energy usage over the course of the test.

The downside of these tests is that they cover only the most common everyday use cases, and do not simulate any heavy multitasking. None of their subtests are particularly storage-intensive, so most scores only vary slightly when changing between fast and slow SSDs.

BAPCo SYSmark 2018

BAPCo's SYSmark 2018 is an application-based benchmark that uses real-world applications to replay usage patterns of business users, with subscores for productivity, creativity and responsiveness. Scores represnt overall system performance and are calibrated against a reference system that is defined to score 1000 in each of the scenarios. A score of, say, 2000, would imply that the system under test is twice as fast as the reference system.

BAPCo SYSmark 2018 Scores
Creativity Productivity Responsiveness Overall

The Kaby Lake desktop and Whiskey Lake notebook trade places depending on the subtest; sometimes the notebook is ahead thanks to its extra RAM, and sometimes the desktop is ahead thanks to its higher TDP. These differences usually have a bigger impact than choice of storage, though the Responsiveness test does show that a hard drive alone is inadequate. The Optane Memory H10's score with caching on is not noticeably better than when using the QLC portion alone, and even the hard drive with an Optane cache is fairly competitive with the all-solid state storage configurations.

Energy Usage

The SYSmark energy usage scores measure total system power consumption, excluding the display. Our Kaby Lake test system idles at around 26 W and peaks at over 60 W measured at the wall during the benchmark run. SATA SSDs seldom exceed 5 W and idle at a fraction of a watt, and the SSDs spend most of the test idle. This means the energy usage scores will inevitably be very close. The notebook uses substantially less power despite this measurement including the display. None of the really power-hungry storage options (hard drives, Optane 900P) can fit in this system, so the energy usage scores are also fairly close together.

BAPCo SYSmark 2018 - Energy Consumption

The Optane Memory H10 was the most power-hungry M.2 option, and leaving the Optane cache off saves a tiny bit of power but not enough to catch up with the good TLC-based drives. The Optane SSD 800P has better power efficiency than most of the flash-based drives, but its low capacity is a hindrance for real-world use.

 

UL PCMark 10

PCMark 10 scores
Subscore:

The Optane cache provides enough of a boost to PCMark 10 Extended scores to bring the H10 into the lead among the M.2 SSDs tested on the Whiskey Lake notebook. The Essentials subtests show the most impact from the Optane storage while the more compute-heavy tasks are relatively unaffected, with the H10 performing about the same with or without caching enabled.

Test Setup Cache Size Effects
POST A COMMENT

65 Comments

View All Comments

  • yankeeDDL - Monday, April 22, 2019 - link

    Is it me or, generally speaking, it is noticeably slower than the 970 Evo? Reply
  • DanNeely - Monday, April 22, 2019 - link

    The 970 can make use of 4 lanes, with only 2 effective lanes in most scenarios any good x4 drive is going to be able to smoke the H10. Reply
  • yankeeDDL - Monday, April 22, 2019 - link

    I still remember that Optane should be 1000x faster and 1000x cheaper. It seems that it is faster, albeit by a much lower factor ... then why hamper it with a slower bus? I mean, I came to read the review thinking that it could be a nice upgrade, and then I see it beaten handily by the 970 Evo. What's the point of such device? It is clearly more complex, so I doubt it'll be cheaper than the 970 Evo... Reply
  • Alexvrb - Monday, April 22, 2019 - link

    Wait, did they say it would be cheaper? I don't remember that. I know they thought it would be a lot faster than it is... to be fair they seemed to be making projections like NAND based solutions wouldn't speed up at all in years LOL.

    It can be a lot faster in certain configs (the high end PCIe add-on cards, for example) but it's insanely expensive. Even then it's mainly faster for low QDs...
    Reply
  • kgardas - Tuesday, April 23, 2019 - link

    Yes, but just in comparison with DRAM prices. E.g. NVDIMM of big size cheaper than DIMM of big size. Reply
  • Irata - Tuesday, April 23, 2019 - link

    It was supposed to be 1000x faster and have 1000x the endurance of NAND as per Intel's official 2016 slides.

    It may be slightly off on those promises - would have loved for the article to include the slide with Intel's original claims.

    Price wasn't mentioned.
    Reply
  • yankeeDDL - Tuesday, April 23, 2019 - link

    You're right. They said 1000x faster, 1000x endurance and 10x denser, but they did not say cheaper, although, the 10x denser somewhat implies it (https://www.micron.com/~/media/documents/products/... Still, this drive is not faster, nor it has significantly higher endurance. Let's see if it is any cheaper. Reply
  • Valantar - Tuesday, April 23, 2019 - link

    Denser than DRAM, not NAND. Speed claims are against NAND, price/density claims against DRAM - where they might not be 1/10th the price, but definitely cheaper. The entire argument for 3D Xpoint is "faster than NAND, cheaper than DRAM (while persistent and closer to the former than the latter in capacity)", after all. Reply
  • CheapSushi - Wednesday, April 24, 2019 - link

    I think this is why there's still negative impressions around 3D Xpoint. Too many people still don't understand it or confuse the information given. Reply
  • cb88 - Friday, May 17, 2019 - link

    Optane itself is *vastly* faster than this... on an NVDIMM it rivals DDR4 with latencies in hundreds of ns instead of micro or milliseconds. And bandwidth basically on par with DDR4.

    I think it's some marketing BS that they don't use 4x PCIe on thier M.2 cards .... perhaps trying to avoid server guys buying them up cheap and putting them on quad m.2 to PCIe adapters.
    Reply

Log in

Don't have an account? Sign up now