Testing Spectre and Meltdown: SYSMark

As we were performing this testing, the issue of Spectre and Meltdown reared its ugly head. After 40 hours of testing, we realised that the motherboard was not BIOS patched for the latest issues, and we reached out to get the latest update, and had to retest all over again.

It was around this time that Intel also reached out to us to give us the results of their own performance testing relating to the patches. The long and short of the discussions about Intel results were that the patches affected systems with older the most, and systems that had fast storage (SSD vs HDD) also took the brunt of the performance hit.

For our testing, we took the SYSMark benchmark and did a before and after comparison. We confirmed the patches were applied by using the Inspectre tool before running in patched mode. You can read our analysis of the Spectre and Meltdown issues in the following articles:

SYSMark 2014 SE

For the overall score, every processor lost some performance:

SYSMark 2014 SE: Overall Score

The biggest overall loser in real terms was the W-2155, which mixes single core performance with many threads. This is interesting - the processor with the most threads, the W-2195, did not have such a percentage dip. This might be related to how each of these processors is laid out differently: the W-2195 uses Intel's HCC 18-core die, whereas the W-2155 uses the LCC 10-core die. The HCC die has extra core-to-core latency because of the larger floorplan, which might hide some of the deficiencies here.

If we compare the percentage decrease across all of the SYSMark sub-tests:

We can see that the biggest decreases are seen in the Response sub-test, which contributes a lot to the overall score decreases. The response sub-test uses a fair amount of storage, which we know is likely to be the biggest loser from the patches. However, our overall decreases in performance range from 2.0% on the small slow core to 5.6% on the 10-core and back down to 3.5% on the largest 18-core part. The hardest hit tests were down 12%.

Benchmarking Performance: CPU Legacy Tests Conclusion: Is Intel Serious About Xeon W?
Comments Locked

74 Comments

View All Comments

  • Alsw - Monday, July 30, 2018 - link

    Nice article, unless i missed it it would be really handy to know all core Turbo boosts you acheived as i don't' think Intel release this info any more? i am in a situation where our FEA/CFD applications beneifit from both frequency and GHz so it is tricky, even before you start taking into acount whether dual CPU is better with the potential for performance variance data being passed between two physical CPU's
  • Ian Cutress - Monday, July 30, 2018 - link

    I thought I had the information, but I do not. I've reached out to Intel - normally the enterprise side of the business gives out this info, but the consumer side does not. I'll update the review when we get the details.
  • eek2121 - Monday, July 30, 2018 - link

    Looks like it's time to update your benchmark suite and redo benchmarks. Octane 2.0 has long since been retired, WebXPRT 15 as well. I like chrome, but you can disable updates in Firefox fairly easily. 7zip is at (!) 18.05. The version you are using is from 2010.
  • Ian Cutress - Monday, July 30, 2018 - link

    We've got a new benchmark suite for our next review, I put the finishing touches to it recently. The time we had these CPUs, it was not ready in time (also for retesting - a new suite takes about a month to bed in with older hardware).
  • HStewart - Monday, July 30, 2018 - link

    Ian, it would be really nice to see the performance benefits of AVX 512 in these benchmarks.

    I try to search for what applications are available that use AVX 512 - only thing I found was the following

    https://www.hpcwire.com/2017/06/29/reinders-avx-51...
  • Ian Cutress - Monday, July 30, 2018 - link

    I've got a few for our new suite. 3DPM has an AVX-512 mode now, and I've got the latest y-cruncher
  • JoeyJoJo123 - Monday, July 30, 2018 - link

    "AMD’s chiplet design will take a few generations to get used to"

    Is chiplet the silicon equivalent of calling someone a manlet? Lol.
  • jcc5169 - Monday, July 30, 2018 - link

    There they go again! More Intel Fanboy propaganda !!!
  • Ian Cutress - Monday, July 30, 2018 - link

    So a review of a product = fan boy? What? Last week I was told I was an AMD shill.
  • BurntMyBacon - Tuesday, July 31, 2018 - link

    CONFIRMED! Ian is a double agent. Now that his cover is blown, he must commence emergency extraction procedures and call in a body double.

    Or maybe people just get a little too upset when things don't go as expected ..... Nah, I like the double agent story better. You'll have to let us know what your call signs were.

Log in

Don't have an account? Sign up now