Testing Spectre and Meltdown: SYSMark

As we were performing this testing, the issue of Spectre and Meltdown reared its ugly head. After 40 hours of testing, we realised that the motherboard was not BIOS patched for the latest issues, and we reached out to get the latest update, and had to retest all over again.

It was around this time that Intel also reached out to us to give us the results of their own performance testing relating to the patches. The long and short of the discussions about Intel results were that the patches affected systems with older the most, and systems that had fast storage (SSD vs HDD) also took the brunt of the performance hit.

For our testing, we took the SYSMark benchmark and did a before and after comparison. We confirmed the patches were applied by using the Inspectre tool before running in patched mode. You can read our analysis of the Spectre and Meltdown issues in the following articles:

SYSMark 2014 SE

For the overall score, every processor lost some performance:

SYSMark 2014 SE: Overall Score

The biggest overall loser in real terms was the W-2155, which mixes single core performance with many threads. This is interesting - the processor with the most threads, the W-2195, did not have such a percentage dip. This might be related to how each of these processors is laid out differently: the W-2195 uses Intel's HCC 18-core die, whereas the W-2155 uses the LCC 10-core die. The HCC die has extra core-to-core latency because of the larger floorplan, which might hide some of the deficiencies here.

If we compare the percentage decrease across all of the SYSMark sub-tests:

We can see that the biggest decreases are seen in the Response sub-test, which contributes a lot to the overall score decreases. The response sub-test uses a fair amount of storage, which we know is likely to be the biggest loser from the patches. However, our overall decreases in performance range from 2.0% on the small slow core to 5.6% on the 10-core and back down to 3.5% on the largest 18-core part. The hardest hit tests were down 12%.

Benchmarking Performance: CPU Legacy Tests Conclusion: Is Intel Serious About Xeon W?
Comments Locked

74 Comments

View All Comments

  • mode_13h - Monday, July 30, 2018 - link

    Just the fact that you need to map your VMs to stay on the same physical core, for best performance (i.e. so that the memory is local to it). If you do that, TR is actually a great VM solution.
  • Death666Angel - Monday, July 30, 2018 - link

    That doesn't sound so complicated as to be a "situation" for someone dealing with VMs. :) Seems like a general setup config thing that you just check off when you do it once.
  • mode_13h - Tuesday, July 31, 2018 - link

    Sorry, I meant "same physical die". Actually, best results are from setting affinity at the CCX (i.e. 4-core) granularity.
  • SanX - Tuesday, July 31, 2018 - link

    The day ARM announced it developed 8 and 16-core server and supercomputer chips at their usual price around $25 per 5 billion transistors, Intel Xeon prices would plunge 10-50x.
  • Infy2 - Monday, July 30, 2018 - link

    It would be helpful on those charts if there was an indicator how many cores/threads in CPU has.
  • Ian Cutress - Monday, July 30, 2018 - link

    We used to have that info in the graphs, however people found it redundant when it was elsewhere, and price/power was requested instead. It's hard to put all the info of every part into every graph!
  • Death666Angel - Monday, July 30, 2018 - link

    Can that not be coded as a tooltip/mouse-over text? That would be neat and not add clutter while adding information to those who want it. :)
  • Ian Cutress - Monday, July 30, 2018 - link

    Graphs are images generated locally from the data. I'm not au fait with how our back end works, but that's require more than a simple rewrite
  • Death666Angel - Tuesday, July 31, 2018 - link

    Alright. :) Shame though, would be a useful feature for some. Maybe add it to an overhaul list, if such a thing exists. :)
  • lkuzmanov - Tuesday, July 31, 2018 - link

    I'm with Infy2 on this one, not sure who protested, but I think something like (10/20) next to the model wouldn't be too distracting or cost too much screen space. I caught myself having to go back to the first page of the article to check the core count of the Xeon parts. That said - great content, you're one of my daily go-to sites.

Log in

Don't have an account? Sign up now