Testing Spectre and Meltdown: SYSMark

As we were performing this testing, the issue of Spectre and Meltdown reared its ugly head. After 40 hours of testing, we realised that the motherboard was not BIOS patched for the latest issues, and we reached out to get the latest update, and had to retest all over again.

It was around this time that Intel also reached out to us to give us the results of their own performance testing relating to the patches. The long and short of the discussions about Intel results were that the patches affected systems with older the most, and systems that had fast storage (SSD vs HDD) also took the brunt of the performance hit.

For our testing, we took the SYSMark benchmark and did a before and after comparison. We confirmed the patches were applied by using the Inspectre tool before running in patched mode. You can read our analysis of the Spectre and Meltdown issues in the following articles:

SYSMark 2014 SE

For the overall score, every processor lost some performance:

SYSMark 2014 SE: Overall Score

The biggest overall loser in real terms was the W-2155, which mixes single core performance with many threads. This is interesting - the processor with the most threads, the W-2195, did not have such a percentage dip. This might be related to how each of these processors is laid out differently: the W-2195 uses Intel's HCC 18-core die, whereas the W-2155 uses the LCC 10-core die. The HCC die has extra core-to-core latency because of the larger floorplan, which might hide some of the deficiencies here.

If we compare the percentage decrease across all of the SYSMark sub-tests:

We can see that the biggest decreases are seen in the Response sub-test, which contributes a lot to the overall score decreases. The response sub-test uses a fair amount of storage, which we know is likely to be the biggest loser from the patches. However, our overall decreases in performance range from 2.0% on the small slow core to 5.6% on the 10-core and back down to 3.5% on the largest 18-core part. The hardest hit tests were down 12%.

Benchmarking Performance: CPU Legacy Tests Conclusion: Is Intel Serious About Xeon W?
Comments Locked

74 Comments

View All Comments

  • 0ldman79 - Tuesday, July 31, 2018 - link

    Agreed.

    I imagine there will be several situations where the 6 core 12 thread i7 will outperform the i9 9700.

    If the cache increase is enough that may not happen, but I'm not betting on it making up enough of a difference.
  • Icehawk - Friday, August 3, 2018 - link

    The 8086 is functionally equivalent to the 8700, they trade blows in a pretty tight grouping so I don’t mind that they used it’s scores.
  • mode_13h - Monday, July 30, 2018 - link

    Their pricing is nuts! These can only make sense if you're desperate for PCIe lanes or lots of cores (and, for some reason, don't want AMD).

    I have an older E5 Xeon and wanted to replace it with a W-series, but I can't justify this pricing (or the performance hit taken on the lower-core-count models, relative to desktop/E-series chips). I will have to opt for either an E-series Xeon or a Ryzen. At this rate, I see myself going for a 7 nm Ryzen, actually.

    I think AMD is smart for using narrow AVX units. > 256-bit doesn't really make sense for much that wouldn't be better-served by a GPU. AVX-512 was a strategic misstep for Intel, and they're just going to have to live with it.
  • mode_13h - Monday, July 30, 2018 - link

    Oh, and let's not forget the IHS TIM issue.

    I'm not in the market for > 8 cores, but those who are will be disappointed by the rate of thermal throttling, due to this being their first (recent) workstation/HEDT chip with a non-soldered IHS.
  • 0ldman79 - Tuesday, July 31, 2018 - link

    I missed that.

    The IHS is using TIM even on the Xeon now?

    That was honestly the one big reason I was looking at the Xeon. That's just a poor business decision. Xeon carries a price premium, they could at least guarantee the heat conductivity is going to be enough to keep it running cool and smoothly for the life of the chip.
  • mode_13h - Wednesday, August 1, 2018 - link

    I don't know this for a fact, but their Xeons are normally just binned HEDT processors without the special features fused off. So, I assume it's the same crappy TIM under that IHS.

    Comparing thermal performance @ the same clock for 10+ core models vs the i9's would easily show whether this is true.
  • HStewart - Monday, July 30, 2018 - link

    "I think AMD is smart for using narrow AVX units. > 256-bit doesn't really make sense for much that wouldn't be better-served by a GPU. AVX-512 was a strategic misstep for Intel, and they're just going to have to live with it."

    AMD's AVX 2 is only 1/2 of Intel AVX 2 - that sound like they are using dual 128 bits instead 256 bits

    Also keep in mind Intel CPU also have AVX 2 support your statement makes no sense.
  • HStewart - Monday, July 30, 2018 - link

    https://forums.anandtech.com/threads/ryzens-halved...
  • bill.rookard - Monday, July 30, 2018 - link

    The thing is - and this is somewhat critical for a workstation based board, you're NOT really going to be using it for single threaded tasks. You'll be using software which has for the most part SHOULD be multi-threaded. Considering that the Threadripper is a 16c/32t CPU in the gen1, and running for a street price of about $800ish, and the gen2 is going to be a 32c/64t beastie of a CPU at a price of $1500ish, why would you spend $2500 on a 18c/36t Intel CPU?

    You could just as easily do some research to find people who have indeed put together some TR/ECC combos, and put a complete AMD system for the price of an Intel CPU alone.
  • mode_13h - Tuesday, July 31, 2018 - link

    That's nuts, dude. For software development, I want lotsa cores for parallel builds. When recompiling only a few files, I want fast single-thread perf.

    The reality is that there are still lots of places in day-to-day computing where single-thread perf matters. I don't know how you can possibly believe you accurately represent the needs of all workstation users, but you don't.

Log in

Don't have an account? Sign up now