Testing Spectre and Meltdown: SYSMark

As we were performing this testing, the issue of Spectre and Meltdown reared its ugly head. After 40 hours of testing, we realised that the motherboard was not BIOS patched for the latest issues, and we reached out to get the latest update, and had to retest all over again.

It was around this time that Intel also reached out to us to give us the results of their own performance testing relating to the patches. The long and short of the discussions about Intel results were that the patches affected systems with older the most, and systems that had fast storage (SSD vs HDD) also took the brunt of the performance hit.

For our testing, we took the SYSMark benchmark and did a before and after comparison. We confirmed the patches were applied by using the Inspectre tool before running in patched mode. You can read our analysis of the Spectre and Meltdown issues in the following articles:

SYSMark 2014 SE

For the overall score, every processor lost some performance:

SYSMark 2014 SE: Overall Score

The biggest overall loser in real terms was the W-2155, which mixes single core performance with many threads. This is interesting - the processor with the most threads, the W-2195, did not have such a percentage dip. This might be related to how each of these processors is laid out differently: the W-2195 uses Intel's HCC 18-core die, whereas the W-2155 uses the LCC 10-core die. The HCC die has extra core-to-core latency because of the larger floorplan, which might hide some of the deficiencies here.

If we compare the percentage decrease across all of the SYSMark sub-tests:

We can see that the biggest decreases are seen in the Response sub-test, which contributes a lot to the overall score decreases. The response sub-test uses a fair amount of storage, which we know is likely to be the biggest loser from the patches. However, our overall decreases in performance range from 2.0% on the small slow core to 5.6% on the 10-core and back down to 3.5% on the largest 18-core part. The hardest hit tests were down 12%.

Benchmarking Performance: CPU Legacy Tests Conclusion: Is Intel Serious About Xeon W?
Comments Locked

74 Comments

View All Comments

  • Lolimaster - Tuesday, July 31, 2018 - link

    I see no point in this when TR exist, not only 1950X already crushes but the 2990X will just made them an afterthought.
  • cm2187 - Tuesday, July 31, 2018 - link

    It says “workstation” but is there any reason not to base an entry level server on these specs? Cheaper than server chips, ample of ram, ECC, vpro. Is there anything Intel will do to enforce a segmentation?
  • GreenReaper - Monday, August 6, 2018 - link

    That is almost certainly one of the purposes of the custom 4-core editions. HP's MicroServer Gen8 had a two-core 2.3Ghz Celeron with ECC support - this has significantly more wattage but I'd expect to see it in hardware with a need for long-term highly-reliable duty like communications equipment.
  • buxe2quec - Tuesday, July 31, 2018 - link

    This may be a stupid question, but how come my E3-1220 (3.1 GHz) from 2011 has 80 W TDP and this Xeon W-2104 (first table) has 120 W?
    I thought that power consumption went down per MHz... this is 50% increase.
  • buxe2quec - Tuesday, July 31, 2018 - link

    I saw the numbers on page 2 about the real tests, but I don't have the ones for the E3-1220 to compare the actual values, so I was comparing only nominal TDP.
  • Hamm Burger - Tuesday, July 31, 2018 - link

    I'm lucky enough to be using a 10-core iMac Pro, so have the Apple-specific W-2150B. I'm afraid I'm not about to prise it out of the system so that you can test it, but here's the result of one anecdotal test: running the CPU portion of Cinebench 15 for macOS gives a mutithreaded score of 2012 and single-threaded of 182 — a spot below your figures for the W-2195. Also, Intel Power Gadget shows the CPU drawing 150W, with the cores hitting almost 100° during the multithreaded test.
  • abufrejoval - Tuesday, July 31, 2018 - link

    I guess Intel wants to ensure AMD Threadripper gets the home-grown workstation market going forward…

    Like you mention, previous generation CPUs, even high core count variants are floating around in the second hand market and I got myself an OEM variant of the E5-2699v3 (E5-2696v) about two years back for around $700 from China via eBay (“extremely affordable”). That’s an 18core chip that will clock a little higher than the 2699, 3.6GHz instead of 3.3 when fewer cores are used, while the all-core clocks and TDP (145 Watts) are the same.

    I am running this in an X99 board with 128GB of ECC UDIMM (bought before the RAM prices hiked 100%) and operating it with a BCLK overclock of 103.8, which results in a clean 4GHz for low-core workloads, 3.8GHz with four cores active and 2.8GHz for all-core unless it’s AVX workloads (prime95), where it may drop to 2.6GHz, all with well below 140 Watts and generally quite cool with an unnoticeable Noctua fan inside a $60 cheapo tower.

    It runs games rather well, clocking high on the few cores most game engines use and it also does well using lots of cores on things like massive compile jobs (make -j40) or machine learning tasks (helped along by GTX 1080ti where GPUs are better).

    It gets 2552 on Cinebench R15, so it won’t quite beat the current generation Threadrippers or these Xeons, but at the premium prices Intel wants to charge for Xeon-W as well as current DRAM prices, I simply couldn’t afford something in this league for the home-lab.
  • abufrejoval - Tuesday, July 31, 2018 - link

    Here is a Geekbench result for this rig: https://browser.geekbench.com/v4/cpu/9220520
  • alpha754293 - Thursday, August 2, 2018 - link

    The other reason why someone might consider the Xeon W (such as myself) - high memory and need a very fast single threaded performance.

    The consumer parts are limited to 64 GB (ECC or not) of RAM whereas the Xeon W caps out at 512 GB.

    Most "normal" people might not need that, but I can tell you right now that for some of the pre and post-processing work that I do, I'm looking now at either a 256 or 512 GB system with very fast single threaded performance.
  • Dug - Thursday, August 2, 2018 - link

    I know you've heard it before, but just want to throw in my 2 cents.

    Could you please try a newer version of Handbrake for H.256 benchmarks. I know when doing comparisons you need consistency and it's best to stick with one version, but x265 is becoming very popular, and the new version fixes previous x265 issues. Plus they have new Production presets which might be helpful. Thanks for any consideration.

Log in

Don't have an account? Sign up now