CPU Tests: SPEC

Page by Andrei Frumusanu

SPEC2017 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0
clang version 7.0.1 (ssh://git@github.com/flang-compiler/flang-driver.git
 24bd54da5c41af04838bbe7b68f830840d47fc03)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labeled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For the new Cypress Cove based i7-11700K, we haven’t had quite the time to investigate the new AVX-512 instruction differences – since this is the first consumer desktop socketed CPU with the new ISA extensions it’s something we’ll revisit in the full review. Based on our testing on the server core counterparts however, it doesn’t make any noticeable differences in SPEC.

SPECint2017 Rate-1 Estimated Scores

In the SPECint2017 suite, we’re seeing the new i7-11700K able to surpass its desktop predecessors across the board in terms of performance. The biggest performance leap is found in 523.xalancbmk which consists of XML processing at a large +54.4% leap versus the 10700K.

The rest of the improvements range in the +0% to +15% range, with an average total geomean advantage of +15.5% versus the 10700K. The IPC advantage should be in the +18.5% range.

SPECfp2017 Rate-1 Estimated Scores

In the FP scores, there’s nothing standing out too much, with general even improvements across the board. The total improvement here is +19.6%, with the IPC improvement in the +22% range.

SPEC2017 Rate-1 Estimated Total

Although the new Cypress Cove cores in the 11700K do have good generational IPC improvements, that’s all compared to the quite old predecessor, meaning that for single-thread performance, the advancements aren’t enough to quite keep up with the latest Zen3 competition from AMD, or for that matter, the Firestorm cores in Apple’s new M1.

SPEC2017 Rate-N Estimated Total

More interesting are the multi-threaded SPEC results. Here, the new generation from Intel is showcasing a +5.8% and +16.2% performance improvement over its direct predecessor. Given the power draw increases we’ve seen this generation, those are rather unimpressive results, and actually represent a perf/W regression. AMD’s current 6-core 5600X actually is very near to the new 11700K, but consuming a fraction of the power.

CPU Tests: Encoding and Legacy/Web Gaming Tests: Deus Ex Mankind Divided
Comments Locked

541 Comments

View All Comments

  • Geef - Tuesday, March 9, 2021 - link

    What, you don't?
  • Hifihedgehog - Saturday, March 6, 2021 - link

    I predicted ~300W peak power about a year or so ago when I first heard they were making the big mistake of bringing AVX-512 to mainstream consumer processors. Why on earth? There was a valid, wise, very, very, very good reason Intel had reserved AVX-512 to just their 14-nm HEDT processors and that had always been the furnace-like heat and nuclear-like power consumption of it. Even with the best logical design improvements from a new microarchitecture, it is still an extremely intensive logical pill of a task to swallow. Now, we see that reason in full, unadulterated display. You bring a massively complex instruction set extension to a higher process node where you have far lengthier physical networks (meaning essentially longer wires, increased resistance, higher power, and maximum heat) and, of course, you are going to have a steaming pile. Remember this review is only looking at the number two product, the Core i7-11700K which has lower clocks and lower power draw. The Core i9-11900K will likely need a 360- or 420-mm AIO just to not thermal throttle like mad. My 5950X with its meager 240mm AIO (Corsair H100i RGB Platinum) that runs at the quiet mode setting is laughing its butt off right about now. When Ian Cutress had to use an obnoxiously loud 170 CFM fan (I have used 100 CFM Deltas and those already annoy most PC enthusiasts) on a massive 4-pound, full copper heatsink to tame the 11700K's 290W, I shudder to think. Will the 11900K be outdoing the FX-9590's record-making peak power draw of 350W (see here: https://www.anandtech.com/show/8316/amds-5-ghz-tur... ? Ian easily could have gotten the 11900K also at retail, but I think he is holding back on that because he already knows the 11900K is going to be a throttling disaster and only the 11700K is an ACTUALLY USABLE PROCESSOR. *mike drop*
  • Hifihedgehog - Saturday, March 6, 2021 - link

    I am referring to 14nm mainstream consumer processors if that wasn't abundantly clear.
  • Santoval - Saturday, March 6, 2021 - link

    "To be clear, I was replying to you to get a reply at the top of the comments".
    You got me.. :)
  • Samus - Saturday, March 6, 2021 - link

    In all fairness, it's actually amazing what Intel has achieved here with, essentially, a 6 year old manufacturing process!
  • Spunjji - Monday, March 8, 2021 - link

    Are we reading the same review? It looks like they *regressed* in performance vs their 5.5-year-old architecture on the same process.
  • dihartnell - Thursday, March 11, 2021 - link

    I think they did they realistically could given the contraints they have. We are not going to see intels true potential until they get thier manufacturing process fixed or they swallow the dead rat and go to another foundry. This will be enough to keep them in the game for another year.
  • Hifihedgehog - Friday, March 5, 2021 - link

    Summary image:

    https://i.redd.it/4i2eu882qbl61.png

    tl;dr: Worse in games, moderately better in synthetics, slower and far more power hungry than Ryzen 5000
  • Bik - Saturday, March 6, 2021 - link

    Thanks
  • Gondalf - Saturday, March 6, 2021 - link

    Main defect of the article: AMD best of the best (low availability) versus a common high volume medium level Intel SKU.
    No matter the price that will change on retails or OEMS.

    I don't understand AMD, they pump hard the pedal still they can not do much to gain market share. New processes are medium volume and with too much customers.
    The NEW AMD will be a company capable to deliver a pile of good dies to all channels, without limitation on volume. Unfortunately Lisa follow the wrong street, in this manner Intel will always dominate the market.
    Bet 2021 will be Lisa last year at AMD.

Log in

Don't have an account? Sign up now