CPU Tests: SPEC

Page by Andrei Frumusanu

SPEC2017 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0
clang version 7.0.1 (ssh://git@github.com/flang-compiler/flang-driver.git
 24bd54da5c41af04838bbe7b68f830840d47fc03)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labeled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For the new Cypress Cove based i7-11700K, we haven’t had quite the time to investigate the new AVX-512 instruction differences – since this is the first consumer desktop socketed CPU with the new ISA extensions it’s something we’ll revisit in the full review. Based on our testing on the server core counterparts however, it doesn’t make any noticeable differences in SPEC.

SPECint2017 Rate-1 Estimated Scores

In the SPECint2017 suite, we’re seeing the new i7-11700K able to surpass its desktop predecessors across the board in terms of performance. The biggest performance leap is found in 523.xalancbmk which consists of XML processing at a large +54.4% leap versus the 10700K.

The rest of the improvements range in the +0% to +15% range, with an average total geomean advantage of +15.5% versus the 10700K. The IPC advantage should be in the +18.5% range.

SPECfp2017 Rate-1 Estimated Scores

In the FP scores, there’s nothing standing out too much, with general even improvements across the board. The total improvement here is +19.6%, with the IPC improvement in the +22% range.

SPEC2017 Rate-1 Estimated Total

Although the new Cypress Cove cores in the 11700K do have good generational IPC improvements, that’s all compared to the quite old predecessor, meaning that for single-thread performance, the advancements aren’t enough to quite keep up with the latest Zen3 competition from AMD, or for that matter, the Firestorm cores in Apple’s new M1.

SPEC2017 Rate-N Estimated Total

More interesting are the multi-threaded SPEC results. Here, the new generation from Intel is showcasing a +5.8% and +16.2% performance improvement over its direct predecessor. Given the power draw increases we’ve seen this generation, those are rather unimpressive results, and actually represent a perf/W regression. AMD’s current 6-core 5600X actually is very near to the new 11700K, but consuming a fraction of the power.

CPU Tests: Encoding and Legacy/Web Gaming Tests: Deus Ex Mankind Divided
Comments Locked

541 Comments

View All Comments

  • brucethemoose - Monday, March 8, 2021 - link

    1: AT benches on Windows, and right now x86 vs ARM is kinda Apples-to-oranges on that platform, especially when one starts mixing in emulation and AVX.

    Give it time. More comparisons will come. But you'll probably see comparisons on Linux/Mac, and open source software in general, sooner.

    2: People uses CPUs for different things. Some of these benches are relevant to those people.

    At the same time, my use cases weren't really covered here, so... I get what you're saying.

    3. Yeah, it seems rather silly to me, especially when Anand test GPU limited AAA games.

    Where you really need a big CPU is in simulation/sandbox games, especially in servers for such things, and sluggish early access stuff. But no one ever benches those :/
  • Silver5urfer - Monday, March 8, 2021 - link

    So with the 19% IPC claim and losing to 9900KS and 10700K what is the point of releasing this chip, from Intel. I never got much from AT benches a lot and preferred the Hardware unboxed, Gamernexus guys and others. But if this is the final performance figures, then this is really a DOA product from Intel. How can they allow this ? I never saw Intel in such a position..maybe X299 got rekted when Zen 2 dropped but this is mainstream segment.

    Damn it. AMD processors have the idiotic stock related issues, add that WHEA and USB shitstorm. Intel has bullshit performance over past gen except a Gen4 addition and extra lanes from chipset. GPUs are out of damn stock as well.

    2020 and 2021 both are completely fucked up for PC HW purchases.
  • Gigaplex - Monday, March 8, 2021 - link

    "I never saw Intel in such a position.."

    The Pentium 4, in particular the Prescott architecture was a dud back in the early 2000's. That era spawned the antitrust lawsuit against Intel for illegally blocking AMD sales since the Intel products weren't competitive.
  • dwbogardus - Monday, March 8, 2021 - link

    The fact that Intel can even remain in a close second place, using a 14 nm process is impressive. Imagine what they could do with TSMC's 7 nm process! It would almost certainly outperform AMD by a significant margin.
  • Bagheera - Monday, March 8, 2021 - link

    Really tired of Intel fanboys saying this.

    architectures are designed for specific nodes - RKL's problems are exactly due to porting an arch onto a node it wasn't designed for.

    the fact is Intel is not a partner for TSMC and their archs are not designed for TSMC processes. if Intel were to outsource CPU production to TSMC, they will either have to make a new arch or make make tweaks to existing ones - a multi-million $ endeavor with risms of issues like your just read with RKL.
  • Hifihedgehog - Tuesday, March 9, 2021 - link

    > a close second place

    ^Here we see in his natural environment your common everyday dude who fails at reading comprehension. I guess you didn't read the part about the serious gaming performance losses and latency regressions gen-over-gen, the 10% performance gap in single-threaded or 10-20%+ performance gaps in multithread, or the inexcusably high peak power draw? Talk about deluded...
  • RanFodar - Monday, March 8, 2021 - link

    Though their efforts may be futile, I am glad Intel attempted to do something out of the ordinary; not a Skylake refresh, but a backport that is found to have worse performance. And yet, it is an attempt for Intel to learn their lessons for generations to come.
  • Hifihedgehog - Tuesday, March 9, 2021 - link

    Backporting is not a lesson; it is a last ditch effort or a fallback when all else fails on the manufacturing side. Half full, half empty cup viewing aside, they wasted even more valuable engineering manhours into a failed backport when it should have been invested into developing new architectures. A best use would have been developing the next release. The problem is Intel had to make Rocket Lake good enough in synthetic benchmarks to appease their investors. That, however, still does not address the elephants in the room of 10-20% single threaded performance gaps or—the one that takes the cake—the latency regressions that makes gaming worse, Intel’s historic crown jewel. Much like movies that fail at release and live on box office bombs that their producers later opine should have been cut early on in development, Intel should have cut this idea early on. If you are looking for a lesson that Intel should have learned here, there it is: avoid another Rocket Lake backporting disaster and just warm over your current microarchitecture with one more middling refresh one last time.
  • Hifihedgehog - Tuesday, March 9, 2021 - link

    Ian, I just want to say thank you for the incredible review. Just ignore the haters on social media and in the comments who get their panties in a bunch. If the product is garbage, say it like it is, like you did and quite well I might add. You were incredibly diplomatic about it and even openly and honestly showed when and where Intel did win on the rare occasion in the benchmarks. It is so silly how people make these CPU companies (who don't know them from Sam Hill) their religion, as if erecting a Gordon Moore or Lisa Su shrine would avail them anything. Silly geese.
  • misiu_mp - Wednesday, March 10, 2021 - link

    Is that the new bulldozer?

Log in

Don't have an account? Sign up now