CPU Tests: SPEC

Page by Andrei Frumusanu

SPEC2017 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0
clang version 7.0.1 (ssh://git@github.com/flang-compiler/flang-driver.git
 24bd54da5c41af04838bbe7b68f830840d47fc03)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labeled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For the new Cypress Cove based i7-11700K, we haven’t had quite the time to investigate the new AVX-512 instruction differences – since this is the first consumer desktop socketed CPU with the new ISA extensions it’s something we’ll revisit in the full review. Based on our testing on the server core counterparts however, it doesn’t make any noticeable differences in SPEC.

SPECint2017 Rate-1 Estimated Scores

In the SPECint2017 suite, we’re seeing the new i7-11700K able to surpass its desktop predecessors across the board in terms of performance. The biggest performance leap is found in 523.xalancbmk which consists of XML processing at a large +54.4% leap versus the 10700K.

The rest of the improvements range in the +0% to +15% range, with an average total geomean advantage of +15.5% versus the 10700K. The IPC advantage should be in the +18.5% range.

SPECfp2017 Rate-1 Estimated Scores

In the FP scores, there’s nothing standing out too much, with general even improvements across the board. The total improvement here is +19.6%, with the IPC improvement in the +22% range.

SPEC2017 Rate-1 Estimated Total

Although the new Cypress Cove cores in the 11700K do have good generational IPC improvements, that’s all compared to the quite old predecessor, meaning that for single-thread performance, the advancements aren’t enough to quite keep up with the latest Zen3 competition from AMD, or for that matter, the Firestorm cores in Apple’s new M1.

SPEC2017 Rate-N Estimated Total

More interesting are the multi-threaded SPEC results. Here, the new generation from Intel is showcasing a +5.8% and +16.2% performance improvement over its direct predecessor. Given the power draw increases we’ve seen this generation, those are rather unimpressive results, and actually represent a perf/W regression. AMD’s current 6-core 5600X actually is very near to the new 11700K, but consuming a fraction of the power.

CPU Tests: Encoding and Legacy/Web Gaming Tests: Deus Ex Mankind Divided
Comments Locked

541 Comments

View All Comments

  • Shorty_ - Monday, March 8, 2021 - link

    I'm not sure if you're being wilfully obtuse or ignorant.. the only reason Skylake is even remotely in the game is that intel's 14nm is refined enough to allow them to push raw clock speeds to the moon. Do you not recall how awful Ice Lake was because it couldn't clock? TGL is starting to clock a bit better but it's still pretty damn close. This is on 10nm "superfin" which is ~= TSMC N7(P).

    So Intel don't have some magic engineering pixie dust that would propel them beyond AMD if they were on the same node.
  • Thesubtlesnake - Saturday, March 6, 2021 - link

    Intel already have process equivalent to 7nm – 10nm SF. And they already designed a new architecture on it: Tiger Lake. And Zen 3 is perfectly competitive with Tiger Lake.
  • Teckk - Friday, March 5, 2021 - link

    Ultimately the latest desktop processors from Intel doesn’t perform well against AMD that’s what it is.
    They chose to release it on 14 nm as their 10nm was still work in progress. The numbers have meaning and not your conjecture about Intel using TSMC advanced node- it’ll be compared whenever that happens, with numbers.
  • Cooe - Saturday, March 6, 2021 - link

    *Zen 3
  • hfm - Saturday, March 6, 2021 - link

    But we have to live in reality that they don't even have 10nm ready for desktop. Fantasies about creating an alternate reality where their core architecture exists on a smaller node for desktop are just that, fantasies. The reality is AMD clearly has the far better product right now aside from niche edge cases.

    I still agree with the conclusion though that given current circumstances, get what you can get if you need to upgrade or build new. But the reality there seems like the 5800X is available at MSRP in-stock at multiple storefronts.
  • blppt - Friday, March 5, 2021 - link

    A chip that is just released, the best Intel currently has to offer for the mainstream consumer, can't match a chip that has been out for months. While using more power.

    Thats not a good look for Intel. I hope the 11900K (or whatever they're going to call it) at least matches the 5900X in games.

    This is the first time in a long time, with generations of chips current, that I cannot think of a single reason to recommend Intel's latest and greatest over AMD.
  • terroradagio - Friday, March 5, 2021 - link

    The 11900k has always been what should be compared to the 5900x anyway. Not the i7-11700k.
  • blppt - Saturday, March 6, 2021 - link

    The point being, the 11700k doesn't even catch the 5800X, which has been out for a few months already. Given that this was supposed to be Intel's "response to Zen 3", its pretty disappointing.
  • Fulljack - Saturday, March 6, 2021 - link

    the only thing that could save Rocket Lake-S are availability and price. otherwise just get Ryzen 5000 processors.
  • SaturnusDK - Saturday, March 6, 2021 - link

    Availability of Ryzen 5000 except 59xx parts is already a non-issue. You can get 5600X with a few days delay at worst, and 5800X is in abundant stock pretty much everywhere.

    The key is price, especially the platform price because Intel MBs are generally more expensive. On top of that you absolutely need a larger cooler, and most likely also need a beefier PSU for the Intel CPUs, so the CPU price for the intel parts have to be substantially lower than a performance equivalent AMD part to be competitive. And given the history of intel that seems very unlikely to happen.

Log in

Don't have an account? Sign up now