CPU Tests: SPEC

Page by Andrei Frumusanu

SPEC2017 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0
clang version 7.0.1 (ssh://git@github.com/flang-compiler/flang-driver.git
 24bd54da5c41af04838bbe7b68f830840d47fc03)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labeled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For the new Cypress Cove based i7-11700K, we haven’t had quite the time to investigate the new AVX-512 instruction differences – since this is the first consumer desktop socketed CPU with the new ISA extensions it’s something we’ll revisit in the full review. Based on our testing on the server core counterparts however, it doesn’t make any noticeable differences in SPEC.

SPECint2017 Rate-1 Estimated Scores

In the SPECint2017 suite, we’re seeing the new i7-11700K able to surpass its desktop predecessors across the board in terms of performance. The biggest performance leap is found in 523.xalancbmk which consists of XML processing at a large +54.4% leap versus the 10700K.

The rest of the improvements range in the +0% to +15% range, with an average total geomean advantage of +15.5% versus the 10700K. The IPC advantage should be in the +18.5% range.

SPECfp2017 Rate-1 Estimated Scores

In the FP scores, there’s nothing standing out too much, with general even improvements across the board. The total improvement here is +19.6%, with the IPC improvement in the +22% range.

SPEC2017 Rate-1 Estimated Total

Although the new Cypress Cove cores in the 11700K do have good generational IPC improvements, that’s all compared to the quite old predecessor, meaning that for single-thread performance, the advancements aren’t enough to quite keep up with the latest Zen3 competition from AMD, or for that matter, the Firestorm cores in Apple’s new M1.

SPEC2017 Rate-N Estimated Total

More interesting are the multi-threaded SPEC results. Here, the new generation from Intel is showcasing a +5.8% and +16.2% performance improvement over its direct predecessor. Given the power draw increases we’ve seen this generation, those are rather unimpressive results, and actually represent a perf/W regression. AMD’s current 6-core 5600X actually is very near to the new 11700K, but consuming a fraction of the power.

CPU Tests: Encoding and Legacy/Web Gaming Tests: Deus Ex Mankind Divided
Comments Locked

541 Comments

View All Comments

  • TheinsanegamerN - Friday, March 5, 2021 - link

    That 14nm chip pulls over twice the power of the 7nm 16 core chip and is consistently slower then 7nm 8 core chip.

    It's not so much "right behind them" but rather "barely keeping up while burning through a nuclear reactor's power output".
  • CiccioB - Friday, March 5, 2021 - link

    Twice the power for being slower?
    If you are referring to the 290W power consumption with AVX-512 test, I'm desolate to inform you that a 7nm CPU with twice the core would not reach those performances and perf/W in that test.

    If you are talking about the 140-160W usage at other "normal" tests, I'm desolate to inform you that a 16 core 7nm CPU does not consumes 80W.

    So stop vomiting meaningless numbers. This is a 14nm CPU and for the process it is build on it is doing miracles. If Intel could ever use an advanced PP like those 7nm by TSMC Zen would still be the underdog.

    For the future just hope the TMSC 5nm are good, early, low cost and really high yielding, because if Intel comes with a decent 7nm I think AMD will not look all that advanced (6 years to surpass a 6 years old architecture and all by the use of a more advanced PP that unfortunately doesn't allow for great deliveries).
  • blppt - Friday, March 5, 2021 - link

    "So stop vomiting meaningless numbers. This is a 14nm CPU and for the process it is build on it is doing miracles. If Intel could ever use an advanced PP like those 7nm by TSMC Zen would still be the underdog."

    Not necessarily. Intel apparently is still behind in IPC/single thread performance, as evidenced by that Cinebench results, so whilst 7nm would let it run with less power (theoretically), it still would lose to its main competitor, the 5800X.
  • CiccioB - Friday, March 5, 2021 - link

    You have missed that with 14nm die area you cannot improve the architecture that much.
    You are still thinking that CPU designed for 7nm, with all the advantages that they would bring, would still be like Skylake which is a 6 years old architecture
    A PP like TSMC 7nm would bring a completely new architecture that would blow Zen away.

    Zen is good because it is based on such a better PP that those Intel has now, but it still struggles at beating Skylake. And to do that it, that is by using such an advanced but production limited PP, it has sacrificed high stock delivery right in the period where demand is much higher than supply.
    Intel can fill the remaining market with whatever it has, being it 9xxx, 10xx or now 11xx generations.
  • barich - Friday, March 5, 2021 - link

    Yes, Intel probably would beat AMD with an imaginary all-new architecture on TSMC's 7nm process. Similarly, I would be a hell of a basketball player if I were a foot taller and had any motor skills.

    Here in reality, Intel has worse performance and worse efficiency. As a consumer, that's what matters to me. What Intel could do with a bunch of "ifs" is irrelevant. I haven't owned an AMD CPU since my Athlon 64 was replaced by a Core 2 Duo. But there's no way my new build this year isn't going to be AMD.
  • CiccioB - Saturday, March 6, 2021 - link

    Yes, what counts is the results, you are right.
    But by that I can't cry for a miracle when I see Zen 3 results as with a much more advanced PP it just can win over Skylake for a few % and all the real advantages it has is smaller power consumption due to the much better PP vs this one 6 years old.

    If you look at the real power consumption, that is not the one with AVX-512 tests where RKL disintegrates Zen for perf/W despite the high power requirements, you'll see that this chip is not that power hungry (though being more power hungry than Zen) and that does make me think that with a better PP this same architecture would be another thing completely, as are the 10nm Tiger Lake which however suffer the not so good power consumption at higher frequency required by desktop SKUs.

    As we are not that distant from finally having something decent that is not the 14nm PP, I will really not call my thought "imaginary". AMD will not be able to pass to 5nm so soon, and seen what this architecture can do, despote the 14nm PP, I think that the future is going to be more interesting that what you hope it ti be (that is, AMD keeps on figure it has better CPUs while not having them in the shelves but what counts for you are.. yes the results... and for these Intel is outselling AMD 5:1).
  • blppt - Saturday, March 6, 2021 - link

    "A PP like TSMC 7nm would bring a completely new architecture that would blow Zen away."

    Based on what, exactly? We've seen die shrinks before without amazing architectural advances from both Intel and AMD. You have an awful lot of confidence in something that doesn't exist.
  • CiccioB - Saturday, March 6, 2021 - link

    Based on the fact that with a 7nm PP AMD still struggles at beating Skylake architecture which is 6 years old and was born on... oh yes, 14nm.
  • schujj07 - Saturday, March 6, 2021 - link

    Zen2 was already faster than Skylake and its derivatives clock for clock by about 7%. While Comet Lake had higher single threaded performance than Zen2, it did so by throwing efficiency and power draw out the window and going for absolute performance. That made it such that Comet Lake could compete in ST applications but it still lost on MT applications against the same thread counted AMD CPUs. Going for absolute performance has been a double edged sword for Intel as the newer architectures hadn't been able to clock as high. Despite the higher IPCs of the newer architectures, absolute performance was no better than a wash due to 20% lower clock speeds.

    Zen3 now has absolute performance dominance over any Skylake architecture CPU. It doesn't "just" beat the older CPU, as in like 2% faster. It is upwards of 20% faster clock for clock and 10%+ faster in absolute ST performance.
  • blppt - Saturday, March 6, 2021 - link

    "Based on the fact that with a 7nm PP AMD still struggles at beating Skylake architecture which is 6 years old and was born on... oh yes, 14nm."

    Struggles? The slightly older AMD chip (5800X) beats the newest and greatest out of Intel, whilst consuming less power, AND hitting lower peak turbo speeds.

    That is complete domination. You could make the same argument about how Intel hadn't even made any significant gains over Sandy Bridge until Skylake, and that was *2* die shrinks.

Log in

Don't have an account? Sign up now