CPU Tests: SPEC

Page by Andrei Frumusanu

SPEC2017 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0
clang version 7.0.1 (ssh://git@github.com/flang-compiler/flang-driver.git
 24bd54da5c41af04838bbe7b68f830840d47fc03)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labeled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For the new Cypress Cove based i7-11700K, we haven’t had quite the time to investigate the new AVX-512 instruction differences – since this is the first consumer desktop socketed CPU with the new ISA extensions it’s something we’ll revisit in the full review. Based on our testing on the server core counterparts however, it doesn’t make any noticeable differences in SPEC.

SPECint2017 Rate-1 Estimated Scores

In the SPECint2017 suite, we’re seeing the new i7-11700K able to surpass its desktop predecessors across the board in terms of performance. The biggest performance leap is found in 523.xalancbmk which consists of XML processing at a large +54.4% leap versus the 10700K.

The rest of the improvements range in the +0% to +15% range, with an average total geomean advantage of +15.5% versus the 10700K. The IPC advantage should be in the +18.5% range.

SPECfp2017 Rate-1 Estimated Scores

In the FP scores, there’s nothing standing out too much, with general even improvements across the board. The total improvement here is +19.6%, with the IPC improvement in the +22% range.

SPEC2017 Rate-1 Estimated Total

Although the new Cypress Cove cores in the 11700K do have good generational IPC improvements, that’s all compared to the quite old predecessor, meaning that for single-thread performance, the advancements aren’t enough to quite keep up with the latest Zen3 competition from AMD, or for that matter, the Firestorm cores in Apple’s new M1.

SPEC2017 Rate-N Estimated Total

More interesting are the multi-threaded SPEC results. Here, the new generation from Intel is showcasing a +5.8% and +16.2% performance improvement over its direct predecessor. Given the power draw increases we’ve seen this generation, those are rather unimpressive results, and actually represent a perf/W regression. AMD’s current 6-core 5600X actually is very near to the new 11700K, but consuming a fraction of the power.

CPU Tests: Encoding and Legacy/Web Gaming Tests: Deus Ex Mankind Divided
Comments Locked

541 Comments

View All Comments

  • jacropolis - Sunday, March 7, 2021 - link

    The 10700K is regularly $279 at Micro Center and isn't a bad buy for that price at all.
  • Jadi - Sunday, March 7, 2021 - link

    350 Dollar you want a 10700k and comparing it with a 5800X?
    Why?
    The 5800X don t have a gpu, and the 10700ks was under 320 euros allways!
    Now the 10700K not KS cost 316 euros!
    Hot a 14nm Chip this is the advantage its only 60 c and a 5800X comes easy over 85 c and more.

    More expansiv a 5800X cost 440 euros! A 5600X cost 350 euros, more then the 10700k!
    Slower its a K CPU and can boost it up. When you OC the 5800X you have less fps.

    Why you think it needs more power? Becouse the 40 sec boost? That is all and under AVX512 is it the only cpu that have it.
    Games and other stuff its the same power, And 5800X dosen t stop at 105 Watt it goes like 118 watts. Thats are very close too 125 watts and more is only a limit time!
  • Spunjji - Monday, March 8, 2021 - link

    @Jadi - that last bit of your post makes no sense. The 10700K and 11700K objectively require more power than the 5800X to do the same amount of work; the 5800X only hits 118 Watts under the same conditions that the other two approach 200 Watts. If you're comparing power draw during gaming, the 5800X uses less power than those two - roughly 25W less than the 10700K.

    Comparing temperatures is daft too. 85 degrees is way under the CPU's junction temperature, so it's not a problem to runa cPU at that temp - and it's objectively easier to keep an AMD CPU at that temperature than to keep an Intel chip at 60 degrees, because your cooler isn't dissipating as much heat.
  • TrueJessNarmo - Tuesday, March 9, 2021 - link

    10700k is $320 on amazon and 9900k is $250 at microcenter.

    At this point 5900x is best for serious users and 9900k/10700k is for gamers. 5600x and 5800x are overpriced for what it is, but 5900x is a sweet deal at $550
  • rfxcasey - Wednesday, March 17, 2021 - link

    I got my 10700K for 330 dollars new, I've seen them go for as low as 300.
  • Makste - Saturday, March 6, 2021 - link

    With gaming it's not the power consumption that's the problem here, it is the gaming performance in these benchmarks against comet lake obviously due to a higher latency.
    Zen 3's efforts have been put into perspective here, those engineers did quite a commendable job.
  • Bfree4me - Sunday, March 7, 2021 - link

    Agreed! The Zen architecture is top shelf. But it took far too long. My last AMD Cpu was an Athlon 1800 because at that time AMD was the value leader and Intel was the performance king for a price. At the time came where I needed performance over value.
    So I have been Intel since then. Time to start looking towards Zen 🤔
  • Timoo - Monday, March 8, 2021 - link

    If I may be so free: "Took way too long": I agree.
    I just keep in mind that it took them way too long, because Intel managed to create some very shady deals with retailers, effectively pushing AMD out of the market and to the brink of bankruptcy. So, for years AMD has been struggling to stay alive.

    Agreed, buying ATi didn't help either. But without those shady deals they might have pulled it off. From 2006 to -say- 2015 AMD was only struggling to stay alive. I believe they had some huge cash injection at the time, together with getting one of the best CPU designers in the world (what's his name?) back in the house.

    So, in my opinion, AMD is forgiven for their 10-year "heart-attack".
  • GeoffreyA - Monday, March 8, 2021 - link

    I also believe it was because the K10, or Phenom, while being faster than the Athlon 64, was not aggressive enough. It had already been designed before Core 2 Duo came out, and by then, Intel was ahead of them. Who knows, they might have narrowed the gap if they had persisted with the K10 design, widening everything till it matched/surpassed Nehalem, but they shot too far, going for Bulldozer, which was supposed to turn the tables round but sank them further in the ditch.
  • TheinsanegamerN - Monday, March 8, 2021 - link

    And yet AMD was having trouble supplying enough chips to the limited market they did have. Much like today. OEMs are not going to sign up with a company that cant consistently supply CPUs.

    Dont forget the billiosn they poured into making GF and pissing off TSMC in the process.

Log in

Don't have an account? Sign up now