CPU Tests: SPEC

Page by Andrei Frumusanu

SPEC2017 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0
clang version 7.0.1 (ssh://git@github.com/flang-compiler/flang-driver.git
 24bd54da5c41af04838bbe7b68f830840d47fc03)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labeled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For the new Cypress Cove based i7-11700K, we haven’t had quite the time to investigate the new AVX-512 instruction differences – since this is the first consumer desktop socketed CPU with the new ISA extensions it’s something we’ll revisit in the full review. Based on our testing on the server core counterparts however, it doesn’t make any noticeable differences in SPEC.

SPECint2017 Rate-1 Estimated Scores

In the SPECint2017 suite, we’re seeing the new i7-11700K able to surpass its desktop predecessors across the board in terms of performance. The biggest performance leap is found in 523.xalancbmk which consists of XML processing at a large +54.4% leap versus the 10700K.

The rest of the improvements range in the +0% to +15% range, with an average total geomean advantage of +15.5% versus the 10700K. The IPC advantage should be in the +18.5% range.

SPECfp2017 Rate-1 Estimated Scores

In the FP scores, there’s nothing standing out too much, with general even improvements across the board. The total improvement here is +19.6%, with the IPC improvement in the +22% range.

SPEC2017 Rate-1 Estimated Total

Although the new Cypress Cove cores in the 11700K do have good generational IPC improvements, that’s all compared to the quite old predecessor, meaning that for single-thread performance, the advancements aren’t enough to quite keep up with the latest Zen3 competition from AMD, or for that matter, the Firestorm cores in Apple’s new M1.

SPEC2017 Rate-N Estimated Total

More interesting are the multi-threaded SPEC results. Here, the new generation from Intel is showcasing a +5.8% and +16.2% performance improvement over its direct predecessor. Given the power draw increases we’ve seen this generation, those are rather unimpressive results, and actually represent a perf/W regression. AMD’s current 6-core 5600X actually is very near to the new 11700K, but consuming a fraction of the power.

CPU Tests: Encoding and Legacy/Web Gaming Tests: Deus Ex Mankind Divided
Comments Locked

541 Comments

View All Comments

  • Hifihedgehog - Friday, March 12, 2021 - link

    LOL. Fanboy delusion.

    First off, let's take a quick looksie at the Cinebench R20 results:
    https://www.hardwareluxx.de/index.php/artikel/hard...
    When switching from BIOS version 0402 to 0603, the 11700K's single-threaded performance actually DROPS from a score of 609 to 600. And its multicore performance is still less than the 10900K and the 5800X.
    Switching gears, the games are no less unflattering:
    https://www.hardwareluxx.de/index.php/artikel/hard...
    The 11700K there, regardless of which of the two BIOS releases it uses, often loses to the 10900K and Ryzen 5000 series. It loses to the Ryzen 5000 series and 10900K in THREE out of the four games: The Division 2, Metro Exodus, The Shadow of the Tomb Raider.

    In short: dude, what are you smoking?
  • Technobile - Friday, March 12, 2021 - link

    The 10700K costs a pittance at the minute, and after the final bios and microcode 11700K will be around 15 to 20% faster than it. Both a bargain when the only other option is dealing with the 'quirks' (to be kind) of an AMD system
  • Qasar - Friday, March 12, 2021 - link

    " The 10700K costs a pittance at the minute, and after the final bios and microcode 11700K will be around 15 to 20% faster than it " i will believe that, when i see it, honestly, that is hopeful thinking.
    " Both a bargain when the only other option is dealing with the 'quirks' (to be kind) of an AMD system " and intel has had its own quirks over the years.
  • dsplover - Saturday, March 13, 2021 - link

    They’re just doing this to give the impression they’re relevant. It’s safe to assume they’ll retake some market share but AMD took servers, laptop and desktop sectors by storm while Intel keeps moving old designs out the door.

    AMD 5750G, if it exists will render Intel designs useless this summer, while Intel struggles to get Alder Lake up and running.

    Motherboard manufacturers will get tired of chasing new sockets after AM5 comes out.

    What happened to these guys? It’s embarrassing and I’m an i7 fan boy..
  • GeoffreyA - Saturday, March 13, 2021 - link

    "What happened to these guys?"

    My guess would be: complacency, underestimating the enemy, putting eggs in too many dead-end baskets, and management that made a mess of excellent engineering talent.
  • CiccioB - Sunday, March 14, 2021 - link

    Motherboard manufacturers will get tired of chasing new sockets after AM5 comes out.

    When reading this I think some of you just ended their school yesterday (with poor results) and just came here to say the first thing they think it is pro AMD. Just to give a (poor) contribution to what they think is an easy (for everyone) task as beating a dead horse (Intel).
    I may shock you if I say that that "chasing designs" effort is the secret trick for motherboard producer too... surprise surprise.. MAKE MONEY!
    So they do not get tired to do anything if this means selling more motherboards, and this just happens if you have to change your motherboards every couple of generations.
    And I may shock you even more if I say you that those that make an upgrade using the same motherboard is just a so small number that no motherboard producer is really interested in supporting.
    Usually when you change your CPU you just do not want only it to go a little faster but you want also the new technological improvements that meantime have been created, from faster bus, new and more connectors (M2 vs SATA), faster USBs, Thunderbolts, better memory support and such.
    And this doesn't come if you do not also change the motherboard.
    And to have and propose a better motherboard to sell, guess what? Yes, motherboard producer have to play the "chasing designs" game.
  • rfxcasey - Thursday, March 18, 2021 - link

    Yeah "useless", please, all of this is splitting hairs, none of these processors are even close to being "useless".
  • rfxcasey - Wednesday, March 17, 2021 - link

    To me, looks like the i7 10700k holds it's own against the 5800x in gaming performance and is much cheaper.
  • rfxcasey - Thursday, March 18, 2021 - link

    i7 10700K actually beating the 5800x in many game benchmarks. I don't have a preference between AMD and Intel generally, but the i7 10700K is a great gaming processor, Intel did seem to make an embarrassing move with the 11th Gen, but for the cost, the 10700k a top, possibly THE top gaming CPU.
  • quadibloc - Thursday, March 18, 2021 - link

    When an official review comes in, with all the details, things may look a little better. But even now, I see one thing that's being overlooked. Since these chips have AVX-512, where that can be used, that will double their performance compared to processors that only have AVX-256. Except, of course, for the necessary slowdown for thermal reasons. So on workloads that involve a lot of AVX-512, they should really shine instead of being as terribly lackluster as they appear when that isn't taken into account.

Log in

Don't have an account? Sign up now