CPU Tests: SPEC

Page by Andrei Frumusanu

SPEC2017 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0
clang version 7.0.1 (ssh://git@github.com/flang-compiler/flang-driver.git
 24bd54da5c41af04838bbe7b68f830840d47fc03)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labeled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For the new Cypress Cove based i7-11700K, we haven’t had quite the time to investigate the new AVX-512 instruction differences – since this is the first consumer desktop socketed CPU with the new ISA extensions it’s something we’ll revisit in the full review. Based on our testing on the server core counterparts however, it doesn’t make any noticeable differences in SPEC.

SPECint2017 Rate-1 Estimated Scores

In the SPECint2017 suite, we’re seeing the new i7-11700K able to surpass its desktop predecessors across the board in terms of performance. The biggest performance leap is found in 523.xalancbmk which consists of XML processing at a large +54.4% leap versus the 10700K.

The rest of the improvements range in the +0% to +15% range, with an average total geomean advantage of +15.5% versus the 10700K. The IPC advantage should be in the +18.5% range.

SPECfp2017 Rate-1 Estimated Scores

In the FP scores, there’s nothing standing out too much, with general even improvements across the board. The total improvement here is +19.6%, with the IPC improvement in the +22% range.

SPEC2017 Rate-1 Estimated Total

Although the new Cypress Cove cores in the 11700K do have good generational IPC improvements, that’s all compared to the quite old predecessor, meaning that for single-thread performance, the advancements aren’t enough to quite keep up with the latest Zen3 competition from AMD, or for that matter, the Firestorm cores in Apple’s new M1.

SPEC2017 Rate-N Estimated Total

More interesting are the multi-threaded SPEC results. Here, the new generation from Intel is showcasing a +5.8% and +16.2% performance improvement over its direct predecessor. Given the power draw increases we’ve seen this generation, those are rather unimpressive results, and actually represent a perf/W regression. AMD’s current 6-core 5600X actually is very near to the new 11700K, but consuming a fraction of the power.

CPU Tests: Encoding and Legacy/Web Gaming Tests: Deus Ex Mankind Divided
Comments Locked

541 Comments

View All Comments

  • Franseven - Monday, March 8, 2021 - link

    I know is a strange request, but i would like to know the iris integrated graphics benchmarks since i'm using my old 2080 ti for mining and i'm playing Minecraft and simple games with my integrated uhd630 of my 9700k, and unfortunately 5900x does not have integrated graphics, so i would like to know 11700k and 11900k perf with that, i have seen mobile benchmarks but as you know, is not the same thing, would like to see quality gaming benchmark as always, from you. thanks
  • kmmatney - Monday, March 8, 2021 - link

    Would also be interested in this. I sold my 2070 Super - I owned it for a year, and sold it for what I paid (so free card for a year). The idea was to buy a 30X0 card with that money. That didn't happen, so lately I've just been playing Minecraft and older games on an old GTX 460. I'm curious about how the Xe graphics compares - with current prices on Ebay, the graphics along can add about $60 worth of value to the cpu.
  • terroradagio - Monday, March 8, 2021 - link

    ASUS just released another BIOS update with Rocket Lake enhancements. Probably more to come closer to the release too. This is why you don't post your review 3 weeks early.
  • Everett F Sargent - Monday, March 8, 2021 - link

    Like maybe an AVX-512 down clocking offset? Either Intel released their Rocket Engine a quarter too early or no amount of BIOS tweaking can do what you think it can do, at this, or any, point in time.

    From this review "Looking at our data, the all-core turbo under AVX-512 is 4.6 GHz, sometimes dipping to 4.5 GHz. Ouch. ... Our temperature graph looks quite drastic. Within a second of running AVX-512 code, we are in the high 90ºC, or in some cases, 100ºC. Our temperatures peak at 104ºC ... "

    So already thermal throttling at Intel's promised 4.6 all core frequency using AVX-512. Makes you wonder what it takes to significantly OC this CPU. Which, you know, has barely been mentioned here in the comments section, OC'ing the damn thing, north or south of 300W or ~300W ...
    https://i.imgur.com/8BEsGVo.png
  • terroradagio - Tuesday, March 9, 2021 - link

    I guess you missed also the spot where normal AVX used less power than the 9900k. The vast majority don't care about AVX-512. It is just there so Intel can say it is. People who buy Rocket Lake will be interested because of gaming and there will probably be more stock than 7nm products from AMD.
  • Qasar - Tuesday, March 9, 2021 - link

    wow. really ? one test ( of a few) where intel was faster, and used less power ? big deal. over all rocket lake, looks to be a joke.
    " People who buy Rocket Lake will be interested because of gaming " wrong, i know a few peope who are not even looking at intel, and are just waiting for zen 3 to be available, and this is for gaming and non gaming usage.
  • terroradagio - Tuesday, March 9, 2021 - link

    I pointed out facts, and you are cherry picking one very selective AVX 512 test. Go away fanboy.
  • Qasar - Tuesday, March 9, 2021 - link

    like you your self have been doing ? and showing how much you love intel?
    hello pot meet kettle.
  • Everett F Sargent - Tuesday, March 9, 2021 - link

    Yes, a 5.0GHz (all core boost clock) at 231.49W for the i9-9900KS versus a 4.6GHz (all core boost clock) at 224.56W for the i7-11700K. Conclusion? The i7-11700K runs 20-25W higher at the same all core boost frequency (4.6-5.0GHz). The i7-11700K wins at test duration though (by a similar margin as the inverse of the power ratio). The CPU energy used is about the same for both.
  • amanpatel - Monday, March 8, 2021 - link

    Few questions:

    1) Why is apple silicon or ARM equivalents not part of the benchmarks?
    2) Why are so many CPU benchmarks needed, especially if they don't tell anything significant about them.
    3) I'm not a huge gamer, but I also don't understand the point of so many gaming benchmarks for a CPU review.

    Perhaps I'm the wrong audience member here, but it does seem a whole lot of charts that roughly say the same thing!

Log in

Don't have an account? Sign up now