CPU Tests: SPEC

Page by Andrei Frumusanu

SPEC2017 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0
clang version 7.0.1 (ssh://git@github.com/flang-compiler/flang-driver.git
 24bd54da5c41af04838bbe7b68f830840d47fc03)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labeled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For the new Cypress Cove based i7-11700K, we haven’t had quite the time to investigate the new AVX-512 instruction differences – since this is the first consumer desktop socketed CPU with the new ISA extensions it’s something we’ll revisit in the full review. Based on our testing on the server core counterparts however, it doesn’t make any noticeable differences in SPEC.

SPECint2017 Rate-1 Estimated Scores

In the SPECint2017 suite, we’re seeing the new i7-11700K able to surpass its desktop predecessors across the board in terms of performance. The biggest performance leap is found in 523.xalancbmk which consists of XML processing at a large +54.4% leap versus the 10700K.

The rest of the improvements range in the +0% to +15% range, with an average total geomean advantage of +15.5% versus the 10700K. The IPC advantage should be in the +18.5% range.

SPECfp2017 Rate-1 Estimated Scores

In the FP scores, there’s nothing standing out too much, with general even improvements across the board. The total improvement here is +19.6%, with the IPC improvement in the +22% range.

SPEC2017 Rate-1 Estimated Total

Although the new Cypress Cove cores in the 11700K do have good generational IPC improvements, that’s all compared to the quite old predecessor, meaning that for single-thread performance, the advancements aren’t enough to quite keep up with the latest Zen3 competition from AMD, or for that matter, the Firestorm cores in Apple’s new M1.

SPEC2017 Rate-N Estimated Total

More interesting are the multi-threaded SPEC results. Here, the new generation from Intel is showcasing a +5.8% and +16.2% performance improvement over its direct predecessor. Given the power draw increases we’ve seen this generation, those are rather unimpressive results, and actually represent a perf/W regression. AMD’s current 6-core 5600X actually is very near to the new 11700K, but consuming a fraction of the power.

CPU Tests: Encoding and Legacy/Web Gaming Tests: Deus Ex Mankind Divided
Comments Locked

541 Comments

View All Comments

  • lmcd - Friday, March 5, 2021 - link

    As an Intel fan, this is abysmal. This is literally only good for an i3 or i5 SKU, for its upgraded iGPU. That's it.
  • Slash3 - Friday, March 5, 2021 - link

    The 11th gen i3 SKUs will not feature the new Xe core - they are still based on Comet Lake.
  • lmcd - Friday, March 5, 2021 - link

    That makes this even sadder. This port was actually worthless. An i5 quad core might be manageable by a cooler that fits in an average case, but that's about it.

    Would've been better off releasing Tiger Lake 35W processors on an LGA package.
  • Slash3 - Friday, March 5, 2021 - link

    The 6C/12T i5-11500 should be "fine," as it has the same 32CU Xe iGPU as the higher end parts. The part below it, the i5-11400, is also 6C/12T but has a cut down 24CU Xe core.

    The Xe upgrade is a nice change, at about a 33% uplift over the previous "Gen11" iGPU, but it's still just what I'd call "passable" for light gaming. Anything above 1080p and you'll want a discrete GPU for the best experience, and that goes for both teams.
  • lmcd - Friday, March 5, 2021 - link

    I understand what you're saying but I still think the 6-core i5 is going to draw a boatload of power.

    Obviously for gaming a dGPU is preferable. In my experience with AMD's Ryzen 2400G, though, it isn't actually too stable with medium resolution monitor configurations (think 1080p + 1440p) and isn't all that well supported by drivers.

    On the Intel side, uplift over Gen11 is cool but uplift over Gen9 is where it gets noticeable, and important. Gen9 was fine when it came out but that was quite some time ago. Shows its age when too many apps want GPU acceleration with that same multi-monitor setup I described.
  • lmcd - Friday, March 5, 2021 - link

    Sorry, didn't complete my thought. Anyway, point is that a decent % of Intel's sales are for business desktops (maybe not right now, but, ya know, offices might open up sometime this calendar year, before Alder Lake ships in volume). Rocket Lake i5 would be perfect, if its real-world power consumption wasn't out the roof. Good enough graphics for multi-monitor, fine performance elsewhere.

    I guess that makes me wonder what performance would be if limited to a 65W avg/95W peak thermal output. That's about what those small Dell towers can handle.
  • Slash3 - Friday, March 5, 2021 - link

    Yep, I'm with you.

    I honestly don't expect the i5 models to have the same outlandish power usage characteristics as the reviewed i7, but will definitely be reading any day one reviews that may pop up. The existing i5-10400 is a pretty compelling product (and priced well), and if the 11400 or 11500 can manage to fit into the same ~100W PL2 envelope I think it'll find a home in a lot of desktops. For OEMs, they're sometimes board limited to the 65W PL1 via a BIOS option, and I'd expect that to continue with the 11th gen versions.

    If they price the 11400 close to the 10400, it'll be a solid choice.
  • Makaveli - Friday, March 5, 2021 - link

    The Turbo for the 5800X should be 4850 that is what I see at stock. So the table is off by 50mhz
  • SaturnusDK - Saturday, March 6, 2021 - link

    Not really. AMD changed the way they report boost frequency numbers. Before it was "up to xxxxMHz boost", now it's the more wordy "given adequate cooling the boost frequency is _at least_ xxxxMHz". This change was driven by the stick they got for the 3000-series only very rarely hitting the listed boost frequency. Now you can generally assume to get 50-100MHz _above_ the listed boost frequency if you have a half-decent cooler.
  • Marlin1975 - Friday, March 5, 2021 - link

    WTF!!! 291 watts?!?!?

    Intel REALLY needs to step up is manufacture game. Designs seem ok but the nodes are killing them.

Log in

Don't have an account? Sign up now