CPU Tests: SPEC

Page by Andrei Frumusanu

SPEC2017 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0
clang version 7.0.1 (ssh://git@github.com/flang-compiler/flang-driver.git
 24bd54da5c41af04838bbe7b68f830840d47fc03)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labeled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For the new Cypress Cove based i7-11700K, we haven’t had quite the time to investigate the new AVX-512 instruction differences – since this is the first consumer desktop socketed CPU with the new ISA extensions it’s something we’ll revisit in the full review. Based on our testing on the server core counterparts however, it doesn’t make any noticeable differences in SPEC.

SPECint2017 Rate-1 Estimated Scores

In the SPECint2017 suite, we’re seeing the new i7-11700K able to surpass its desktop predecessors across the board in terms of performance. The biggest performance leap is found in 523.xalancbmk which consists of XML processing at a large +54.4% leap versus the 10700K.

The rest of the improvements range in the +0% to +15% range, with an average total geomean advantage of +15.5% versus the 10700K. The IPC advantage should be in the +18.5% range.

SPECfp2017 Rate-1 Estimated Scores

In the FP scores, there’s nothing standing out too much, with general even improvements across the board. The total improvement here is +19.6%, with the IPC improvement in the +22% range.

SPEC2017 Rate-1 Estimated Total

Although the new Cypress Cove cores in the 11700K do have good generational IPC improvements, that’s all compared to the quite old predecessor, meaning that for single-thread performance, the advancements aren’t enough to quite keep up with the latest Zen3 competition from AMD, or for that matter, the Firestorm cores in Apple’s new M1.

SPEC2017 Rate-N Estimated Total

More interesting are the multi-threaded SPEC results. Here, the new generation from Intel is showcasing a +5.8% and +16.2% performance improvement over its direct predecessor. Given the power draw increases we’ve seen this generation, those are rather unimpressive results, and actually represent a perf/W regression. AMD’s current 6-core 5600X actually is very near to the new 11700K, but consuming a fraction of the power.

CPU Tests: Encoding and Legacy/Web Gaming Tests: Deus Ex Mankind Divided
Comments Locked

541 Comments

View All Comments

  • ottonis - Saturday, March 6, 2021 - link

    How is Ryzen 5800x AMD's "best of the best"?
    It clearly is not but just about somewhere in the middle. If you want AMD's best consumer CPU, you gonna look at the 5950x.
    From the perspective of AMD, completely outsourcing manufacturing was the only way to reap the benefits of latest and greatest process nodes.
    Now, that everybody else, incl. the automobile industry, the consoles, Apple, and even Intel are booking production capacities at TMSC, has certainly contributed to Reaching capacity limits and thus to AMD CPU shortages.
    It is predicted that shortages will be mostly (hopefully) sorted out by this summer.
    But yes, now that AMD are earning quite a lot of money they should buy some TSMC stock and try to partner up, getting more production capacity in the future.
  • Fulljack - Saturday, March 6, 2021 - link

    did you just forgetting why AMD spin-off it's fab on the first place?

    we don't know exactly how much AMD put out their desktop chips from TSMC plant, and as far as we know, the market are still growing in size. while AMD market share are still a long way from reaching 50%, they still have their processor sold out all over the place.

    honestly I don't get what you're trying to say here.
  • shady28 - Saturday, March 6, 2021 - link

    Actually we do know, they made ~1M Zen 3 chips in Q4. 140M PCs were shipped. Based on their market share, about 3% of the chips AMD shipped in Q4 were Zen 3.

    Source:https://wccftech.com/amd-shipped-nearly-1-million-...
  • inighthawki - Monday, March 8, 2021 - link

    5800X = "Best of the best"
    11700K = "medium level SKU"

    Your bias is showing.
  • Bluetooth - Saturday, March 6, 2021 - link

    Does any one know the details of Intels 10 nm node problems. Any article discussing that in technical details?
  • dihartnell - Thursday, March 11, 2021 - link

    I think adored tv did a few articles, videos on this. As I understand it thier main issues are they use a monolithic die (everything on on a single die) that gets harder to make as the process shrinks, IE more dies have defects... AMD got around this by going chiplet, lots of small dies which meant more of them are good. Until Intel chnages to chiplet or they find a way to improve the manufacturing process to lower the defect rate then they will struggle.
  • Santoval - Saturday, March 6, 2021 - link

    Class action for what, excessive power bills? :) Imagine using this during a heatwave in an airless room. Since it doubles as a heater you would need to have the AC on all the time. If you have no AC there will be a competition between who dies first, you or the processor? ^.^
  • sabot00 - Friday, March 5, 2021 - link

    Indeed! I have been reading nothing but wccftech and Tom's leaks. Absolutely amazing surprise this Friday night while searching Rocket Lake
  • Beaver M. - Saturday, March 6, 2021 - link

    I would be very cautious testing or believing results with the Z590 platforms long before Rocket Lakes official release.
    Ive tested 3 of those boards with my Comet Lake (Asus, MSI and ASRock) and they all had pretty bad BIOS versions still, with PCIe/IO performance being low in SSD 4K benchmarks and getting weird frame time stutters from time to time (only noticeable when actually playing or looking at a realtime graph). Not to mention Intels drivers are still bad as well.
    On none of them even very basic features like the sleep state worked!
    Comparing this review with user benchmarks in German forums shows huge differences, so theres not much to add to this.

    That said, I have to laugh when Americans or people from other countries with cheap power complain about the power draw.
    And seeing fanboys downplay the performance of AVX and ignoring that it was always power hungry, even 6 years ago, is another obvious thing. Without it RKL actually runs pretty cool for being ancient 14nm.

    And of course I love the geniuses who still think that you cant use the iGPU without it being connected to a monitor, or dont know about the new power saving/GPU switching feature. Not that this article didnt fail at pretty much everything, incl. explaining things like that.
  • Gigaplex - Saturday, March 6, 2021 - link

    "That said, I have to laugh when Americans or people from other countries with cheap power complain about the power draw."
    Electricity is expensive where I live, but that's not why I want low power consumption. The more power it consumes, the louder the cooling solution will be. That's why my last system was Intel (Ivy Bridge) and my current system is AMD (Zen 3).

Log in

Don't have an account? Sign up now