CPU Tests: SPEC

Page by Andrei Frumusanu

SPEC2017 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0
clang version 7.0.1 (ssh://git@github.com/flang-compiler/flang-driver.git
 24bd54da5c41af04838bbe7b68f830840d47fc03)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labeled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For the new Cypress Cove based i7-11700K, we haven’t had quite the time to investigate the new AVX-512 instruction differences – since this is the first consumer desktop socketed CPU with the new ISA extensions it’s something we’ll revisit in the full review. Based on our testing on the server core counterparts however, it doesn’t make any noticeable differences in SPEC.

SPECint2017 Rate-1 Estimated Scores

In the SPECint2017 suite, we’re seeing the new i7-11700K able to surpass its desktop predecessors across the board in terms of performance. The biggest performance leap is found in 523.xalancbmk which consists of XML processing at a large +54.4% leap versus the 10700K.

The rest of the improvements range in the +0% to +15% range, with an average total geomean advantage of +15.5% versus the 10700K. The IPC advantage should be in the +18.5% range.

SPECfp2017 Rate-1 Estimated Scores

In the FP scores, there’s nothing standing out too much, with general even improvements across the board. The total improvement here is +19.6%, with the IPC improvement in the +22% range.

SPEC2017 Rate-1 Estimated Total

Although the new Cypress Cove cores in the 11700K do have good generational IPC improvements, that’s all compared to the quite old predecessor, meaning that for single-thread performance, the advancements aren’t enough to quite keep up with the latest Zen3 competition from AMD, or for that matter, the Firestorm cores in Apple’s new M1.

SPEC2017 Rate-N Estimated Total

More interesting are the multi-threaded SPEC results. Here, the new generation from Intel is showcasing a +5.8% and +16.2% performance improvement over its direct predecessor. Given the power draw increases we’ve seen this generation, those are rather unimpressive results, and actually represent a perf/W regression. AMD’s current 6-core 5600X actually is very near to the new 11700K, but consuming a fraction of the power.

CPU Tests: Encoding and Legacy/Web Gaming Tests: Deus Ex Mankind Divided
Comments Locked

541 Comments

View All Comments

  • terroradagio - Friday, March 5, 2021 - link

    I'd be saying this if they did with an AMD part too. This is something I would expect from someone unknown guy doing reviews, not Anandtech. Extremely disappointing they are going for the click bait.
  • Otritus - Saturday, March 6, 2021 - link

    We used to have performance previews all the time back in the day to let people have an idea of what is coming and if it is worth it to wait for the next generation. Consider this to be a performance preview, meaning final performance will be higher. In general workloads rocket lake is 10-15 % faster while being reasonable with power. In gaming it's trash, but that could be a consequence of the non-finalized nature of the product.

    By the time someone can reasonably buy this product it will be April/May. If I need a pc now, the performance uplift isn't that high to make me have buyers remorse. If I can wait, I can get a comet lake chip at probably a great discount, or if rocket lake's gaming is fixed, get that instead.

    Performance previews like this are excellent journalism because the public is informed of things that are otherwise concealed. My only gripe with the article is calling it a "review" instead of "early review" or "preview" because someone may stumble on this article and mistakenly believe this is how the product will be shipped.
  • sonny73n - Saturday, March 6, 2021 - link

    This is a REVIEW! Not early review or preview or any excuse you're making. You've lost touch with reality if you hope for a higher performance on official release date. If Intel haven't released these CPUs (before their announced release date), no way in hell Anandtech or anyone would be able to purchase them.
  • Spunjji - Saturday, March 6, 2021 - link

    I'm amused by how many people have stumbled upon this and are mistakenly assuming that performance will be significantly better when it's shipped. This was a CPU bought at retail running on a retail motherboard. If they do noticeably improve performance (and it's a big if) consumers will have to install updates themselves to get it.
  • Makste - Saturday, March 6, 2021 - link

    This is not an engineering sample. If hundreds were being sold by a retailer then that is what Intel was intending to sell to its clients. If you are not satisfied with the performance don't blame it on Anandtech
  • shady28 - Saturday, March 6, 2021 - link

    This is labelled review, not performance preview.

    Cutress is officially a hack now. I'll wait for real reviews on systems that officially and properly support the chip.
  • schujj07 - Saturday, March 6, 2021 - link

    It was a review. This was a retail CPU running on a motherboard with a BIOS that supports RKL. Just because RKL doesn't look good isn't surprising.
  • shady28 - Sunday, March 7, 2021 - link

    On a ++BETA++ BIOS.
  • schujj07 - Monday, March 8, 2021 - link

    Unless you have something like the X570 and Zen2 being released at the same time, quite often it is a Beta BIOS that gives initial support for new chips. Eventually with further validation that Beta BIOS becomes official. Perfect example of this is in October 2019 I was installing vSphere 6.7 onto some servers. With the initial BIOS release I was getting the Purple Screen of Death. The only BIOS that was available was a Beta BIOS and on that vSphere installed just fine. Push forward a month later and the official supported BIOS by VMware for that server was the Beta BIOS for vSphere 6.7. It took another 6 months before a non-beta BIOS was available for that server as well. Hence just because it is Beta doesn't mean it is bad. Things in Beta testing generally have full features and performance just hasn't been validated long enough to become the official release.
  • Qasar - Sunday, March 7, 2021 - link

    like that will change much, if anything shady28.

Log in

Don't have an account? Sign up now