CPU Tests: SPEC

SPEC2017 and SPEC2006 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. SPEC2006 is deprecated in favor of 2017, but remains an interesting comparison point in our data. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0-svn350067-1~exp1+0~20181226174230.701~1.gbp6019f2 (trunk)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labelled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For each of the SPEC targets we are doing, SPEC2006 rate-1, SPEC2017 rate-1, and SPEC2017 rate-N, rather than publish all the separate test data in our reviews, we are going to condense it down into a few interesting data points. The full per-test values are in our benchmark database.

(9-0a) SPEC2006 1T Geomean Total(9-0b) SPEC2017 1T Geomean Total

Single thread is very much what we expected, with the consumer processors out in the lead and no real major differences between TR and TR Pro.

(9-0c) SPEC2017 nT Geomean Total

That changes when we move into full thread mode. The extra bandwidth of TR Pro is clear to see, even in the 32C/64T model. In this test we're using 128 GB of memory for all TR and TR Pro processors, and we're seeing a small bump when in 64C/64T mode, perhaps due to the increased memory cap/thread and memory bandwidth/thread as well. The 3990X 64C/128T run kept failing for an odd reason, so we do not have a score for that test.

CPU Tests: Synthetic CPU Tests: Microbenchmarks
Comments Locked

98 Comments

View All Comments

  • Qasar - Tuesday, July 27, 2021 - link

    sorry but that is not HEDT, workstation, sure. the last HEDT platform intel had was x299 and socket 2066
    socket 3647, is there server/workstation platform, but hey if you consider a US $3k cpu to be a HEDT processor, then that's your choice :-)
  • mode_13h - Monday, July 26, 2021 - link

    > at least amd HAS a HEDT cpu, when was the last one from intel ?

    Intel is doing an Ice Lake workstation platform. Not sure if HEDT will follow.
  • mode_13h - Sunday, July 25, 2021 - link

    > 7 days to August

    The rumor was that it would be *announced* at some point in August. It didn't say when, in August, but the rumored ship date wasn't until sometime in September. But it's just a rumor.
  • croc - Monday, July 26, 2021 - link

    MY point is that the BIOS updates usually happen about a month before the product announcement. Not to mention some benchmarks and other 'leaked' information. Y'know,,, Hype generation, direct from horsey's mouth. August announcement? Don't think so. Chagall? Possible, but would break convention, not that AMD really has any when it comes to code names...
  • mode_13h - Monday, July 26, 2021 - link

    > BIOS updates usually happen about a month before the product announcement.

    Before announcement or ship?

    > Hype generation

    Seems to me that it's not necessary, in this case. AMD will already have more demand than it can satisfy.
  • Qasar - Tuesday, July 27, 2021 - link

    " Not to mention some benchmarks and other 'leaked' information "
    considering how few leaks and info have come out about amd's products as of late until quite close to release, im not surprised there is little info out there about zen 3 TR

    " Hype generation "
    which amd doesnt need all that much, their products are more interesting then intels right now, intel needs the hype, not amd ;-)
  • Mikewind Dale - Monday, July 19, 2021 - link

    Given how much trouble Intel has had with their new process - even though Intel used to be the industry leader in fabrication - I suspect that if AMD had kept fabrication in-house, they'd be in serious trouble right.

    GlobalFoundries has also had trouble moving to a new, cutting-edge process. At the moment, they'd decided to stay one process behind TSMC, and cater to the portion of the market that doesn't need a cutting-edge process.
  • anakhizer - Monday, July 19, 2021 - link

    The article is excellent! However, the ordering of data in the tables is absolutely terrible.

    Please figure out how to sort the tables in a more logical manner like performance. As the tables are they are pretty much unreadable if you want to get the performance numbers with a glance.
  • kensiko - Monday, July 19, 2021 - link

    Performance wise, looking at all those graphs, the 5950x is such a great deal ! I really love my 5950x. I did love my TR1950x, it was not getting as hot at my 5950x. But no way I'm going back to Threadripper for just a home PC. Event at work I don't think we would get a Threadripper again, the Epyc gives what we want even if the frequency is a bit lower.
  • mode_13h - Tuesday, July 20, 2021 - link

    Threadripper still makes a lot of sense for people who have scalable workloads (or run lots of VMs) and who don't need the full memory bandwidth or PCIe lanes of EPYC or TR Pro.

    I personally wouldn't buy one, but they're popular for deep learning workstations and Linus Torvalds famously has one.

Log in

Don't have an account? Sign up now