CPU Tests: SPEC

SPEC2017 and SPEC2006 is a series of standardized tests used to probe the overall performance between different systems, different architectures, different microarchitectures, and setups. The code has to be compiled, and then the results can be submitted to an online database for comparison. It covers a range of integer and floating point workloads, and can be very optimized for each CPU, so it is important to check how the benchmarks are being compiled and run.

We run the tests in a harness built through Windows Subsystem for Linux, developed by our own Andrei Frumusanu. WSL has some odd quirks, with one test not running due to a WSL fixed stack size, but for like-for-like testing is good enough. SPEC2006 is deprecated in favor of 2017, but remains an interesting comparison point in our data. Because our scores aren’t official submissions, as per SPEC guidelines we have to declare them as internal estimates from our part.

For compilers, we use LLVM both for C/C++ and Fortan tests, and for Fortran we’re using the Flang compiler. The rationale of using LLVM over GCC is better cross-platform comparisons to platforms that have only have LLVM support and future articles where we’ll investigate this aspect more. We’re not considering closed-sourced compilers such as MSVC or ICC.

clang version 10.0.0-svn350067-1~exp1+0~20181226174230.701~1.gbp6019f2 (trunk)

-Ofast -fomit-frame-pointer
-march=x86-64
-mtune=core-avx2
-mfma -mavx -mavx2

Our compiler flags are straightforward, with basic –Ofast and relevant ISA switches to allow for AVX2 instructions. We decided to build our SPEC binaries on AVX2, which puts a limit on Haswell as how old we can go before the testing will fall over. This also means we don’t have AVX512 binaries, primarily because in order to get the best performance, the AVX-512 intrinsic should be packed by a proper expert, as with our AVX-512 benchmark. All of the major vendors, AMD, Intel, and Arm, all support the way in which we are testing SPEC.

To note, the requirements for the SPEC licence state that any benchmark results from SPEC have to be labelled ‘estimated’ until they are verified on the SPEC website as a meaningful representation of the expected performance. This is most often done by the big companies and OEMs to showcase performance to customers, however is quite over the top for what we do as reviewers.

For each of the SPEC targets we are doing, SPEC2006 rate-1, SPEC2017 rate-1, and SPEC2017 rate-N, rather than publish all the separate test data in our reviews, we are going to condense it down into a few interesting data points. The full per-test values are in our benchmark database.

(9-0a) SPEC2006 1T Geomean Total(9-0b) SPEC2017 1T Geomean Total

Single thread is very much what we expected, with the consumer processors out in the lead and no real major differences between TR and TR Pro.

(9-0c) SPEC2017 nT Geomean Total

That changes when we move into full thread mode. The extra bandwidth of TR Pro is clear to see, even in the 32C/64T model. In this test we're using 128 GB of memory for all TR and TR Pro processors, and we're seeing a small bump when in 64C/64T mode, perhaps due to the increased memory cap/thread and memory bandwidth/thread as well. The 3990X 64C/128T run kept failing for an odd reason, so we do not have a score for that test.

CPU Tests: Synthetic CPU Tests: Microbenchmarks
Comments Locked

98 Comments

View All Comments

  • Threska - Friday, July 16, 2021 - link

    One could have more than one.

    https://www.reddit.com/r/hardware/comments/mc8j2c/...

    Especially depending upon the memory type.

    https://semiengineering.com/what-designers-need-to...
  • croc - Monday, July 19, 2021 - link

    The first year anniversary of the Threadripper Pro! And a timely review to celebrate it! Not the CPU, the release... Because that might be the last Threadripper we see for quite some time....

    I get the impression that AMD has been in the position of runer up for so long now that they don't know how to capitalize on a lead. Either that or by selling off their fab they have lost touch with how to design for fabrication, which might explain their failure to deliver product at the 7NM node. Unfilled Epycs, no Zen 3 Threadrippers...
  • mode_13h - Monday, July 19, 2021 - link

    What a croc!
    ; )

    > that might be the last Threadripper we see for quite some time....

    The announcement for Chagall is rumored to be coming in August, with volume shipping in September. Have you heard otherwise?

    > by selling off their fab they have lost touch with how to design for fabrication

    Huh? Why do you think the issue is design-related?

    And if we're contemplating counter-factuals, then let's not lose sight of the fact that GF is still on 12 nm. And if AMD still owned them, we should also consider whether the whole enterprise would still be doing business, at all.

    > which might explain their failure to deliver product at the 7NM node.

    Or maybe they're in the same boat as everyone else, facing higher demand and restricted supply?
  • croc - Tuesday, July 20, 2021 - link

    Lemmee see.... Wasn't Chagall an artist? And wasn't the code name for the Threadripper Zen 3 to be Genesis Peak? And aren't you quoting a rumor? Have there been ANY leaked benchmarks? Has ANY processor EVER launched without leaked benchmarks?

    The lack of things often speak volumes...
  • mode_13h - Tuesday, July 20, 2021 - link

    > And aren't you quoting a rumor?

    Yes, that's exactly what I said.

    > Have there been ANY leaked benchmarks?
    > Has ANY processor EVER launched without leaked benchmarks?

    It's a niche product. There typically aren't many motherboard options for them (and even fewer, at launch). I think it's not surprising, if there aren't any benchmark leaks as of yet. There won't be many engineering samples floating around.

    Anyway, we'll know within about 5 weeks if there's any truth to the rumor.
  • croc - Wednesday, July 21, 2021 - link

    I will know when a bios with AGESA support for a Zen 3 Threadripper arrives. None have arrived yet, and typically they do so about a month prior. Chagal is an artist, and artist names are usually reserved for apus. Supposedly the next TR will use the TRX40 socket, which means that there are many MBs for testing / leaking. Given that the Epyc still uses the SP3 socket, I believe this to be tue The latest 'chagall' rumor is that it won't release before Sept, mebbe as late as November. Gotta love them movable goal posts

    Personaly, I don't think that AMD can get the cores to deliver at the frequencies required for an HEDT product Hell, given their back orders for their server chip I doubt that they even care. Bigger fish to fry, all that. Still, a bit of egg-on-face for Dr. Su. And niche? don't tell the CGI world that they are niche...
  • Qasar - Wednesday, July 21, 2021 - link

    sorry croc, but you could be wrong, looks like TR based on zen 3 could be Chagall, and on both STRX4 and SWRX8, at least according to here :
    https://www.guru3d.com/news-story/amd-threadripper...
    https://www.notebookcheck.net/Zen-3-based-Ryzen-Th...

    from moores law is dead, a video about TR zen 3 :
    https://www.youtube.com/watch?v=la-7Q_VsWUM
    but no concrete info as of yet, but a search on google for threadripper zen 3, all seems to say the same, code name chagall
  • croc - Saturday, July 24, 2021 - link

    No new BIOS since April... 7 days to August, so looks like another no-show, eh...
  • Qasar - Saturday, July 24, 2021 - link

    and the point is ? considering their has been no official announcement from amd when zen 3 TR is to be released, why would there be a bios for it yet ?
    the point of my reply was that you could be wrong on the code name, not the release date.

    at least amd HAS a HEDT cpu, when was the last one from intel ?

    bottom line : all there are, are rumors, which should be taken with some salt.
  • croc - Monday, July 26, 2021 - link

    That would be the Xeon w-3175x

Log in

Don't have an account? Sign up now