Single Threaded Integer Performance: SPEC CPU2006

Even in the server market where high core count CPUs are ruling the roost, high single threaded performance is still very desirable. It makes sure that a certain level of performance is guaranteed in every situation, not just in "throughput situations" of "embarrassingly parallel" software. 

SPEC CPU2017 has finally launched, but it did so while our testing was already under way. So SPEC CPU2006 was still our best option to evaluate single threaded performance. Even though SPEC CPU2006 is more HPC and workstation oriented, it contains a good variety of integer workloads.

It is our conviction that we should try to mimic how performance critical software is compiled instead of trying to achieve the highest scores. To that end, we:

  • use 64 bit gcc : by far the most used compiler on linux for integer workloads, good all round compiler that does not try to "break" benchmarks (libquantum...) or favor a certain architecture
  • use gcc version 5.4: standard compiler with Ubuntu 16.04 LTS. (Note that this is upgraded from 4.8.4 used in earlier articles)
  • use -Ofast -fno-strict-aliasing optimization: a good balance between performance and keeping things simple
  • added "-std=gnu89" to the portability settings to resolve the issue that some tests will not compile with gcc 5.x
  • run one copy of the test

The ultimate objective is to measure performance in non-"aggressively optimized" applications where for some reason – as is frequently the case – a "multi-thread unfriendly" task keeps us waiting. 

First the single threaded results. It is important to note that thanks to modern turbo technology, all CPUs will run at higher clock speeds than their base clock speed. 

  • The Xeon E5-2690 ("Sandy Bridge") is capable of boosting up to 3.8 GHz
  • The Xeon E5-2690 v3 ("Haswell") is capable of boosting up to 3.5GHz
  • The Xeon E5-2699 v4  ("Broadwell") is capable of boosting up to 3.6 GHz
  • The Xeon 8176 ("Skylake-SP") is capable of boosting up to 3.8 GHz
  • The EPYC 7601 ("Naples") is capable of boosting up to 3.2 GHz

First we look at the absolute numbers. 

Subtest Application type Xeon E5-2690
@ 3.8
Xeon E5-2690 v3
@ 3.5
Xeon E5-2699 v4
@ 3.6
EPYC 7601
@3.2
Xeon 8176
@3.8
400.perlbench Spam filter 35 41.6 43.4 31.1 50.1
401.bzip2 Compression 24.5 24.0 23.9 24.0 27.1
403.gcc Compiling 33.8 35.5 23.7 35.1 24.5
429.mcf Vehicle scheduling 43.5 42.1 44.6 40.1 43.3
445.gobmk Game AI 27.9 27.8 28.7 24.3 31.0
456.hmmer Protein seq. analyses 26.5 28.0 32.3 27.9 35.4
458.sjeng Chess 28.9 31.0 33.0 23.8 33.6
462.libquantum Quantum sim 55.5 65.0 97.3 69.2 102
464.h264ref Video encoding 50.7 53.7 58.0 50.3 67.0
471.omnetpp Network sim 23.3 31.3 44.5 23.0 40.8
473.astar Pathfinding 25.3 25.1 26.1 19.5 27.4
483.xalancbmk XML processing 41.8 46.1 64.9 35.4 67.3

As raw SPEC scores can be a bit much to deal with in a dense table, we've also broken out our scores on a percentage basis. Sandy Bridge EP (Xeon E5 v1) is about 5 years old, the servers based upon this CPU are going to get replaced by newer ones. So we've made "Single threaded Sandy Bridge-EP performance" our reference (100%) , and compare the single threaded performance of all other architectures accordingly.

Subtest Application type Xeon E5-2690
@ 3.8
Xeon E5-2690 v3
@ 3.5
Xeon E5-2699 v4 @ 3.6 EPYC 7601 @3.2 Xeon 8176 @ 3.8
400.perlbench Spam filter 100% 119% 124% 89% 143%
401.bzip2 Compression 100% 98% 98% 98% 111%
403.gcc Compiling 100% 105% 70% 104% 72%
429.mcf Vehicle scheduling 100% 97% 103% 92% 100%
445.gobmk Game AI 100% 100% 103% 87% 111%
456.hmmer Protein seq. analyses 100% 106% 122% 105% 134%
458.sjeng Chess 100% 107% 114% 82% 116%
462.libquantum Quantum sim 100% 117% 175% 125% 184%
464.h264ref Video encoding 100% 106% 114% 99% 132%
471.omnetpp Network sim 100% 134% 191% 99% 175%
473.astar Pathfinding 100% 99% 103% 77% 108%
483.xalancbmk XML processing 100% 110% 155% 85% 161%

SPEC CPU2006 analysis is complicated, and with only a few days spend on the EPYC server, we must admit that what follows is mostly educated guessing. 

First off, let's gauge the IPC efficiency of the different architectures. Considering that the EPYC core runs at 12-16% lower clockspeeds (3.2 vs 3.6/3.8 GHz), getting 90+% of the performance of the Intel architectures can be considered a "strong" (IPC) showing for the AMD "Zen" architecture. 

As for Intel's latest CPU, pay attention to the effect of the much larger L2-cache of the Skylake-SP core (Xeon 8176) compared to the previous generation "Broadwell". Especially perlbench, gobmk, hmmer and h264ref (the instruction part) benefit. 

Meanwhile with the new GCC 5.4 compiler, Intel's performance on the "403.gcc benchmark" seems to have regressed their newer rchitectures. While we previously saw the Xeon E5-2699v4 perform at 83-95% of the "Sandy Bridge" Xeon E5-2690, this has further regressed to 70%. The AMD Zen core, on the other hand, does exceptionally well when running GCC. The mix of a high percentage of (easy to predict) branches in the instruction mix, a relatively small footprint, and a heavy reliance on low latency (mostly L1/L2/8 MB L3) seems to work well. The workloads where the impact of branch prediction is higher (somewhat higher percentage of branch misses) - gobmk, sjeng, hmmer - perform quite well on "Zen" too, which has a much lower branch misprediction penalty than AMD's previous generation architecture thanks to the µop cache. 

Otherwise the pointer chasing benchmarks – XML procesing and Path finding – which need a large L3-cache, are the worst performing on EPYC. 

Also notice the fact that the low IPC omnetpp ("network sim") runs slower on Skylake-SP than on Broadwell, but still much faster than AMD's EPYC. Omnetpp is an application that benefited from the massive 55 MB L3-cache of Broadwell, and that is why performance has declined on Skylake. Of course, this also means that the fractured 8x8 MB L3 of AMD's EPYC processor causes it to perform much slower than the latest Intel server CPUs. In the video encoding benchmark "h264ref" this plays a role too, but that benchmark relies much more on DRAM bandwidth. The fact that the EPYC core has higher DRAM bandwidth available makes sure that the AMD chip does not fall too far behind the latest Intel cores. 

All in all, we think we can conclude that the single threaded performance of the "Zen architecture" is excellent, but it somewhat let down by the lower turbo clock and the "smaller" 8x8 MB L3-cache. 

Memory Subsystem: Latency SMT Integer Performance With SPEC CPU2006
POST A COMMENT

219 Comments

View All Comments

  • mpbello - Friday, July 14, 2017 - link

    If you are rushing, you reduce scope and deliver fewer pieces with high quality instead of insisting on delivering a full set of benchmarks that you are not sure about its quality.
    The article came to a very strong conclusion: Intel is better for database scenarios. Whatever you do, whether you are rushing or not, you cannot state something like that if the benchmarks supporting your conclusion are not well designed.
    So I agree that the design of the DB benchmark was incredibly weak to sustain such an important conclusion that Intel is the best choice for DB applications.
    Reply
  • StargateSg7 - Sunday, August 06, 2017 - link

    Maybe I'm spoiled, but to me a BIG database is something I usually deal with on a daily basis
    such as 500,000 large and small video files ranging from two megabytes to over a PETABYTE
    (1000 Terabytes) per file running on a Windows and Linux network.

    What sort of read and write speeds do we get between disk, main memory and CPU
    and when doing special FX LIVE on such files which can be 960 x 540 pixel youtube-style
    videos up to full blown 120 fps 8192 x 4320 pixel RAW 64 bits per pixel colour RGBA files
    used for editing and video post-production.

    AND I need for the smaller files, total I/O-transaction rates at around
    OVER 500,000 STREAMS of 1-to-1000 64 kilobyte unique packets
    read and written PER SECOND. Basically 500,000 different users
    simultaneously need up to one thousand 64 kilobyte packets per
    second EACH sent to and read from their devices.

    Obviously Disk speed and network comm speed is an issue here, but on
    a low-level hardware basis, how much can these new Intel and AMD chips
    handle INTERNALLY on such massive data requirements?

    I need EXABYTE-level storage management on a chip! Can EITHER
    Xeon or EPyC do this well? Which One is the winner? ... Based upon
    this report it seems multiple 4-way EPyC processors on waterblocked
    blades could be racked on a 100 gigabit (or faster) fibre backbone
    to do 500,000 simultaneous users at a level MUCH CHEAPER than
    me having to goto IBM or HP for a 30+ million dollar HPC solution!
    Reply
  • PixyMisa - Tuesday, July 11, 2017 - link

    It seems like a well-balanced article to me. Sure the DB performance issue is a corner case, but from a technical point of view its worth knowing.

    I'd love to see a test on a larger database (tens of GB) though.
    Reply
  • philehidiot - Wednesday, July 12, 2017 - link

    It seems to me that some people should set up their own server review websites in order that they might find the unbiased balance that they so crave. They might also find a time dilation device that will allow them to perform the multitude of different workload tests they so desire. I believe this article stated quite clearly the time constraints and the limitations imposed by such constraints. This means that the benchmarks were scheduled down to the minute to get as many in as possible and therefore performing different tests based on the results of the previous benchmarks would have put the entire review dataset in jeopardy.

    It might be nice to consider just how much data has been acquired here, how it might have been done and the degree of interpretation. It might also be worth considering, if you can do a better job, setting up shop on your own and competing as obviously the standard would be so much higher.

    Sigh.
    Reply
  • JohanAnandtech - Thursday, July 13, 2017 - link

    Thank you for being reasonable. :-) Many of the benchmarks (Tinymembench, Stream, SPEC) etc. can be repeated, so people can actually check that we are unbiased. Reply
  • Shankar1962 - Monday, July 17, 2017 - link

    Don't go by the labs idiot
    Understand what real world workloads are.....understand what owning an entire rack means ......you started foul language so you deserve the same respect from me......
    Reply
  • roybotnik - Wednesday, July 12, 2017 - link

    EPYC looks extremely good here aside from the database benchmark, which isn't a useful benchmark anyways. Need to see the DB performance with 100GB+ of memory in use. Reply
  • CarlosYus - Friday, July 14, 2017 - link

    A detailed and unbiased article. I'm awaiting for more tests as testing time passes.
    3.2 Ghz is a moderate Turbo for AMD EPYC, I think AMD could push it further with a higher thermal envelope i/o 14 nm process improvement in the coming months.
    Reply
  • mdw9604 - Tuesday, July 11, 2017 - link

    Nice, comprehensive article. Glad to see AMD is competitive once again in the server CPU space. Reply
  • nathanddrews - Tuesday, July 11, 2017 - link

    "Competitive" seems like an understatement, but yes, AMD is certainly bringing it! Reply

Log in

Don't have an account? Sign up now