Single-Threaded Integer Performance: SPEC CPU2006

Even though SPEC CPU2006 is more HPC and workstation oriented, it contains a good variety of integer workloads. Running SPEC CPU2006 is a good way to evaluate single threaded (or core) performance. The main problem is that the results submitted are "overengineered" and it is very hard to make any fair comparisons.

So we wanted to keep the settings as "real world" as possible. We welcome constructive criticism to reach that goal. So we used:

  • 64 bit gcc: most used compiler on Linux, good all round compiler that does not try to "break" benchmarks (libquantum...)
  • -Ofast: compiler optimization that many developers may use
  • -fno-strict-aliasing: necessary to compile some of the subtests
  • base run: every subtest is compiled in the same way.

The ultimate objective is to measure performance in applications where for some reason – as is frequently the case – a "multi thread unfriendly" task keeps us waiting.

Nobody expect the ThunderX to be a single threaded performance wonder. Cavium clearly stated that they deliberately went for a high core count with pretty simple cores. As a result, single threaded performance was not a priority.

However, Facebook and other hyperscalers have indicated that they definitely prefer to get the single threaded performance of a Xeon D. So any competitor challenging Intel should try to keep up with the Xeon D in single threaded performance and offer a throughput-per-dollar/watt bonus. So it is very interesting to measure what single threaded performance the current ThunderX can offer.

Subtest
SPEC CPU2006
Integer
Application Type Cavium
ThunderX
2 GHz
Xeon D-1557
1.5-2.1
Xeon D-1587
1.8-2.4
Xeon E5-2640 v4
2.4-2.6
Xeon E5-2690 v3
2.6-3.5
Xeon E5-2699 v4
2.2-3.6
Xeon E5-2699 v4
2.2-3.6
(+HT)
400.perlbench Spam filter 8.3 24.7 29 33.4 39 32.2 36.6
401.bzip2 Compression 6.5 15.1 17.2 19.8 24.2 19.2 25.3
403.gcc Compiling 10.8 23.1 27.2 30 37.2 28.9 33.3
429.mcf Vehicle scheduling 10.2 32.6 38.4 40.4 44.8 39 43.9
445.gobmk Game AI 9.2 17.4 20.2 22.7 28.1 22.4 27.7
456.hmmer Protein seq. analyses 4.8 19 21.7 25.1 28 24.2 28.4
458.sjeng Chess 8.8 19.8 22.8 25.6 31.5 24.8 28.3
462.libquantum Quantum sim 5.8 47.9 58.2 60.3 78 59.2 67.3
464.h264ref Video encoding 11.9 32 36.6 41.9 56 40.7 40.7
471.omnetpp Network sim 7 17.3 23 23.6 30.9 23.5 29.9
473.astar Pathfinding 7.9 14.7 17.2 19.8 24.4 18.9 23.6
483.xalancbmk XML processing 8.4 27.8 33.3 36.2 45.1 35.4 41.8

Although some of you have a mathematical mind and are able to easily decipher these kinds of tables, let the rest of us be lazy and translate this into percentages. We make the Xeon D-1581 the baseline. The Xeon D-1557's performance is more or less the single threaded performance some of the important customers such as Facebook like to have.

Subtest
SPEC CPU2006
Integer
Application Type Cavium
ThunderX
2 GHz
Xeon D-1557
1.5-2.1
Xeon D-1581
1.5-2.1
Xeon E5-2640
2.4-2.6
400.perlbench Spam filter 29% 85% 100% 115%
401.bzip2 Compression 38% 88% 100% 115%
403.gcc Compiling 40% 85% 100% 110%
429.mcf Vehicle scheduling 27% 85% 100% 105%
445.gobmk Game AI 46% 86% 100% 112%
456.hmmer Protein seq. analyses 22% 88% 100% 116%
458.sjeng Chess 39% 87% 100% 112%
462.libquantum Quantum sim 10% 82% 100% 104%
464.h264ref Video encoding 33% 87% 100% 114%
471.omnetpp Network sim 30% 75% 100% 103%
473.astar Pathfinding 46% 85% 100% 115%
483.xalancbmk XML processing 25% 83% 100% 109%

First of all, single threaded is somewhat better than we expected when we received the first architectural details (a very simple dual issue core with high latency shared L2). However, this is still a fraction of the Xeon D's single threaded performance, which means that ThunderX doesn't look very impressive to companies which feel that single threaded performance should not be lower than a low end Xeon D. The latter is 2 to 4 times faster. On average, the Xeon D-1581 delivers 3 times faster single threaded performance than the ThunderX, but not 5!

SPEC CPU2006 allows us to characterize the ThunderX core a bit better. We ignore libquantum because it has a very special profile: you can triple the score with specific compiler settings, but those settings reduce performance by 2-30%(!) in some other subtests. Those compiler settings optimize cache utilization by splitting records of an array in separate arrays. Combine this with software loop prefetching and libquantum numbers can indeed double or triple. Since libquantum is hardly relevant for the server world and is known for being a target for all kind of benchmark trickery, we ignore it in our comparison.

Mcf exhibits a large amount of data cache misses and memory controller usage. Mcf is also "horribly low IPC" software, so beefy execution backends do not help. Despite those facts, the ThunderX does not do well in mcf. Mcf does a lot of pointer chasing, so the high latency L2-cache and the high latency DRAM access are slowing things down. That is probably also true for XML processing and the network simulator: those subtests have the highest data cache misses.

The shallow pipeline and relatively powerful gshare branch predictor make the ThunderX a better than expected performer in the chess (sjeng), pathfinding (astar), compiling (gcc) and AI (gobmk). Although the gobmk has a relatively high branch misprediction rate on a gshare branch predictor (the highest of all subtests), the ThunderX core can recover very quickly thanks to its 9 stage pipeline. Notice also that gobmk and gcc have relatively large instruction footprints, which gives the ThunderX and its 78 KB I-cache an advantage.

That is also true for the perl, but that benchmark has a relatively high IPC and needs a beefier execution backend. Indeed, the more compute intensive (and thus high IPC sub tests) perlbench and hmmer perform badly relative to the Intel core. In these benchmarks, the wide architecture of the Intel cores pays off.

Benchmarks Versus Reality Multi-Threaded Integer Performance: SPEC CPU2006
POST A COMMENT

82 Comments

View All Comments

  • Spunjji - Wednesday, June 15, 2016 - link

    Well, this is certainly promising. Absent AMD, Intel need some healthy competition in this market - even if it is in something of a niche area. Reply
  • niva - Wednesday, June 15, 2016 - link

    This is the area where profits are made, not "something of a niche area." Reply
  • Shadow7037932 - Wednesday, June 15, 2016 - link

    Yeah, I mean getting some big customers like Facebook or Google would be rather profitable I'd imagine. Reply
  • JohanAnandtech - Thursday, June 16, 2016 - link

    More than 30% of Intel's revenue, and the most profitable area for years, and for years to come... Reply
  • prisonerX - Wednesday, June 15, 2016 - link

    This is the future. Single thread performance has reached a dead end and parallelism is the only way forward. Intel's legacy architecture is a millstone around its neck. ARM's open model and efficient implementation will deliver more cores and more performance as software adapts.

    The monopolists monopolise themselves into irrelevance yet again.
    Reply
  • CajunArson - Wednesday, June 15, 2016 - link

    " Intel's legacy architecture is a millstone around its neck."

    I wouldn't call those Xeon-D parts putting up excellent performance at lower prices and vastly lower power consumption levels to be any kind of "millstone".

    "ARM's open model and efficient implementation "

    What's "open" about these Cavium chips exactly? They can only run a few specialized Linux flavors that don't even have the full range of standard PC software available to them.

    What is efficient about a brand-new ARM chip from 2016 losing at performance per watt to the 4.5 year old Sandy Bridge parts that you were insulting?

    As for monopolies, ARM has monopolized the mobile market and brought us "open" ecosystems like the iPhone walled-garden and Android devices that literally never receive security updates. I'd take a plain x86 PC that I can slap Linux on any day of the week over the true monopoly that ARM has over locked-down smartphones.
    Reply
  • shelbystripes - Wednesday, June 15, 2016 - link

    You're right to criticize the "millstone" comment, Intel has done quite well achieving both high performance and high performance-per-watt in their server designs.

    But your comment about a "true monopoly" in the "locked-down smartphone" market is ridiculous. The openness (or lack thereof) that you're complaining about has nothing to do with the CPU architecture at all. An x86 smartphone or tablet can just as easily be locked down, and they are. I own a Dell Venue 8 7000, which is an Android tablet with an Intel Atom SoC inside. It's a great tablet with great hardware. But it's got a bunch of uninstallable crapware installed, Dell abandoned it after 5.1 (it's ridiculous that a tablet with a quad-core 2GHz SoC and 2GB RAM will never see Marshmallow), and the locked smartphone-esque bootloader means I can't repurpose it to a Linux distro even if one existed that supported all the hardware inside this thing.

    On the flipside, the most popular open-source learning/development solution out there right now is the ARM-based Raspberry Pi. There are a number of Linux distros available for it, and everything is OSS, even the GPU driver.
    Reply
  • TheLightbringer - Thursday, June 16, 2016 - link

    You haven't done your homework.

    Some mobile devices were coming with Intel. But like Microsoft it entered the market too late, without offering any real value. The phrase "Too little, too late" fit them both.

    ARM didn't do a monopoly. They just simply saw an opportunity and embrace it. In the early IBM clone days Intel licensed their architecture to allow competition and broad arrange of products. After the market was won, they went greedy, didn't licensed the architecture anymore and cut a lot of players out, leaving a need for a chip licensing scheme. And that's where ARM got in.

    Google develops Android OS, but is up to phone vendors and carriers to deploy them. And they don't want to for economic reasons. They prefer to sell you a new phone for $$$.

    Intel and MS got in the mobile/car market exactly what they deserve, nothing else.
    Reply
  • junky77 - Friday, June 17, 2016 - link

    they all greedy. Some just play it smartly or have more luck in decision making
    But, yea, when you read about the way IBM behaved when things were fresh - it's quite amazing. They had much of the market and could do a lot of stuff, but they simply had a very narrow mind set
    Reply
  • soaringrocks - Wednesday, June 15, 2016 - link

    You make it sound like it's mostly a SW problem, I think it's more complex than that. Actual performance is very dependent on the types of workload and some tasks fit Intel CPUs nicely and the performance per watt for ARM is lacking despite the hype of that architecture being uniquely qualified for low-power. It will be fun to watch how the battle evolves though. Reply

Log in

Don't have an account? Sign up now