Single-Threaded Integer Performance: SPEC CPU2006

Even though SPEC CPU2006 is more HPC and workstation oriented, it contains a good variety of integer workloads. Running SPEC CPU2006 is a good way to evaluate single threaded (or core) performance. The main problem is that the results submitted are "overengineered" and it is very hard to make any fair comparisons.

So we wanted to keep the settings as "real world" as possible. We welcome constructive criticism to reach that goal. So we used:

  • 64 bit gcc: most used compiler on Linux, good all round compiler that does not try to "break" benchmarks (libquantum...)
  • -Ofast: compiler optimization that many developers may use
  • -fno-strict-aliasing: necessary to compile some of the subtests
  • base run: every subtest is compiled in the same way.

The ultimate objective is to measure performance in applications where for some reason – as is frequently the case – a "multi thread unfriendly" task keeps us waiting.

Nobody expect the ThunderX to be a single threaded performance wonder. Cavium clearly stated that they deliberately went for a high core count with pretty simple cores. As a result, single threaded performance was not a priority.

However, Facebook and other hyperscalers have indicated that they definitely prefer to get the single threaded performance of a Xeon D. So any competitor challenging Intel should try to keep up with the Xeon D in single threaded performance and offer a throughput-per-dollar/watt bonus. So it is very interesting to measure what single threaded performance the current ThunderX can offer.

Application Type Cavium
2 GHz
Xeon D-1557
Xeon D-1587
Xeon E5-2640 v4
Xeon E5-2690 v3
Xeon E5-2699 v4
Xeon E5-2699 v4
400.perlbench Spam filter 8.3 24.7 29 33.4 39 32.2 36.6
401.bzip2 Compression 6.5 15.1 17.2 19.8 24.2 19.2 25.3
403.gcc Compiling 10.8 23.1 27.2 30 37.2 28.9 33.3
429.mcf Vehicle scheduling 10.2 32.6 38.4 40.4 44.8 39 43.9
445.gobmk Game AI 9.2 17.4 20.2 22.7 28.1 22.4 27.7
456.hmmer Protein seq. analyses 4.8 19 21.7 25.1 28 24.2 28.4
458.sjeng Chess 8.8 19.8 22.8 25.6 31.5 24.8 28.3
462.libquantum Quantum sim 5.8 47.9 58.2 60.3 78 59.2 67.3
464.h264ref Video encoding 11.9 32 36.6 41.9 56 40.7 40.7
471.omnetpp Network sim 7 17.3 23 23.6 30.9 23.5 29.9
473.astar Pathfinding 7.9 14.7 17.2 19.8 24.4 18.9 23.6
483.xalancbmk XML processing 8.4 27.8 33.3 36.2 45.1 35.4 41.8

Although some of you have a mathematical mind and are able to easily decipher these kinds of tables, let the rest of us be lazy and translate this into percentages. We make the Xeon D-1581 the baseline. The Xeon D-1557's performance is more or less the single threaded performance some of the important customers such as Facebook like to have.

Application Type Cavium
2 GHz
Xeon D-1557
Xeon D-1581
Xeon E5-2640
400.perlbench Spam filter 29% 85% 100% 115%
401.bzip2 Compression 38% 88% 100% 115%
403.gcc Compiling 40% 85% 100% 110%
429.mcf Vehicle scheduling 27% 85% 100% 105%
445.gobmk Game AI 46% 86% 100% 112%
456.hmmer Protein seq. analyses 22% 88% 100% 116%
458.sjeng Chess 39% 87% 100% 112%
462.libquantum Quantum sim 10% 82% 100% 104%
464.h264ref Video encoding 33% 87% 100% 114%
471.omnetpp Network sim 30% 75% 100% 103%
473.astar Pathfinding 46% 85% 100% 115%
483.xalancbmk XML processing 25% 83% 100% 109%

First of all, single threaded is somewhat better than we expected when we received the first architectural details (a very simple dual issue core with high latency shared L2). However, this is still a fraction of the Xeon D's single threaded performance, which means that ThunderX doesn't look very impressive to companies which feel that single threaded performance should not be lower than a low end Xeon D. The latter is 2 to 4 times faster. On average, the Xeon D-1581 delivers 3 times faster single threaded performance than the ThunderX, but not 5!

SPEC CPU2006 allows us to characterize the ThunderX core a bit better. We ignore libquantum because it has a very special profile: you can triple the score with specific compiler settings, but those settings reduce performance by 2-30%(!) in some other subtests. Those compiler settings optimize cache utilization by splitting records of an array in separate arrays. Combine this with software loop prefetching and libquantum numbers can indeed double or triple. Since libquantum is hardly relevant for the server world and is known for being a target for all kind of benchmark trickery, we ignore it in our comparison.

Mcf exhibits a large amount of data cache misses and memory controller usage. Mcf is also "horribly low IPC" software, so beefy execution backends do not help. Despite those facts, the ThunderX does not do well in mcf. Mcf does a lot of pointer chasing, so the high latency L2-cache and the high latency DRAM access are slowing things down. That is probably also true for XML processing and the network simulator: those subtests have the highest data cache misses.

The shallow pipeline and relatively powerful gshare branch predictor make the ThunderX a better than expected performer in the chess (sjeng), pathfinding (astar), compiling (gcc) and AI (gobmk). Although the gobmk has a relatively high branch misprediction rate on a gshare branch predictor (the highest of all subtests), the ThunderX core can recover very quickly thanks to its 9 stage pipeline. Notice also that gobmk and gcc have relatively large instruction footprints, which gives the ThunderX and its 78 KB I-cache an advantage.

That is also true for the perl, but that benchmark has a relatively high IPC and needs a beefier execution backend. Indeed, the more compute intensive (and thus high IPC sub tests) perlbench and hmmer perform badly relative to the Intel core. In these benchmarks, the wide architecture of the Intel cores pays off.

Benchmarks Versus Reality Multi-Threaded Integer Performance: SPEC CPU2006


View All Comments

  • silverblue - Thursday, June 16, 2016 - link

    I think AMD themselves admitted that the Opteron X1100 was for testing the waters, with K12 being the first proper solution, but that was delayed to get Zen out of the door. I imagine that both products will be on sale concurrently at some point, but even with AMD's desktop-first approach for Zen, it will probably still come to the server market before K12 (both are due 2017). Reply
  • junky77 - Thursday, June 16, 2016 - link

    still, quite strange, no? AMD is in the server business for years. I'm not talking about their ARM solution only, but their other solutions seem to be less interesting.. Reply
  • silverblue - Thursday, June 16, 2016 - link

    I am looking forward to both Zen and K12; there's very little chance that AMD will fail with both. Reply
  • name99 - Wednesday, June 15, 2016 - link

    " It is the first time the Xeon D gets beaten by an ARM v8 SoC..."

    The Apple A9X in the 12" iPad Pro delivers 40GB/s on Stream...
    (That's the Stream built into Geekbench. Conceivably it's slightly different from what's being measured here, but it delivers around 25GB/s for standard desktop/laptop Intel CPUs, and for the A9 and the 9" iPad's A9X, so it seems in the same sort of ballpark.)
  • aryonoco - Thursday, June 16, 2016 - link

    Fantastic article as always Johan. Thank you so much for your very informative articles. I can only imagine how much time and effort writing this article took. It is very much appreciated.

    The first good showing by an ARMv8 server. Nearly 5 years later than expected, but they are getting there. This thing was still produced on 28 HKMG. Give it one more year, a jump to 14nm, and a more mature software ecosystem, and I think the Xeons might finally have some competition on their hands.
  • JohanAnandtech - Thursday, June 16, 2016 - link

    Thank you, and indeed it was probably the most time consuming review ... since Calxeda. :-)
    Yes, there is potential.
  • iwod - Thursday, June 16, 2016 - link

    Even if the ThunderX is half the price of equivalent Xeon, I would still buy Intel Xeon instead. This isn't Smartphone market. In Server, The cost memory and Storage, Networking etc adds up. Not only does it uses a lot more power in Idle, the total TCO AND Pref / Watts still flavours Intel.

    There is also the switching cost of Software involved.
    And those who say Single Core / Thread Performance dont matter have absolutely no idea what they are talking about.

    As far as I can tell, Xeon-D offers a very decent value proposition for even the ARM SoC minded vendors. This will likely continue to be the case as we move to 10nm. I just dont see how ARM is going to get their 20% market share by 2020 as they described in their Shareholder meetings.
  • rahvin - Thursday, June 16, 2016 - link

    If you have to switch software on your severs because you switch architecture you are doing something wrong and are far too dependent on proprietary products. I'm being a bit facetious here but the only reason architecture should limit you is you are using Microsoft products or are in a highly specialized computing field. Linux should dominate your general servers. Reply
  • kgardas - Friday, June 17, 2016 - link

    Even if you are on Linux, still stack support is best on i386/amd64. Look at IBM how it throws a lot of money to get somewhere with POWER8. ARM can't do that, so it's more on vendors to do that and they are doing it a little bit more slowly. Anyway, even AArch64 will mature in LLVM/GCC tool chain, GNU libC, musl libC, linux kernel etc but it'll take some time... Reply
  • tuxRoller - Thursday, June 16, 2016 - link

    Aarch64 has very limited conditional execution support.

Log in

Don't have an account? Sign up now