Assessing IBM's POWER8, Part 1: A Low Level Look at Little Endian
by Johan De Gelas on July 21, 2016 8:45 AM ESTSingle-Threaded Integer Performance: SPEC CPU2006
Even though SPEC CPU2006 is more HPC and workstation oriented, it contains a good variety of integer workloads. Running SPEC CPU2006 is a good way to evaluate single threaded (or core) performance. The main problem is that the results submitted are "overengineered" and it is very hard to make any fair comparisons.
For that reason, we wanted to keep the settings as "real world" as possible. So we used:
- 64 bit gcc 5.2.1: most used compiler on Linux, good all round compiler that does not try to "break" benchmarks (libquantum...)
- -Ofast: compiler optimization that many developers may use
- -fno-strict-aliasing: necessary to compile some of the subtests
- base run: every subtest is compiled in the same way.
The ultimate objective is to measure performance in applications where for some reason – as is frequently the case – a "multi-thread unfriendly" task keeps us waiting.
Here is the raw data. Perlbench failed to compile on Ubuntu 15.10, so we skipped it. Still we are proud to present you the very first SPEC CPU2006 benchmarks on Little Endian POWER8.
On the IBM server, numactl was used to physically bind the 2, 4, or 8 copies of SPEC CPU to the first 2, 4, or 8 threads of the first core. On the Intel server, the 2 copy benchmark was bound to the first core.
| Subtest SPEC CPU2006 Integer |
Application Type |
IBM POWER8 10c@3.5 Single Thread |
IBM POWER8 10c@3.5 SMT-2 |
IBM POWER8 10c@3.5 SMT-4 |
IBM POWER8 10c@3.5 SMT-8 |
Xeon E5-2699 v4 2.2-3.6 |
Xeon E5-2699 v4 2.2-3.6 (+HT) |
| 400.perlbench | Spam filter | N/A | N/A | N/A | N/A | 32.2 | 36.6 |
| 401.bzip2 | Compress | 17.5 | 26.9 | 33.7 | 35.2 | 19.2 | 25.3 |
| 403.gcc | Compiling | 32.1 | 44.6 | 56.6 | 61.5 | 28.9 | 33.3 |
| 429.mcf | Vehicle scheduling | 47.1 | 50 | 64.1 | 73.5 | 39 | 43.9 |
| 445.gobmk | Game AI | 20.2 | 31.3 | 41.4 | 43.1 | 22.4 | 27.7 |
| 456.hmmer | Protein seq. analyses | 19.1 | 27.1 | 28.6 | 22.5 | 24.2 | 28.4 |
| 458.sjeng | Chess | 17.1 | 25.4 | 32.6 | 33.1 | 24.8 | 28.3 |
| 462.libquantum | Quantum sim |
44.7 | 82.1 | 109 | 108 | 59.2 | 67.3 |
| 464.h264ref | Video encoding | 32.7 | 45.4 | 53.3 | 48.8 | 40.7 | 40.7 |
| 471.omnetpp | Network sim |
23.5 | 29.1 | 37.1 | 42.5 | 23.5 | 29.9 |
| 473.astar | Pathfinding | 16.5 | 24.8 | 33.5 | 36.9 | 18.9 | 23.6 |
| 483.xalancbmk | XML processing | 24.9 | 35.3 | 44.7 | 48.4 | 35.4 | 41.8 |
First we look at how well SMT-2, SMT-4 and SMT-8 work on the IBM POWER8.
| Subtest SPEC CPU2006 Integer |
Application Type |
IBM POWER8 10c@3.5 Single Thread |
IBM POWER8 10c@3.5 SMT-2 |
IBM POWER8 10c@3.5 SMT-4 |
IBM POWER8 10c@3.5 SMT-8 |
| 400.perlbench | Spam filter | N/A | N/A | N/A | N/A |
| 401.bzip2 | Compress | 100% | 154% | 193% | 201% |
| 403.gcc | Compiling | 100% | 139% | 176% | 192% |
| 429.mcf | Vehicle scheduling | 100% | 106% | 136% | 156% |
| 445.gobmk | Game AI | 100% | 155% | 205% | 213% |
| 456.hmmer | Protein seq. analyses | 100% | 142% | 150% | 118% |
| 458.sjeng | Chess | 100% | 149% | 191% | 194% |
| 462.libquantum | Quantum sim |
100% | 184% | 244% | 242% |
| 464.h264ref | Video encoding | 100% | 139% | 163% | 149% |
| 471.omnetpp | Network sim |
100% | 124% | 158% | 180% |
| 473.astar | Pathfinding | 100% | 150% | 203% | 224% |
| 483.xalancbmk | XML processing | 100% | 142% | 180% | 194% |
The performance gains from single threaded operation to two threads are very impressive, as expected. While Intel's SMT-2 offers in most subtests between 10 and 25% better performance, the dual threaded mode of the POWER8 boosts performance by 40 to 50% in most applications, or more than twice as much relative to the Xeons. Not one benchmark regresses when we throw 4 threads upon the IBM POWER8 core. The benchmarks with high IPC such as hmmer peak at SMT-4, but most subtests gain a few % when running 8 threads.
Facebook
Twitter
RSS
124 Comments
View All Comments
Michael Bay - Sunday, July 24, 2016 - link
Hardware does not exist for its own sake, it exists to run software. AT is entirely correct in their methodology.jospoortvliet - Tuesday, July 26, 2016 - link
I'd argue it is the other way around, GCC might leave 5-10% performance on the table in some niche cases but does just fine most of the time. There's a reason Intel and IBM contribute to GCC - to make sure it doesn't get too far behind as they know very well most of their customers use these compilers and not their proprietary ones.Of course, for scientific computing and other niches it makes all the difference and one can argue these heavy systems ARE for niche markets but I still think it was a sane choice to go with GCC.
abufrejoval - Thursday, August 4, 2016 - link
Actually exercising 90% of all transistors on a CPU die these days, is both very hard to do (next to impossible) and will only slow the clock to avoid overstepping TDP.And I seriously doubt that the GCC will underuse a CPU at 10% its computational capacity.
Actually from what I saw the GCC by itself (compiling) was best at exploiting the full 8T potential of the Power8. And since the GCC is compiled by itself, that speaks for the quality of machine code that it can produce, if the source allows it. And that speaks for the quality of the GCC source code, ergo prove you can do better before you rant.
abufrejoval - Thursday, August 4, 2016 - link
Well this is part 1 and describes one scenario. What you want is another scenario and of course it's a valid if a very distinct one.Actually distinct is the word here: You'd be using a vendor's compiler if your main job is a distinct workload, because you'd want to squeeze every bit of performance out of that.
The problem with that is of course, that any distinct workload makes it rather boring for the general public because they cannot translate the benchmark to their environment.
AT aims to satisfy the broadest meaningful audience and Johan as done a great, great job at that.
I'm sure he'll also write a part 4711 for you specifically, if you make it economically attractive.
Hell, even I'd do that given the proper incentive!
Zan Lynx - Sunday, July 24, 2016 - link
Using GCC as the compiler is also why (in my opinion) the Intel chips aren't using their full TDP. Large areas of Intel chips are dedicated to vector operations in SSE and AVX. If you don't issue those instructions then half the chip isn't even being used.Some gamers who love their overclocked Intel chips have actually complained to game engine developers who add AVX to the game engine. Because it ruins their overclock even if the game runs much faster. Then they're in the situation of being forced to clock down from 4.5 GHz to 3.7 in order to avoid lockups or thermal throttling.
Kevin G - Sunday, July 24, 2016 - link
The Xeon E3 v3's had different clock speeds for AVX code: it consumed too much power and got too hot while under total load.This holds true on the E5 v4's but the AVX penalty is done on a core-by-core basis, not across the entire chip. The result is improved performance in mixed workloads. This is a good thing as AVX hasn't broken out much beyond the HPC markets.
talonted - Monday, July 25, 2016 - link
For those interested in getting a Power8 workstation. Check out Talos.https://www.raptorengineering.com/TALOS/prerelease...
137ben - Monday, July 25, 2016 - link
I made an account to say that this article (along with the subsequent stock-cooler comparison article) is why I really love Anandtech. A lot of the code I run/write for my research is CPU-bottlenecked. Still, until the last year or so, I didn't know very much about hardware. Now, reading Anandtech, I have learned so much more about the hardware I depend on from this website than from any other website. Most just repeat announcements or run meaningless cursory synthetic benchmarks. The fact that Johan De Gelas has written such a deep dive into the inner workings of something as complex as a server CPU architecture, and done it in a way that I can understand, is remarkable. Great job Anandtech, keep it up and I'll always come back.JohanAnandtech - Thursday, July 28, 2016 - link
You made me a happy man, I achieved my goal :-)alpha754293 - Wednesday, July 27, 2016 - link
Excellent work and review as always Johan. I would have been interest to see how the two processors perform in floating point intensive benchmarks though...