Assessing IBM's POWER8, Part 1: A Low Level Look at Little Endian
by Johan De Gelas on July 21, 2016 8:45 AM ESTSingle-Threaded Integer Performance: SPEC CPU2006
Even though SPEC CPU2006 is more HPC and workstation oriented, it contains a good variety of integer workloads. Running SPEC CPU2006 is a good way to evaluate single threaded (or core) performance. The main problem is that the results submitted are "overengineered" and it is very hard to make any fair comparisons.
For that reason, we wanted to keep the settings as "real world" as possible. So we used:
- 64 bit gcc 5.2.1: most used compiler on Linux, good all round compiler that does not try to "break" benchmarks (libquantum...)
- -Ofast: compiler optimization that many developers may use
- -fno-strict-aliasing: necessary to compile some of the subtests
- base run: every subtest is compiled in the same way.
The ultimate objective is to measure performance in applications where for some reason – as is frequently the case – a "multi-thread unfriendly" task keeps us waiting.
Here is the raw data. Perlbench failed to compile on Ubuntu 15.10, so we skipped it. Still we are proud to present you the very first SPEC CPU2006 benchmarks on Little Endian POWER8.
On the IBM server, numactl was used to physically bind the 2, 4, or 8 copies of SPEC CPU to the first 2, 4, or 8 threads of the first core. On the Intel server, the 2 copy benchmark was bound to the first core.
| Subtest SPEC CPU2006 Integer |
Application Type |
IBM POWER8 10c@3.5 Single Thread |
IBM POWER8 10c@3.5 SMT-2 |
IBM POWER8 10c@3.5 SMT-4 |
IBM POWER8 10c@3.5 SMT-8 |
Xeon E5-2699 v4 2.2-3.6 |
Xeon E5-2699 v4 2.2-3.6 (+HT) |
| 400.perlbench | Spam filter | N/A | N/A | N/A | N/A | 32.2 | 36.6 |
| 401.bzip2 | Compress | 17.5 | 26.9 | 33.7 | 35.2 | 19.2 | 25.3 |
| 403.gcc | Compiling | 32.1 | 44.6 | 56.6 | 61.5 | 28.9 | 33.3 |
| 429.mcf | Vehicle scheduling | 47.1 | 50 | 64.1 | 73.5 | 39 | 43.9 |
| 445.gobmk | Game AI | 20.2 | 31.3 | 41.4 | 43.1 | 22.4 | 27.7 |
| 456.hmmer | Protein seq. analyses | 19.1 | 27.1 | 28.6 | 22.5 | 24.2 | 28.4 |
| 458.sjeng | Chess | 17.1 | 25.4 | 32.6 | 33.1 | 24.8 | 28.3 |
| 462.libquantum | Quantum sim |
44.7 | 82.1 | 109 | 108 | 59.2 | 67.3 |
| 464.h264ref | Video encoding | 32.7 | 45.4 | 53.3 | 48.8 | 40.7 | 40.7 |
| 471.omnetpp | Network sim |
23.5 | 29.1 | 37.1 | 42.5 | 23.5 | 29.9 |
| 473.astar | Pathfinding | 16.5 | 24.8 | 33.5 | 36.9 | 18.9 | 23.6 |
| 483.xalancbmk | XML processing | 24.9 | 35.3 | 44.7 | 48.4 | 35.4 | 41.8 |
First we look at how well SMT-2, SMT-4 and SMT-8 work on the IBM POWER8.
| Subtest SPEC CPU2006 Integer |
Application Type |
IBM POWER8 10c@3.5 Single Thread |
IBM POWER8 10c@3.5 SMT-2 |
IBM POWER8 10c@3.5 SMT-4 |
IBM POWER8 10c@3.5 SMT-8 |
| 400.perlbench | Spam filter | N/A | N/A | N/A | N/A |
| 401.bzip2 | Compress | 100% | 154% | 193% | 201% |
| 403.gcc | Compiling | 100% | 139% | 176% | 192% |
| 429.mcf | Vehicle scheduling | 100% | 106% | 136% | 156% |
| 445.gobmk | Game AI | 100% | 155% | 205% | 213% |
| 456.hmmer | Protein seq. analyses | 100% | 142% | 150% | 118% |
| 458.sjeng | Chess | 100% | 149% | 191% | 194% |
| 462.libquantum | Quantum sim |
100% | 184% | 244% | 242% |
| 464.h264ref | Video encoding | 100% | 139% | 163% | 149% |
| 471.omnetpp | Network sim |
100% | 124% | 158% | 180% |
| 473.astar | Pathfinding | 100% | 150% | 203% | 224% |
| 483.xalancbmk | XML processing | 100% | 142% | 180% | 194% |
The performance gains from single threaded operation to two threads are very impressive, as expected. While Intel's SMT-2 offers in most subtests between 10 and 25% better performance, the dual threaded mode of the POWER8 boosts performance by 40 to 50% in most applications, or more than twice as much relative to the Xeons. Not one benchmark regresses when we throw 4 threads upon the IBM POWER8 core. The benchmarks with high IPC such as hmmer peak at SMT-4, but most subtests gain a few % when running 8 threads.
Facebook
Twitter
RSS
124 Comments
View All Comments
jospoortvliet - Tuesday, July 26, 2016 - link
The point Johan makes is that his goal is not to get the best bechmark scores but the most relevant real life data. One can argue if he succeeded, certainly the results are interesting but there is much more to the CPU's as usual. And I do think his choice is justified, while much scientific code would be recompiled with a faster compiler (though the cost of ICC might be a problem in a educational setting), many businesses wouldn't go through that effort.I personally would love to see a newer Ubuntu & GCC being used, just to see what the difference is, if any. The POWER ecosystem seems to evolve fast so a newer platform and compiler could make a tangible difference.
But, of course, if you in your usecase would use ICC or xLC, these benches are not perfect.
Eris_Floralia - Friday, July 22, 2016 - link
Are these two processor both tested at the same frequency？or at their stock clock？tipoo - Friday, July 22, 2016 - link
The latter, page 52.92-3.5 GHz vs 2.2-3.6 GHz
abufrejoval - Thursday, August 4, 2016 - link
Well since Johan really only tested one core on each CPU, it would have been nice to have him verify the actual clock speed of those cores. You'd assume that they'd be able to maintain top speed for any single core workload independent of the number of threads, but checking is better than guessing.roadapathy - Friday, July 22, 2016 - link
22nm? *yawwwwwwwwwn* Come on IBM, you can do better than that, brah.Michael Bay - Saturday, July 23, 2016 - link
Nope, 22 is the best SOI has right now. You have to remember it`s nowhere near standard litographies customer-wise.tipoo - Monday, July 25, 2016 - link
In addition to what Michael Bay (lel) said, remember that only Intel really has 14nm, when TSMC and GloFlo say 14/16nm they really mean 20nm with finfetts.feasibletrash0 - Saturday, July 23, 2016 - link
using a less capable compiler (GCC) to test a chip, and not using everything the chip has to offer seems incredibly flawed to me, what are you testing exactlyaryonoco - Saturday, July 23, 2016 - link
He's testing what actual software people actually run on these things.On your typical Linux host, pretty most everything is compiled with GCC. You want to get into exotic compilers? Sure both IBM and Intel have their exotic proprietary costly compilers, so what. Very few people outside of niche industries use them.
You want to compare a CPU with CPU? You keep the compiler the same. That's just common sense. It's also how the scientific method works!
feasibletrash0 - Sunday, July 24, 2016 - link
right, but you're comparing, say 10% of the silicon on that chip, and saying that the remaining 90% of the transistors making the chip does not matter, they do; if the software is not using them, that's fine, but it's not an accurate comparison of the hardware, it's a comparison of the software