Assessing IBM's POWER8, Part 1: A Low Level Look at Little Endian
by Johan De Gelas on July 21, 2016 8:45 AM ESTSingle-Threaded Integer Performance: SPEC CPU2006
Even though SPEC CPU2006 is more HPC and workstation oriented, it contains a good variety of integer workloads. Running SPEC CPU2006 is a good way to evaluate single threaded (or core) performance. The main problem is that the results submitted are "overengineered" and it is very hard to make any fair comparisons.
For that reason, we wanted to keep the settings as "real world" as possible. So we used:
- 64 bit gcc 5.2.1: most used compiler on Linux, good all round compiler that does not try to "break" benchmarks (libquantum...)
- -Ofast: compiler optimization that many developers may use
- -fno-strict-aliasing: necessary to compile some of the subtests
- base run: every subtest is compiled in the same way.
The ultimate objective is to measure performance in applications where for some reason – as is frequently the case – a "multi-thread unfriendly" task keeps us waiting.
Here is the raw data. Perlbench failed to compile on Ubuntu 15.10, so we skipped it. Still we are proud to present you the very first SPEC CPU2006 benchmarks on Little Endian POWER8.
On the IBM server, numactl was used to physically bind the 2, 4, or 8 copies of SPEC CPU to the first 2, 4, or 8 threads of the first core. On the Intel server, the 2 copy benchmark was bound to the first core.
| Subtest SPEC CPU2006 Integer |
Application Type |
IBM POWER8 10c@3.5 Single Thread |
IBM POWER8 10c@3.5 SMT-2 |
IBM POWER8 10c@3.5 SMT-4 |
IBM POWER8 10c@3.5 SMT-8 |
Xeon E5-2699 v4 2.2-3.6 |
Xeon E5-2699 v4 2.2-3.6 (+HT) |
| 400.perlbench | Spam filter | N/A | N/A | N/A | N/A | 32.2 | 36.6 |
| 401.bzip2 | Compress | 17.5 | 26.9 | 33.7 | 35.2 | 19.2 | 25.3 |
| 403.gcc | Compiling | 32.1 | 44.6 | 56.6 | 61.5 | 28.9 | 33.3 |
| 429.mcf | Vehicle scheduling | 47.1 | 50 | 64.1 | 73.5 | 39 | 43.9 |
| 445.gobmk | Game AI | 20.2 | 31.3 | 41.4 | 43.1 | 22.4 | 27.7 |
| 456.hmmer | Protein seq. analyses | 19.1 | 27.1 | 28.6 | 22.5 | 24.2 | 28.4 |
| 458.sjeng | Chess | 17.1 | 25.4 | 32.6 | 33.1 | 24.8 | 28.3 |
| 462.libquantum | Quantum sim |
44.7 | 82.1 | 109 | 108 | 59.2 | 67.3 |
| 464.h264ref | Video encoding | 32.7 | 45.4 | 53.3 | 48.8 | 40.7 | 40.7 |
| 471.omnetpp | Network sim |
23.5 | 29.1 | 37.1 | 42.5 | 23.5 | 29.9 |
| 473.astar | Pathfinding | 16.5 | 24.8 | 33.5 | 36.9 | 18.9 | 23.6 |
| 483.xalancbmk | XML processing | 24.9 | 35.3 | 44.7 | 48.4 | 35.4 | 41.8 |
First we look at how well SMT-2, SMT-4 and SMT-8 work on the IBM POWER8.
| Subtest SPEC CPU2006 Integer |
Application Type |
IBM POWER8 10c@3.5 Single Thread |
IBM POWER8 10c@3.5 SMT-2 |
IBM POWER8 10c@3.5 SMT-4 |
IBM POWER8 10c@3.5 SMT-8 |
| 400.perlbench | Spam filter | N/A | N/A | N/A | N/A |
| 401.bzip2 | Compress | 100% | 154% | 193% | 201% |
| 403.gcc | Compiling | 100% | 139% | 176% | 192% |
| 429.mcf | Vehicle scheduling | 100% | 106% | 136% | 156% |
| 445.gobmk | Game AI | 100% | 155% | 205% | 213% |
| 456.hmmer | Protein seq. analyses | 100% | 142% | 150% | 118% |
| 458.sjeng | Chess | 100% | 149% | 191% | 194% |
| 462.libquantum | Quantum sim |
100% | 184% | 244% | 242% |
| 464.h264ref | Video encoding | 100% | 139% | 163% | 149% |
| 471.omnetpp | Network sim |
100% | 124% | 158% | 180% |
| 473.astar | Pathfinding | 100% | 150% | 203% | 224% |
| 483.xalancbmk | XML processing | 100% | 142% | 180% | 194% |
The performance gains from single threaded operation to two threads are very impressive, as expected. While Intel's SMT-2 offers in most subtests between 10 and 25% better performance, the dual threaded mode of the POWER8 boosts performance by 40 to 50% in most applications, or more than twice as much relative to the Xeons. Not one benchmark regresses when we throw 4 threads upon the IBM POWER8 core. The benchmarks with high IPC such as hmmer peak at SMT-4, but most subtests gain a few % when running 8 threads.
Facebook
Twitter
RSS
124 Comments
View All Comments
zodiacfml - Thursday, July 21, 2016 - link
Like a good TV series, I can't wait for the next episode.aryonoco - Friday, July 22, 2016 - link
OK, this is literally why Anandtech is the best in the tech journalism industry.There is nowhere else on the net that you can find a head to head comparison between POWER and Xeon, and unless you work in the tech department of a Fortune 500 company, this information has just not been available, until now.
Johan, thank you for your work on this article. I did give you beef in your previous article about using LE Ubuntu but I concede your point. Very happy to you are writing more for Anandtech these days.
Xeons really need some competition. Whether that competition comes from POWER or ARM or Zen, I am happy to see some competition. IBM has big plans for POWER9. Hopefully this is just the start of things to come.
JohanAnandtech - Friday, July 22, 2016 - link
Thanks! it is very exciting to perform benchmarks that nobody has published yet :-).In hindsight, I have to admit that the first article contained too few benchmarks that really mattered for POWER8. Most of our usual testing and scripting did not work, and so after lot of tinkering, swearing and sweat I got some benchmarks working on this "exotic to me" platform. The contrast between what one would expect to see on POWER8 and me being proud of being able to somewhat "tame the beast" could not have been greater :-). In other words, there was a learning curve.
tipoo - Friday, July 22, 2016 - link
I found it very interesting as well and would certainly not mind seeing more from this space, like maybe Xeon Phi and SPARC M7jospoortvliet - Tuesday, July 26, 2016 - link
Amen. But, to not ask to much, just the prospect of part 2 of the Power benchmark is already super exciting. Yes, the Internetz need more of this!Daniel Egger - Friday, July 22, 2016 - link
Not quite sure what the Endianess of a systems adds to the competitive factor. Maybe someone could elaborate why it is so important to run a system in LE?ZeDestructor - Friday, July 22, 2016 - link
Not much, really, with the compilers being good and all that.Really, it's quite clearly there just for some excellent alliteration.
JohanAnandtech - Friday, July 22, 2016 - link
Basically LE reduces the barrier for an IBM server being integrated in x86 dominated datacenter.see https://www.ibm.com/developerworks/community/blogs...
Just a few reasons:
"Numerous clients, software partners, and IBM’s own software developers have told us that porting their software to Power becomes simpler if the Linux environment on Power supports little endian mode, more closely matching the environment provided by Linux on x86. This new level of support will *** lower the barrier to entry for porting Linux on x86 software to Linux on Power **."
"A system accelerator programmer (GPU or FPGA) who needs to share memory with applications running in the system processor must share data in an pre-determined endianness for correct application functionality."
Daniel Egger - Friday, July 22, 2016 - link
While correct in theory, this hasn't been a problem for the last 20 years. People are used to using BE on PPC/POWER, the software, the drivers and the infrastructure are very mature (as a matter of fact it was my job 15 years ago to make sure they are). PPC/POWER actually have configurable endianess so if someone wanted to go LE earlier it would have easily been possible but only few ever attempted that stunt; so why have the big disruption now?KAlmquist - Friday, July 22, 2016 - link
I assume that this is about selling POWER boxes to companies that currently run all x86 servers, and have a bunch of custom software that they might be willing to recompile. If the customer has to spend a bunch of time fixing endian dependencies in his software in order to get it to work on POWER, it will probably be less expensive for them to simply stick with x86.