Assessing IBM's POWER8, Part 1: A Low Level Look at Little Endian
by Johan De Gelas on July 21, 2016 8:45 AM ESTSingle-Threaded Integer Performance: SPEC CPU2006
Even though SPEC CPU2006 is more HPC and workstation oriented, it contains a good variety of integer workloads. Running SPEC CPU2006 is a good way to evaluate single threaded (or core) performance. The main problem is that the results submitted are "overengineered" and it is very hard to make any fair comparisons.
For that reason, we wanted to keep the settings as "real world" as possible. So we used:
- 64 bit gcc 5.2.1: most used compiler on Linux, good all round compiler that does not try to "break" benchmarks (libquantum...)
- -Ofast: compiler optimization that many developers may use
- -fno-strict-aliasing: necessary to compile some of the subtests
- base run: every subtest is compiled in the same way.
The ultimate objective is to measure performance in applications where for some reason – as is frequently the case – a "multi-thread unfriendly" task keeps us waiting.
Here is the raw data. Perlbench failed to compile on Ubuntu 15.10, so we skipped it. Still we are proud to present you the very first SPEC CPU2006 benchmarks on Little Endian POWER8.
On the IBM server, numactl was used to physically bind the 2, 4, or 8 copies of SPEC CPU to the first 2, 4, or 8 threads of the first core. On the Intel server, the 2 copy benchmark was bound to the first core.
| Subtest SPEC CPU2006 Integer |
Application Type |
IBM POWER8 10c@3.5 Single Thread |
IBM POWER8 10c@3.5 SMT-2 |
IBM POWER8 10c@3.5 SMT-4 |
IBM POWER8 10c@3.5 SMT-8 |
Xeon E5-2699 v4 2.2-3.6 |
Xeon E5-2699 v4 2.2-3.6 (+HT) |
| 400.perlbench | Spam filter | N/A | N/A | N/A | N/A | 32.2 | 36.6 |
| 401.bzip2 | Compress | 17.5 | 26.9 | 33.7 | 35.2 | 19.2 | 25.3 |
| 403.gcc | Compiling | 32.1 | 44.6 | 56.6 | 61.5 | 28.9 | 33.3 |
| 429.mcf | Vehicle scheduling | 47.1 | 50 | 64.1 | 73.5 | 39 | 43.9 |
| 445.gobmk | Game AI | 20.2 | 31.3 | 41.4 | 43.1 | 22.4 | 27.7 |
| 456.hmmer | Protein seq. analyses | 19.1 | 27.1 | 28.6 | 22.5 | 24.2 | 28.4 |
| 458.sjeng | Chess | 17.1 | 25.4 | 32.6 | 33.1 | 24.8 | 28.3 |
| 462.libquantum | Quantum sim |
44.7 | 82.1 | 109 | 108 | 59.2 | 67.3 |
| 464.h264ref | Video encoding | 32.7 | 45.4 | 53.3 | 48.8 | 40.7 | 40.7 |
| 471.omnetpp | Network sim |
23.5 | 29.1 | 37.1 | 42.5 | 23.5 | 29.9 |
| 473.astar | Pathfinding | 16.5 | 24.8 | 33.5 | 36.9 | 18.9 | 23.6 |
| 483.xalancbmk | XML processing | 24.9 | 35.3 | 44.7 | 48.4 | 35.4 | 41.8 |
First we look at how well SMT-2, SMT-4 and SMT-8 work on the IBM POWER8.
| Subtest SPEC CPU2006 Integer |
Application Type |
IBM POWER8 10c@3.5 Single Thread |
IBM POWER8 10c@3.5 SMT-2 |
IBM POWER8 10c@3.5 SMT-4 |
IBM POWER8 10c@3.5 SMT-8 |
| 400.perlbench | Spam filter | N/A | N/A | N/A | N/A |
| 401.bzip2 | Compress | 100% | 154% | 193% | 201% |
| 403.gcc | Compiling | 100% | 139% | 176% | 192% |
| 429.mcf | Vehicle scheduling | 100% | 106% | 136% | 156% |
| 445.gobmk | Game AI | 100% | 155% | 205% | 213% |
| 456.hmmer | Protein seq. analyses | 100% | 142% | 150% | 118% |
| 458.sjeng | Chess | 100% | 149% | 191% | 194% |
| 462.libquantum | Quantum sim |
100% | 184% | 244% | 242% |
| 464.h264ref | Video encoding | 100% | 139% | 163% | 149% |
| 471.omnetpp | Network sim |
100% | 124% | 158% | 180% |
| 473.astar | Pathfinding | 100% | 150% | 203% | 224% |
| 483.xalancbmk | XML processing | 100% | 142% | 180% | 194% |
The performance gains from single threaded operation to two threads are very impressive, as expected. While Intel's SMT-2 offers in most subtests between 10 and 25% better performance, the dual threaded mode of the POWER8 boosts performance by 40 to 50% in most applications, or more than twice as much relative to the Xeons. Not one benchmark regresses when we throw 4 threads upon the IBM POWER8 core. The benchmarks with high IPC such as hmmer peak at SMT-4, but most subtests gain a few % when running 8 threads.
Facebook
Twitter
RSS
124 Comments
View All Comments
abufrejoval - Thursday, August 4, 2016 - link
I believe "heavily threaded" is somewhat imprecise here: Knights Landing (KNL) is really more about vectorized workloads, or one very loopy and computationally expensive problem, which has been partitioned into lots of chunks, but has high locality. Same code, related data, far more computational throughput than data flowthrough.Power8 will do better on such workloads than perhaps Intel, but never as good as a GPU or KNL.
However it does evidently better per core on highly threaded workloads, where lots of execution threads share the same code but distinct or less related datasets, less scientific and more commercial workloads, more data flowing through.
Funnily KNL might even do well there, beating its Xeon-D sibling in every benchmark, even in terms of energy efficience.
But I'm afraid that's because most of the KNL surface area would remain dark on such workload while the invests would burn through any budget.
KNL is an odd beast designed for a rather specific job and only earn its money there, even if you can run Minecraft or Office on it.
Kevin G - Friday, July 22, 2016 - link
I do think comparison with Xeon Phi is fair since it can run/boot itself now with Knight's Landing. Software parity with the normal x86 ecosystem is now there so it can run off the shelf binaries.I am very curious how well such a dense number of cores perform for workloads that don't need high single threaded performance.
Another interest factor would be memory bandwidth performance as Xeon Phi has plenty. The HMC only further enhances that metric and worth exploring it as both a cache and main memory region for benchmarks.
Ratman6161 - Thursday, July 21, 2016 - link
Will you be addressing virtualization in a future article. I ask this because you are saying the lower cost Power8 systems are intended to compete with the Dell's, HP's, Lenovo etc x86 servers. But these days, a very high percentage of x86 work loads are virutalized either on VMWare or competing products. In 2009 Gartner had it at about 50% and by 2014 it was at 70%. I didn't find a number for '15 or '16 but I expect the percentage would have continued to rise. So if they want to take the place of x86 boxes, they have to be able to do the tasks those boxes do...which tends to largely be to run virtual machines that do the actual workloads.And, what about all the x86 boxes running Windows Server or more commonly Windows Server Virtual machines? Windows Server shops aren't likely to ditch windows in favor of Linux solely for the privilege of running on Power8?
One last thing to consider regarding price. These days we can buy quite robust Intel based server for around $10K. So, supposing I can buy a Power8 system for about the same price? Essentially the hardware has gotten so cheap compared to the licensing and support costs for the software we are running that its a drop in the bucket. If we needed 10 Intel servers or 6 Power 8's to do the same job (assuming the Power8's could run all our VM's), the Power8's could come out lower priced hardware wise, but the difference is, as I said, a drop in the bucket in the overall scheme of things. Performance wise, with the x86 boxes, you just throw more cores at it.
aryonoco - Friday, July 22, 2016 - link
KVM works well on POWER.No idea about proprietary things like VMWare. But that would be up to them to port.
Ratman6161 - Friday, July 22, 2016 - link
Near as I can tell, there is a PowerKVM that runs on Power 8 but that doesn't allow you to run Windows Server VM's - seems to support only Linux guests.Zetbo - Saturday, July 23, 2016 - link
Windows does not support POWER, so there is no point of using POWER if you need Windows!utroz - Thursday, July 21, 2016 - link
AMD should have used IBM's 22nm SOI to make cpu's so that they would not have been totally dead in the performance and server cpu market for years. GF now owns this process as they "bought" IBM's fabs and tech. I think that 22nm SOI might be better for high speed cpu's than the 14nm LPP FinFet that AMD is using for ZEN at the cost of die size.amagriva - Thursday, July 21, 2016 - link
How much you payed your cristal ball?spikebike - Thursday, July 21, 2016 - link
So a single socket Power8 is somewhat faster than the intel chip. But is being compared in a single socket configuration where the intel is designed for a two socket. Unless the power8 is cheaper than an intel dual socket seems most fare to compare both CPU as they are designed to be used.SarahKerrigan - Friday, July 22, 2016 - link
Power is designed for systems up to 16 sockets (IBM E880.) One socket is just the entry point.