Assessing IBM's POWER8, Part 1: A Low Level Look at Little Endian
by Johan De Gelas on July 21, 2016 8:45 AM ESTSingle-Threaded Integer Performance: SPEC CPU2006
Even though SPEC CPU2006 is more HPC and workstation oriented, it contains a good variety of integer workloads. Running SPEC CPU2006 is a good way to evaluate single threaded (or core) performance. The main problem is that the results submitted are "overengineered" and it is very hard to make any fair comparisons.
For that reason, we wanted to keep the settings as "real world" as possible. So we used:
- 64 bit gcc 5.2.1: most used compiler on Linux, good all round compiler that does not try to "break" benchmarks (libquantum...)
- -Ofast: compiler optimization that many developers may use
- -fno-strict-aliasing: necessary to compile some of the subtests
- base run: every subtest is compiled in the same way.
The ultimate objective is to measure performance in applications where for some reason – as is frequently the case – a "multi-thread unfriendly" task keeps us waiting.
Here is the raw data. Perlbench failed to compile on Ubuntu 15.10, so we skipped it. Still we are proud to present you the very first SPEC CPU2006 benchmarks on Little Endian POWER8.
On the IBM server, numactl was used to physically bind the 2, 4, or 8 copies of SPEC CPU to the first 2, 4, or 8 threads of the first core. On the Intel server, the 2 copy benchmark was bound to the first core.
| Subtest SPEC CPU2006 Integer |
Application Type |
IBM POWER8 10c@3.5 Single Thread |
IBM POWER8 10c@3.5 SMT-2 |
IBM POWER8 10c@3.5 SMT-4 |
IBM POWER8 10c@3.5 SMT-8 |
Xeon E5-2699 v4 2.2-3.6 |
Xeon E5-2699 v4 2.2-3.6 (+HT) |
| 400.perlbench | Spam filter | N/A | N/A | N/A | N/A | 32.2 | 36.6 |
| 401.bzip2 | Compress | 17.5 | 26.9 | 33.7 | 35.2 | 19.2 | 25.3 |
| 403.gcc | Compiling | 32.1 | 44.6 | 56.6 | 61.5 | 28.9 | 33.3 |
| 429.mcf | Vehicle scheduling | 47.1 | 50 | 64.1 | 73.5 | 39 | 43.9 |
| 445.gobmk | Game AI | 20.2 | 31.3 | 41.4 | 43.1 | 22.4 | 27.7 |
| 456.hmmer | Protein seq. analyses | 19.1 | 27.1 | 28.6 | 22.5 | 24.2 | 28.4 |
| 458.sjeng | Chess | 17.1 | 25.4 | 32.6 | 33.1 | 24.8 | 28.3 |
| 462.libquantum | Quantum sim |
44.7 | 82.1 | 109 | 108 | 59.2 | 67.3 |
| 464.h264ref | Video encoding | 32.7 | 45.4 | 53.3 | 48.8 | 40.7 | 40.7 |
| 471.omnetpp | Network sim |
23.5 | 29.1 | 37.1 | 42.5 | 23.5 | 29.9 |
| 473.astar | Pathfinding | 16.5 | 24.8 | 33.5 | 36.9 | 18.9 | 23.6 |
| 483.xalancbmk | XML processing | 24.9 | 35.3 | 44.7 | 48.4 | 35.4 | 41.8 |
First we look at how well SMT-2, SMT-4 and SMT-8 work on the IBM POWER8.
| Subtest SPEC CPU2006 Integer |
Application Type |
IBM POWER8 10c@3.5 Single Thread |
IBM POWER8 10c@3.5 SMT-2 |
IBM POWER8 10c@3.5 SMT-4 |
IBM POWER8 10c@3.5 SMT-8 |
| 400.perlbench | Spam filter | N/A | N/A | N/A | N/A |
| 401.bzip2 | Compress | 100% | 154% | 193% | 201% |
| 403.gcc | Compiling | 100% | 139% | 176% | 192% |
| 429.mcf | Vehicle scheduling | 100% | 106% | 136% | 156% |
| 445.gobmk | Game AI | 100% | 155% | 205% | 213% |
| 456.hmmer | Protein seq. analyses | 100% | 142% | 150% | 118% |
| 458.sjeng | Chess | 100% | 149% | 191% | 194% |
| 462.libquantum | Quantum sim |
100% | 184% | 244% | 242% |
| 464.h264ref | Video encoding | 100% | 139% | 163% | 149% |
| 471.omnetpp | Network sim |
100% | 124% | 158% | 180% |
| 473.astar | Pathfinding | 100% | 150% | 203% | 224% |
| 483.xalancbmk | XML processing | 100% | 142% | 180% | 194% |
The performance gains from single threaded operation to two threads are very impressive, as expected. While Intel's SMT-2 offers in most subtests between 10 and 25% better performance, the dual threaded mode of the POWER8 boosts performance by 40 to 50% in most applications, or more than twice as much relative to the Xeons. Not one benchmark regresses when we throw 4 threads upon the IBM POWER8 core. The benchmarks with high IPC such as hmmer peak at SMT-4, but most subtests gain a few % when running 8 threads.
Facebook
Twitter
RSS
124 Comments
View All Comments
Kevin G - Tuesday, August 2, 2016 - link
"These oracle sparc m7 benchmarks vs IBM power8 are not worst case.">Eh? Did Oracle release the complete system configuration of the POWER8 for their testing? From your stream link you can find this PDF ( https://blogs.oracle.com/BestPerf/resource/stream/... ) where Oracle only test with 24 threads out of 96 possible in the environment and out of 192 possible supported with the hardware. This document does not detail how many cDIMMs were installed in a system which has a direct impact on available bandwidth. Case in point, the 512 GB of memory on the POWER8 system can be configured with the bare minimum number of cDIMMs in a system. That is a worst case scenario for POWER8 and we don't know if Oracle used it.
Oracle also made a source code change for STREAM for reverse allocation. The thing that is missing here is a comparison to the original code. This could impact how well prefetchers work and favor a particular architecture and thus impact performance. Thus we don't know if this change is a best or worst case scenario for comparison purposes.
"If you find other IBM power8 benchmarks I am sure oracle will compare to them instead. But you can only bench against ibm's own results, right?"
>I find it perfectly fair to use submitted benchmarks from IBM to compare against similarly configured systems submitted by Oracle. POWER8 systems are available with higher clocks and more cores than what is generally used in the open benchmarks IBM has submitted. Thus it is deceptive to claim that SPARC is decisively faster when there is beefier IBM hardware available.
"I am an sparc supporter. What is the problem with being an supporter?"
>Nothing inherently wrong with that but you are incredibly closed minded to any other alternative. You are blind to the idea that anything could be better or competitive in any metric. The reality of IT is that there no one tool that best fits every job. Anyone claiming otherwise is trying to sell you something.
"I would like anandtech to talk about the best CPU in the world instead of slow IBM power or Intel Xeon CPUs. But anandtech don't."
>How about you use your contacts at Oracle to get Anandtech a test system for some real independent analysis?
"I have never worked in IT."
>This explains a lot.
wingar - Saturday, July 23, 2016 - link
So, no one in the entire comment section mentioned SPARC at all. You come along, start ragging on POWER8, how SPARC is so much better, and then link to benchmarks on Oracle's website, with results provided by Oracle, with the conclusion of Oracle being so much better. Not only that, but the benchmarks you link require Oracle to use much higher end and incredibly higher cost hardware to beat low and mid-range POWER8 with.On top of all that you make dubious and unsubstantiated claims about server workloads and claims of performance of POWER8 and x86.
And finally to top it off, your comment is barely even related to the comment you replied to. It seems you picked the comment most visible in the thread to reply to.
So, to everyone else I think it's quite clear this is just an Oracle shill, please just ignore him.
wingar - Saturday, July 23, 2016 - link
So, adding on to this I was curious. I decided to make a simple google search, "site:anandtech.com brutalizer". What did I find? Comments on anything x86 and POWER8, every single one talking about how Oracle and SPARC are so much better than whatever the review is talking about. Consistently linking to Oracle-ran benchmarks on Oracles own site with the conclusion that Oracle is better. Consistently making dubious claims about the non-Oracle hardware. Every single comment I found shilling for SPARC, and every single one as close to the top of the comments list as possible. You seem to want to be as visible as possible.Have some links.
http://www.anandtech.com/comments/10158/the-intel-...
http://www.anandtech.com/comments/9193/the-xeon-e7...
http://www.anandtech.com/comments/10230/ibm-nvidia...
http://www.anandtech.com/comments/9567/the-power-8...
http://www.anandtech.com/comments/7757/quad-ivy-br...
http://www.anandtech.com/comments/7852/intel-xeon-...
http://www.anandtech.com/comments/7285/intel-xeon-...
Infact I found a couple of comments you left that *weren't* shilling. Have some links.
http://www.anandtech.com/comments/7334/a-look-at-a...
http://www.anandtech.com/comments/7371/understandi...
http://www.anandtech.com/comments/5831/amd-trinity...
It's hard to draw a conclusion from those two links but I'll point a few things. All of the non-shilling comments you made were in 2013. Every single pro-Oracle comment you made was at minimum 2014. Sounds to me like you were either bought out at that time, or you bought someone else's account, or perhaps this was the time you were put on Oracle's pay-cheque. It's quite possible that there's more comments that aren't shilling that I've missed here.
So, please. Try again.
Zetbo - Saturday, July 23, 2016 - link
He is a known Oracle Troll/Shill Kebbabert who is probably paid by Oracle to post crap all over internet. If he is not paid then thats just sad...wingar - Saturday, July 23, 2016 - link
Ohhh yes I am well aware, I encounter him on El Reg and other places all the time. But hey, I hate shills, so I'm quite happy to destroy any sense of credibility he may have for those not in the know.tipoo - Tuesday, July 26, 2016 - link
Ooh, good callout. It would almost be weirder if Oracle *didn't* pay him after all those links, lol.Kevin G - Wednesday, July 27, 2016 - link
Here is an interview with him (in Swedish) about how he was invited by then Sun to a party for his efforts:http://it24.idg.se/2.2275/1.202161/staende-ovation...
wingar - Thursday, July 28, 2016 - link
I'd call it sad, really. Very sad.alpha754293 - Wednesday, July 27, 2016 - link
Dude, SPARC sucks.Look at SWaP and TCO. Re-run your "analysis", it's obvious that SPARC sucks.
Can you even RUN Ubuntu on SPARC anymore?
Their FP performance sucks and it always have. That's why The Niagara T2 had to have FPUs ADDED to ALL of the cores because sharing a single FPU with 8 cores was a really bad/dumb idea.
I've looked at SPARC before. Had a couple of them and had a SunFire server before as well, and POWER/Intels can easily beat SPARC, especially once you consider TCO.
The company that I work for now (a Fortune 10 company) dumped all of the SPARC workstations for Intel.
RISC is RISKY! - Tuesday, August 2, 2016 - link
I would support "Brutalizer". Every processor has its strength and weakness. If memory architecture is considered, for the same capacity, Intel is conjested memory, IBM is very distributed and Oracle-Sun is something in between. So Intel will always have memory B/W problem every way. IBM has memory efficiency problem. Oracle in theory doesn't have problem, but with 2 dimm per ch, that look like have problem. Oracle-Sun is for highly branched workload in the real world. Intel is for 1T/Core more of single threaded workloads and IBM is for mixed workloads with 2T-4T/Core priority. So supercomputing workloads will work fast on IBM now, compared to intel and sparc, while analytics and graph and other distributed will work faster on SPARC M7 and S7 (although S7 is resource limited). While for intel, a soft mix of applications and highly customized os is better. Leave the business decisions and the sales price. List prices are twice as much as sales price in the real world. These three processors (xeon e5v4, power8-9, sparc m7-s7) are thoroughly tuned for different work spaces with very little overlap. So there's no point in comparing them other than their specs. Its like comparing a falcon and a lion and a swordfish. Their environments are different even though all of them hunt. Thats in the real world. So benchmarks are not the real proof. We at the university of IITD have lots and lots of intel xeon e5v4, some P8 (10-15 single and dual sockets), and a very few (1-2 two socket M7 and 2 two socket S7). We run anything and every thing on any of these, we get our hands on. And this is the real world conclusion. So don't fight. Its a context centric supply of processors!