Assessing IBM's POWER8, Part 1: A Low Level Look at Little Endian
by Johan De Gelas on July 21, 2016 8:45 AM ESTMemory Subsystem: Bandwidth
As we mentioned before, the IBM POWER8 has a memory subsystem which is more similar to the Xeon E7's than the E5's. The IBM POWER8 connects to 4 "Centaur" buffer cache chips, which have both a 19.2 GB/s read and 9.6 GB/s write link to the processor, or 28.8 GB/s in total. So the 105 GB/s aggregate bandwidth of the POWER8 is not comparable to Intel's peak bandwidth. Intel's peak bandwidth is the result of 4 channels of DDR4-2400 that can either write or read at 76.8 GB/s (2.4 GHz x 8 bytes per channel x 4 channels).
Bandwidth is of course measured with John McCalpin's Stream bandwidth benchmark. We compiled the stream 5.10 source code with gcc 5.2.1 64 bit. The following compiler switches were used on gcc:
-Ofast -fopenmp -static -DSTREAM_ARRAY_SIZE=120000000
The latter option makes sure that stream tests with array sizes which are not cacheable by the Xeons' huge L3 caches.
It is important to note why we use the GCC compiler and not vendors' specialized compilers: the GCC compiler is not as good at vectorizing the code. Intel's ICC compiler does that very well, and as result shows the bandwidth available to highly optimized HPC code, which is great for that code in the real world, but it's not realistic for multi-threaded server applications.
With ICC, Intel can use the very wide 256-bit load units to their full potential and we measured up to 65 GB/s per socket. But you also have to consider that ICC is not free, and GCC is much easier to integrate and automate into the daily operations of any developer. No licensing headaches, no time consuming registrations.
The combination of the powerful four load and two store subsystem of the POWER8 and the read/write interconnect between the CPU and the Centaur chips makes it much easier to offer more bandwidth. The IBM POWER8 delivers a solid 90 GB/s despite using old DDR3-1333 memory technology.
Intel claims higher bandwidth numbers, but those numbers can only be delivered in vectorized software.
Facebook
Twitter
RSS
124 Comments
View All Comments
Kevin G - Tuesday, August 2, 2016 - link
"These oracle sparc m7 benchmarks vs IBM power8 are not worst case.">Eh? Did Oracle release the complete system configuration of the POWER8 for their testing? From your stream link you can find this PDF ( https://blogs.oracle.com/BestPerf/resource/stream/... ) where Oracle only test with 24 threads out of 96 possible in the environment and out of 192 possible supported with the hardware. This document does not detail how many cDIMMs were installed in a system which has a direct impact on available bandwidth. Case in point, the 512 GB of memory on the POWER8 system can be configured with the bare minimum number of cDIMMs in a system. That is a worst case scenario for POWER8 and we don't know if Oracle used it.
Oracle also made a source code change for STREAM for reverse allocation. The thing that is missing here is a comparison to the original code. This could impact how well prefetchers work and favor a particular architecture and thus impact performance. Thus we don't know if this change is a best or worst case scenario for comparison purposes.
"If you find other IBM power8 benchmarks I am sure oracle will compare to them instead. But you can only bench against ibm's own results, right?"
>I find it perfectly fair to use submitted benchmarks from IBM to compare against similarly configured systems submitted by Oracle. POWER8 systems are available with higher clocks and more cores than what is generally used in the open benchmarks IBM has submitted. Thus it is deceptive to claim that SPARC is decisively faster when there is beefier IBM hardware available.
"I am an sparc supporter. What is the problem with being an supporter?"
>Nothing inherently wrong with that but you are incredibly closed minded to any other alternative. You are blind to the idea that anything could be better or competitive in any metric. The reality of IT is that there no one tool that best fits every job. Anyone claiming otherwise is trying to sell you something.
"I would like anandtech to talk about the best CPU in the world instead of slow IBM power or Intel Xeon CPUs. But anandtech don't."
>How about you use your contacts at Oracle to get Anandtech a test system for some real independent analysis?
"I have never worked in IT."
>This explains a lot.
wingar - Saturday, July 23, 2016 - link
So, no one in the entire comment section mentioned SPARC at all. You come along, start ragging on POWER8, how SPARC is so much better, and then link to benchmarks on Oracle's website, with results provided by Oracle, with the conclusion of Oracle being so much better. Not only that, but the benchmarks you link require Oracle to use much higher end and incredibly higher cost hardware to beat low and mid-range POWER8 with.On top of all that you make dubious and unsubstantiated claims about server workloads and claims of performance of POWER8 and x86.
And finally to top it off, your comment is barely even related to the comment you replied to. It seems you picked the comment most visible in the thread to reply to.
So, to everyone else I think it's quite clear this is just an Oracle shill, please just ignore him.
wingar - Saturday, July 23, 2016 - link
So, adding on to this I was curious. I decided to make a simple google search, "site:anandtech.com brutalizer". What did I find? Comments on anything x86 and POWER8, every single one talking about how Oracle and SPARC are so much better than whatever the review is talking about. Consistently linking to Oracle-ran benchmarks on Oracles own site with the conclusion that Oracle is better. Consistently making dubious claims about the non-Oracle hardware. Every single comment I found shilling for SPARC, and every single one as close to the top of the comments list as possible. You seem to want to be as visible as possible.Have some links.
http://www.anandtech.com/comments/10158/the-intel-...
http://www.anandtech.com/comments/9193/the-xeon-e7...
http://www.anandtech.com/comments/10230/ibm-nvidia...
http://www.anandtech.com/comments/9567/the-power-8...
http://www.anandtech.com/comments/7757/quad-ivy-br...
http://www.anandtech.com/comments/7852/intel-xeon-...
http://www.anandtech.com/comments/7285/intel-xeon-...
Infact I found a couple of comments you left that *weren't* shilling. Have some links.
http://www.anandtech.com/comments/7334/a-look-at-a...
http://www.anandtech.com/comments/7371/understandi...
http://www.anandtech.com/comments/5831/amd-trinity...
It's hard to draw a conclusion from those two links but I'll point a few things. All of the non-shilling comments you made were in 2013. Every single pro-Oracle comment you made was at minimum 2014. Sounds to me like you were either bought out at that time, or you bought someone else's account, or perhaps this was the time you were put on Oracle's pay-cheque. It's quite possible that there's more comments that aren't shilling that I've missed here.
So, please. Try again.
Zetbo - Saturday, July 23, 2016 - link
He is a known Oracle Troll/Shill Kebbabert who is probably paid by Oracle to post crap all over internet. If he is not paid then thats just sad...wingar - Saturday, July 23, 2016 - link
Ohhh yes I am well aware, I encounter him on El Reg and other places all the time. But hey, I hate shills, so I'm quite happy to destroy any sense of credibility he may have for those not in the know.tipoo - Tuesday, July 26, 2016 - link
Ooh, good callout. It would almost be weirder if Oracle *didn't* pay him after all those links, lol.Kevin G - Wednesday, July 27, 2016 - link
Here is an interview with him (in Swedish) about how he was invited by then Sun to a party for his efforts:http://it24.idg.se/2.2275/1.202161/staende-ovation...
wingar - Thursday, July 28, 2016 - link
I'd call it sad, really. Very sad.alpha754293 - Wednesday, July 27, 2016 - link
Dude, SPARC sucks.Look at SWaP and TCO. Re-run your "analysis", it's obvious that SPARC sucks.
Can you even RUN Ubuntu on SPARC anymore?
Their FP performance sucks and it always have. That's why The Niagara T2 had to have FPUs ADDED to ALL of the cores because sharing a single FPU with 8 cores was a really bad/dumb idea.
I've looked at SPARC before. Had a couple of them and had a SunFire server before as well, and POWER/Intels can easily beat SPARC, especially once you consider TCO.
The company that I work for now (a Fortune 10 company) dumped all of the SPARC workstations for Intel.
RISC is RISKY! - Tuesday, August 2, 2016 - link
I would support "Brutalizer". Every processor has its strength and weakness. If memory architecture is considered, for the same capacity, Intel is conjested memory, IBM is very distributed and Oracle-Sun is something in between. So Intel will always have memory B/W problem every way. IBM has memory efficiency problem. Oracle in theory doesn't have problem, but with 2 dimm per ch, that look like have problem. Oracle-Sun is for highly branched workload in the real world. Intel is for 1T/Core more of single threaded workloads and IBM is for mixed workloads with 2T-4T/Core priority. So supercomputing workloads will work fast on IBM now, compared to intel and sparc, while analytics and graph and other distributed will work faster on SPARC M7 and S7 (although S7 is resource limited). While for intel, a soft mix of applications and highly customized os is better. Leave the business decisions and the sales price. List prices are twice as much as sales price in the real world. These three processors (xeon e5v4, power8-9, sparc m7-s7) are thoroughly tuned for different work spaces with very little overlap. So there's no point in comparing them other than their specs. Its like comparing a falcon and a lion and a swordfish. Their environments are different even though all of them hunt. Thats in the real world. So benchmarks are not the real proof. We at the university of IITD have lots and lots of intel xeon e5v4, some P8 (10-15 single and dual sockets), and a very few (1-2 two socket M7 and 2 two socket S7). We run anything and every thing on any of these, we get our hands on. And this is the real world conclusion. So don't fight. Its a context centric supply of processors!