Assessing IBM's POWER8, Part 1: A Low Level Look at Little Endian
by Johan De Gelas on July 21, 2016 8:45 AM EST
It is the widest superscalar processor on the market, one that can issue up to 10 instructions and sustain 8 per clock: IBM's POWER8. IBM's POWER CPUs have always captured the imagination of the hardware enthusiast; it is the Tyrannosaurus Rex, the M1 Abrams of the processor world. Still, despite a flood of benchmarks and reports, it is very hard to pinpoint how it compares to the best Intel CPUs in performance wise. We admit that our own first attempt did not fully demystify the POWER8 either, due to the fact that some immature LE Linux software components (OpenJDK, MySQL...) did not allow us to run our enterprise workloads.
Hence we're undertaking another attempt to understand what the strengths and weaknesses are of Intel's most potent challenger. And we have good reasons besides curiosity and geekiness: IBM has just recently launched the IBM S812LC, the most affordable IBM POWER based server ever. IBM advertises the S812LC with "Starting at $4,820". That is pretty amazing if you consider that this is not some basic 1U server, but a high expandable 2U server with 32 (!) DIMM slots, 14 disk bays, 4 PCIe Gen 3 slots, and 2 redundant power supplies.
Previous "scale out" models SL812 and SL822 were competitively priced too ... until you start populating the memory slots! The required CDIMMs cost no less than 4(!) times more than RDIMMs, which makes those servers very unattractive for the price conscious buyers that need lots of memory. The S812LC does not have that problem: it makes use of cheap DDR3 RDIMMs. And when you consider that the actual street prices are about 20-25% lower, you know that IBM is in Dell territory. There is more: servers from Inventec, Inspur, and Supermicro are being developed, so even more affordable POWER8 servers are on the way. A POWER8 server is thus quite affordable now, and it looks like the trend is set.
To that end, we decided that we want to more accurately measure how the POWER8 architecture compares to the latest Xeons. In this first article we are focusing on characterizing the microarchitecture and the "raw" integer performance. Although the POWER8 architecture has been around for 2 years now, we could not find any independent Little Endian benchmark data that allowed us to compare POWER8 processors with Intel's Xeon processors in a broad range of applications.
Notice our emphasis on "Little Endian". In our first review, we indeed tested on a relatively immature LE Ubuntu 14.04 for OpenPOWER. Some people felt that this was not fair as the POWER8 would do a lot better on top of a Big Endian operating system simply because of the software maturity. But the market says otherwise: if IBM does not want to be content with fighting Oracle in an ever shrinking high-end RISC market, they need to convince the hyper scalers and the thousands of smaller hosting companies. POWER8 Server will need to find a place inside their x86 dominated datacenters. A rich LE Linux software ecosystem is the key to open the door to those datacenters.
When it comes to taking another crack at our testing, we found out that running Ubuntu 15.10 (16.04 was just out yet when we started testing) solved a lot of the issues (OpenJDK, MySQL) that made our previous attempt at testing the POWER8 so hard and incomplete. Therefore we felt that despite 2 years of benchmarking on POWER8, an independent LE Linux-focused article could still add value.
Facebook
Twitter
RSS
124 Comments
View All Comments
HellStew - Wednesday, July 27, 2016 - link
It depends what kind of software you are running. If you are running giant backend workloads on x86, you can seamlessly migrate that data to PPC while keeping custom front ends running on x86.aryonoco - Saturday, July 23, 2016 - link
Johan, maybe the little endian-ness makes a difference in porting proprietary software, but pretty much all open source software on Linux has supported BE POWER for a long time.If you get the time and the inclination Johan, it would be great if you could say do some benchmarks on BE RHEL 7 vs LE RHEL 7 on the same POWER 8 system. I think it would make for fascinating reading in itself, and would show if there are any differences when POWER operates in BE mode vs LE mode.
aryonoco - Saturday, July 23, 2016 - link
Actually scrap that, seems like IBM is fully focusing on LE for Linux on POWER in future. I'm not sure there will be many BE Linux distributions officially supporting POWER9 anyway. So your choice of focusing on LE Linux on POWER is fully justified.HellStew - Wednesday, July 27, 2016 - link
Side note: Once you are running KVM, you can run any mix of BE and LE linux varieties side by side. I'm running FedoraBE, SuSE BE, Ubuntu LE, CentOS LE, and (yes a very slow copy of windows) on one of these chipsrootbeerrail - Saturday, July 23, 2016 - link
If a machine is completely isolated, it doesn't matter much to the machine. I personally find BE easier to read in hex dumps because it follows the left-to-right nature of English numbers, but there are reasons to use LE for human understanding as well.The problem shows up the instance one tries to interchange binary data. If the endian order does not match, the data is going to get scrambled. Careful programming can work around this issue, but not everyone is a careful programmer - there's a lot of 'get something out the door' from inexperienced or lazy people. If everything is using the same conventions (not only endian, but size of the binary data types (less of a problem now that most everything has converged to 64-bit)), it's not an issue. Thus having LE on Power makes the interchange of binary data easier with the X86 world.
errorr - Friday, July 22, 2016 - link
Great Article! Just an FYI, the term "just" as in "just out" on the first page has different meanings on opposite sides of the Atlantic and is usually avoided in writing for international audiences. I'm not quite sure which one is used her. The NaE would mean 'just out' in that it had come out right before while the BrE would mean it came out right after the time period referenced in the sentence.xCalvinx - Friday, July 22, 2016 - link
awesome!!..keepup the good work..looking forward to Part2!! ... actualy cant wait.. hurryup lol.. :)double thumbsup
Mpat - Friday, July 22, 2016 - link
Skylake does not have 5 decoders, it is still 4. I know that that segment of the optimization manual is written in a cryptic way, but this's what actually happened: up until Broadwell there are 4 decoders and a max bandwidth from the decoder segment of 4 uops. If the first decoder (the complex one) produces 4 uops from one x86 op, the other decoders can't work. If the first produces 3, then the second can produce 1, etc. this means that the decoders can produce one of these combinations of uops from an x86 op, depending on how complex a task the first decoder has: 1/1/1/1, 2/1/1, 3/1, or 4. Skylake changes this so the max bandwidth from that segment is now 5, and the legal combinations become 1/1/1/1, 2/1/1/1, 3/1/1, and 4/1. You still can't do 1/1/1/1/1, so there is still only 4 decoders. Make sense?ReaperUnreal - Friday, July 22, 2016 - link
Why do the tests with GCC? Why not give each platform their full advantage and go with ICC on Intel and xLC on Power? The compiler can make a HUGE difference with benchmarks.Michael Bay - Saturday, July 23, 2016 - link
It`s right in the text why.