Assessing IBM's POWER8, Part 1: A Low Level Look at Little Endian
by Johan De Gelas on July 21, 2016 8:45 AM ESTMemory Subsystem: Bandwidth
As we mentioned before, the IBM POWER8 has a memory subsystem which is more similar to the Xeon E7's than the E5's. The IBM POWER8 connects to 4 "Centaur" buffer cache chips, which have both a 19.2 GB/s read and 9.6 GB/s write link to the processor, or 28.8 GB/s in total. So the 105 GB/s aggregate bandwidth of the POWER8 is not comparable to Intel's peak bandwidth. Intel's peak bandwidth is the result of 4 channels of DDR4-2400 that can either write or read at 76.8 GB/s (2.4 GHz x 8 bytes per channel x 4 channels).
Bandwidth is of course measured with John McCalpin's Stream bandwidth benchmark. We compiled the stream 5.10 source code with gcc 5.2.1 64 bit. The following compiler switches were used on gcc:
-Ofast -fopenmp -static -DSTREAM_ARRAY_SIZE=120000000
The latter option makes sure that stream tests with array sizes which are not cacheable by the Xeons' huge L3 caches.
It is important to note why we use the GCC compiler and not vendors' specialized compilers: the GCC compiler is not as good at vectorizing the code. Intel's ICC compiler does that very well, and as result shows the bandwidth available to highly optimized HPC code, which is great for that code in the real world, but it's not realistic for multi-threaded server applications.
With ICC, Intel can use the very wide 256-bit load units to their full potential and we measured up to 65 GB/s per socket. But you also have to consider that ICC is not free, and GCC is much easier to integrate and automate into the daily operations of any developer. No licensing headaches, no time consuming registrations.
The combination of the powerful four load and two store subsystem of the POWER8 and the read/write interconnect between the CPU and the Centaur chips makes it much easier to offer more bandwidth. The IBM POWER8 delivers a solid 90 GB/s despite using old DDR3-1333 memory technology.
Intel claims higher bandwidth numbers, but those numbers can only be delivered in vectorized software.
Facebook
Twitter
RSS
124 Comments
View All Comments
tipoo - Thursday, July 21, 2016 - link
They made PowerPC Windows? Source? I remember the Powermac G5s were the early dev kits for the xbox 360 due to the architecture similarity, but I assumed those stories meant they were just working in OSX or Linux on them.thunderbird32 - Thursday, July 21, 2016 - link
AFAIK, the last build of Windows for PPC was NT 4. So, it's been a while.Sunner - Thursday, July 21, 2016 - link
There were early builds of Windows 2000 for the RISC's as well, during the times when it was still called NT5. I had one of those from WinHEC, but alas I lost it when moving at some point. :(yuhong - Thursday, July 21, 2016 - link
AFAIK, the little endian PowerPC mode that NT4 used was killed when they went to 64-bit and is different from today's POWER8 little endian mode that was only recently introduced.Kevin G - Thursday, July 21, 2016 - link
I used to have such a disc for Windows NT4. That disk also had binaries for DEC Alpha and MIPS.BillyONeal - Thursday, July 21, 2016 - link
The Xbox 360 is a PPC machine, and runs a (heavily modified) version of Windows. My understanding is that most x86 assumptions had to be ferreted out to run on Itanium (early) and then on ARM (later).Einy0 - Thursday, July 21, 2016 - link
MS has builds that will run on anything. The real question is why would you want to? These chips are designed from the ground up to run massive work loads. It's a completely different style of computing than a Windows machine. Even MS server OSes aren't designed for this type of work. We are talking Banking, ERP and other big data applications. MS is still dreaming about scaling on that level. Right now their answer is clustering but that comes with it's own obstacles too.abufrejoval - Thursday, August 4, 2016 - link
Well there is always QEMU.And IBM has a much better binary translator from when they bought QuickTransit. That one originally translated Power to x86 for the Mac, then Sparc to x86 for Quicktransit and eventually x86 to Power for IBM so they could run Linux workloads on AIX.
Then what exactly do you mean with Windows (assuming this is actually a reasonable question)?
Server applications or desktop?
.NET has been ported to Linux and I guess could be made to run on Power. A Power runtime could certainly be done by Microsoft, if they wanted to.
I don't see why anyone would want to run Windows desktop workloads on this hardware, other than to show that it can be done: QEMU to that!
BedfordTim - Thursday, July 21, 2016 - link
I was intrigued to see how little effect hyper-threading with your Xeon. My own experience is that it gives a 50% boost although I appreciate there are many variables.Taracta - Thursday, July 21, 2016 - link
Something seems to be wrong with the Mem Hierarchy charts in the Intel L3 and 16MB section.