The IBM POWER8 Review: Challenging the Intel Xeon
by Johan De Gelas on November 6, 2015 8:00 AM EST- Posted in
- IT Computing
- CPUs
- Enterprise
- Enterprise CPUs
- IBM
- POWER
- POWER8
The L4-cache and Memory Subsystem
Each POWER8 memory controller has access to four "Custom DIMMs" or CDIMMs. Each CDIMMs is in fact a "Centaur" chip and 40 to 80 DRAM chips. The Centaur chip contains the DDR3 interfaces, the memory management logic and a 16 MB L4-cache.
The 16 MB L4-cache is eDRAM technology like the on-die L3-cache. Let us see how the CDIMMs look in reality.
Considering that 4Gb DRAM chips were available in mid 2013, the 1600 MHz 2Gb DRAM chips used here look a bit outdated. Otherwise the (much) more expensive 64GB CDIMMs use the current 4Gb DRAM chips. The S822L has 16 slots and can thus use up to 1TB (64GB x 16) in DIMMs.
Considering that many Xeon E5 servers are limited to 768 GB, 1 TB is more than competitive. Some Xeon E5 servers can reach 1.5 TB with 64 GB LR-DIMMs but not every server supports this rather expensive memory technology. It is very easy to service the CDIMMs: a gentle push on the two sides will allow you to slide them out. The black pieces of plastic between the CDIMMS are just place-holders that protect the underlying memory slots. For our testing we had CDIMMs installed in 8 of our system's 16 slots.
The Centaur chip acts as a 16MB L4-cache to save memory accesses and thus energy, but it needs quite a bit of power (10-20 W) itself and as a result is covered by heatsink. CDIMMs have ECC enabled (8+1 for ECC) and have also an extra spare DRAM chip. As result, a CDIMM has 10 DRAM chips while offering capacity of 8 chips.
That makes the DRAM subsystem of the S822L much more similar to the E7 memory subsystem with the "Scalable memory interconnect" and "Jordan Creek" memory buffer technology than to the typical Xeon E5 servers.
Facebook
Twitter
RSS
146 Comments
View All Comments
joegee - Thursday, November 19, 2015 - link
It was an awesome community. I learned so much from everyone. I remember the days when we'd write pages arguing whether AMD's new 64 bit extension to x86 was truly 64 bit. The discussions could be heated, but they were seldom rude. I wish there were something similar today. :/Kevin G - Saturday, November 7, 2015 - link
Aces brings back memories for me as well even though I mainly lurked there.A solid chunk of that group have moved over to RWT.
joegee - Thursday, November 19, 2015 - link
What is RWT?psychobriggsy - Friday, November 6, 2015 - link
Get back to Aces Hardware you!JohanAnandtech - Saturday, November 7, 2015 - link
Like Ryan said, I have been working 11 years at Anand. In other words, it is great working at Anandtech. AT is one of the few tech sites out there that still values deep analysis and allows the editors to take the time to delve deep.joegee - Friday, November 6, 2015 - link
And still writing as well as you ever did! Keep up the good work, Johan!rrossi - Saturday, November 7, 2015 - link
Dear Johan nice article. Did u ever consider sparse system solving (with preconditioning) as a sensitive benchmark? It is a crucial stage of most scientific applications and it is a bandwidth limited operation with a high degree of parallelism. It would be definitely interesting to see how the power 8 fares on such a test. If you are interested I think I could provide a pointer to a simple benchmark (to be compiled). If you feel it may be interesting just drop me an email.JohanAnandtech - Saturday, November 7, 2015 - link
Interested... mail me, I don't have your mail. See the author link on top of the article.Ian Cutress - Saturday, November 7, 2015 - link
I'd also like to be pointed to such a benchmark for workstation style tests on x86. Please email ian@anandtech.com with info :)MartinT - Friday, November 6, 2015 - link
Johan's been with Anandtech for more than a decade, and has been publishing on the subject since the late 90s.But I very much second your "Niiiiice!," as reading his name always reminds me of the old days over at aceshardware, and I'm always looking forward to his insights!