The IBM POWER8 Review: Challenging the Intel Xeon
by Johan De Gelas on November 6, 2015 8:00 AM EST- Posted in
- IT Computing
- CPUs
- Enterprise
- Enterprise CPUs
- IBM
- POWER
- POWER8
Energy and Pricing
Unfortunately, accurately and fairly comparing energy consumption at the system level between the S822L and other systems wasn't something we were able to do, as there were quite a few differences in the hardware configuration. For example, the IBM S822L had two SAS controllers and we had no idea how power hungry that chip under the copper heatsink was. Still there is no doubt that the dual CPU system is by far the most important power consumer when the server system is under load. In case of the IBM system, the Centaur chips will take their fair share too, but those chips are not optional. So we can only get a very rough idea how the power consumption compares.
| Xeon E5 299 v3/POWER8 Comparison (System) | ||
| Feature | 2x Xeon E5-2699v3 | 2x IBM POWER8 3.4 10c IBM S822L |
| Idle | 110-120W | 360-380W |
|
Running NAMD (FP) |
540-560W |
700-740W |
| Running 7-zip (Integer) |
300-350W |
780-800W |
The Haswell core was engineered for mobile use, and there is no denying that Intel's engineers are masters at saving power at low load.
The mightly POWER8 is cooled by a huge heatsink
IBM's POWER8 has pretty advanced power management, as besides p-states, power gating cores and the associated L3-cache should be possible. However, it seems that these features were not enabled out-of-the box for some reason as idle power was quite high. To be fair, we spent much more time on getting our software ported and tuned than on finding the optimal power settings. In the limited time we had with the machine, producing some decent benchmarking numbers was our top priority.
Also, the Centaur chips consume about 16W per chip (Typical, 20W TDP) and as we had 8 of them inside our S822L, those chips could easily be responsible for consuming around 100W.
Interestingly, the IBM POWER8 consumes more energy processing integers than floating point numbers. Which is the exact opposite of the Xeon, which consumes vastly more when crunching AVX/FP code.
Pricing
Though the cost of buying a system might be only "a drop in the bucket" in the total TCO picture in traditional IT departements running expensive ERP applications, it is an important factor for almost everybody else who buys Xeon systems. It is important to note that the list prices of IBM on their website are too high. It is a bad habit of a typical tier-one OEM.
Thankfully we managed to get some "real street prices", which are between 30% (one server) and 50% (many) lower. To that end we compared the price of the S822L with a discounted DELL R730 system. The list below is not complete, as we only show the cost of the most important components. The idea is to focus on the total system price and show which components contribute the most to the total system cost.
| Xeon E7v3/POWER8 Price Comparison | ||||
| Feature | Dell R730 | IBM S822L | ||
| Type | Price | Type | Price | |
| Chassis | R730 | N/A | S822L | N/A |
| Processor | 2x E5-2697 | $5000 | 2x POWER8 3.42 | $3000 |
| RAM | 8x 16GB DDR4 DIMM |
$2150 | 8x 16 GB CDIMM (DDR3) | $8000 |
| PSU | 2x 1100W | $500 | 2x 1400W | $1000 |
| Disks | SATA or SSD | Starting at $200 |
SAS HD/SSD | +/- $450 |
| Total system price (approx.) | $10k | $15k | ||
With more or less comparable specs, the S822L was about 50% more expensive. However, it was almost impossible to make an apples-to-apples comparison. The biggest "price issue" are the CDIMMs, which are almost 4 times as expensive as "normal" RDIMMs. CDIMMs offer more as they include an L4-cache and some extra features (such as a redundant memory chip for each 9 chips). For most typical current Xeon E5 customers, the cost issue will be important. For a few, the extra redundancy and higher bandwidth will be interesting. Less important, but still significant is the fact that IBM uses SAS disks, which increase the cost of the storage system, especially if you want lots of them.
This cost issue will be much less important on most third party POWER8 systems. Tyan's "Habanero" system for example integrates the Centaur chips on the motherboard, making the motherboard more expensive but you can use standard registered DDR3L RDIMMs, which are much cheaper. Meanwhile the POWER8 processor tends to be very reasonably priced, at around $1500. That is what Dell would charge for an Intel Xeon E5-2670 (12 cores at 2.3-2.6 GHz, 120W). So while Intel's Xeon are much more power efficient than the POWER8 chips, the latter tends to be quite a bit cheaper.
Facebook
Twitter
RSS
146 Comments
View All Comments
JohanAnandtech - Saturday, November 7, 2015 - link
suggestions on how to to do this? OpenSSL 1.02 will support the build in crypto accelerator, but I am not sure on how I would be able to see if the crypto code uses VMX.SarahKerrigan - Monday, November 9, 2015 - link
Compile with -qreport in XL C/C++.Oxford Guy - Saturday, November 7, 2015 - link
Typo on page 2:The resuls are that Google is supporting the efforts and Rackspace has even build their own OpenPOWER server called "Barreleye".
Ryan Smith - Saturday, November 7, 2015 - link
Thanks.iwod - Saturday, November 7, 2015 - link
In terms of 100, POWER Software Ecosystem manage to scale from 10 to 20, so that is a 100% increase but still very very low. Will we see POWER CPU / Server that is cheap enough to compete with Xeon E3 / E5, where most of the volume are? Compared to E7 is like comparing Server CPU for the 10% of the market.Intel will be moving to 14nm E7, I don't see anyone making POWER CPU at 14nm anytime soon.
Intel DC business are growing, and it desperately need a competitor, such as POWER to combat E7 and AMD Zen from the bottom.
Frenetic Pony - Saturday, November 7, 2015 - link
Nice review! It just confirms my question however of "What does IBM do?" Seriously, what do they do anymore? All I see are headlines for things that never come out as actual products. Their servers suck up too much power per watt, they don't have their own semi conductor foundries, their semi conductor research seems like a bunch of useless paper tiger stuff, their much vaunted AI is better at playing Jeapordy than seemingly any real world use.Countdown to complete IBM bankruptcy/spinoff/selloff is closer than ever.
ws3 - Saturday, November 7, 2015 - link
Since the dawn of computing, IBM has been in the business of providing solutions, rather than merely hardware. When you buy IBM you pay a huge amount of money, and what you get for that is support, with some hardware thrown in.Obviously this only appeals to wealthy customers who don't have or don't want to have an internal support organization that can duplicate what IBM offers. It seems to me that the number of such customers is decreasing over time, but as long as the US government is around, IBM will have at least one customer.
xype - Sunday, November 8, 2015 - link
They make 2-5 Billion dollars of profit per quarter. "Countdown to complete IBM bankruptcy/spinoff/selloff is closer than ever." my ass.PowerTrumps - Sunday, November 8, 2015 - link
Pretty fair and even handed review; don't agree with it all and definitely feel there is room to learn and improve. Btw, full disclosure, I am a System Architect focusing on Power technology for a Business Partner.With regard to compilers I would suggest IBM's SDK for Linux on Power & Advanced Tool Chain (ATC) provide development tools and open source optimized dev stack (ie gcc) for POWER8. Details at: https://www-304.ibm.com/webapp/set2/sas/f/lopdiags... and https://www.ibm.com/developerworks/community/wikis...
MySQL is definitely relevant but with the new Linux distro's packaging MariaDB in place of MySQL I would have liked to see an Intel vs Power comparison with this MySQL alternative. MariaDB just announced v10.1 is delivering over 1M queries per second on POWER8. https://blog.mariadb.org/10-1-mio-qps/
A commenter asked about Spark with POWER8. This blog discusses how it performs vs Intel. https://www.ibm.com/developerworks/community/blogs...
In addition to the commercial benchmarks often quoted such as SPEC, SAP and TPC like this SAP HANA result with SUSE on POWER8 ; SAP BW-EML (ie HANA) shows tremendous scaling with POWER8. http://www.smartercomputingblog.com/power-systems/... many of the ISV's have produced their own. I have seen results for PostgreSQL, STAC (http://financial.mcobject.com/press-release-novemb... Redis Labs, etc.
Benchmarks are great, all vendors do them and most people realize you should take them with a grain of salt. One benefit of Power servers when using PowerVM, its native firmware based hypervisor is that it delivers tremendous compute efficiency to VM's. On paper things like TDP seem higher for Power vs Intel (especially E5_v3 chips) but when Power servers deliver consolidation ratio's with 2-4X (and greater) more VM's per core the TCA & TCO get real interesting. One person commented how SAP on Power would blow out a budget. It does just the opposite because how you can run in a Tier-2 architecture obtaining intra-server VM to VM efficiencies, compute efficiencies with fewer cores & servers which impacts everything in the datacenter. Add in increased reliability & serviceability features and you touch the servers less which means your business is running longer.
And for more details on the open platform or those based on the OpenPOWER derivative using the "LC" designator such as S822LC in contrast to the S822L used as the focus in this article. http://www.smartercomputingblog.com/power-systems/... and http://businesspartnervoices.com/ibm-power-systems...
JohanAnandtech - Sunday, November 8, 2015 - link
Great feedback. We hope to get access to another POWER8(+) server and build further upon our existing knowledge. We have real world experience with Spark, so it is definitely on the list. The blog you linked seems to have used specific SPARK optimization for POWER, but the x86 reference system looks a bit "neglected". A real independent test would be very valuable there. The interesting part of Spark is that a good benchmark would be also very relevant for the real world as peak performance is one of the most important aspects of Spark, in contrast with databases where maximum performance is only a very small part of the experience.About MySQL, people have pointed out that the 5.7 version seems to scale a lot better, so that is together with MariaDB also on my "to test" list. Redis does not seem relevant for this kind of machine, it is single-threaded, almost impossible to test 160 instances.
The virtualization part is indeed one of the most interesting parts, but it is a benchmarking nightmare. You got to keep response times at more or less the same levels while loading the machine with more and more VMs. We did that kind of testing until 2 years ago on x86, but it was very time consuming and we had a deep understanding on how vSphere worked. Building that kind of knowledge on PowerVM might be beyond our manpower and time :-).