The IBM POWER8 Review: Challenging the Intel Xeon
by Johan De Gelas on November 6, 2015 8:00 AM EST- Posted in
- IT Computing
- CPUs
- Enterprise
- Enterprise CPUs
- IBM
- POWER
- POWER8
Energy and Pricing
Unfortunately, accurately and fairly comparing energy consumption at the system level between the S822L and other systems wasn't something we were able to do, as there were quite a few differences in the hardware configuration. For example, the IBM S822L had two SAS controllers and we had no idea how power hungry that chip under the copper heatsink was. Still there is no doubt that the dual CPU system is by far the most important power consumer when the server system is under load. In case of the IBM system, the Centaur chips will take their fair share too, but those chips are not optional. So we can only get a very rough idea how the power consumption compares.
| Xeon E5 299 v3/POWER8 Comparison (System) | ||
| Feature | 2x Xeon E5-2699v3 | 2x IBM POWER8 3.4 10c IBM S822L |
| Idle | 110-120W | 360-380W |
|
Running NAMD (FP) |
540-560W |
700-740W |
| Running 7-zip (Integer) |
300-350W |
780-800W |
The Haswell core was engineered for mobile use, and there is no denying that Intel's engineers are masters at saving power at low load.
The mightly POWER8 is cooled by a huge heatsink
IBM's POWER8 has pretty advanced power management, as besides p-states, power gating cores and the associated L3-cache should be possible. However, it seems that these features were not enabled out-of-the box for some reason as idle power was quite high. To be fair, we spent much more time on getting our software ported and tuned than on finding the optimal power settings. In the limited time we had with the machine, producing some decent benchmarking numbers was our top priority.
Also, the Centaur chips consume about 16W per chip (Typical, 20W TDP) and as we had 8 of them inside our S822L, those chips could easily be responsible for consuming around 100W.
Interestingly, the IBM POWER8 consumes more energy processing integers than floating point numbers. Which is the exact opposite of the Xeon, which consumes vastly more when crunching AVX/FP code.
Pricing
Though the cost of buying a system might be only "a drop in the bucket" in the total TCO picture in traditional IT departements running expensive ERP applications, it is an important factor for almost everybody else who buys Xeon systems. It is important to note that the list prices of IBM on their website are too high. It is a bad habit of a typical tier-one OEM.
Thankfully we managed to get some "real street prices", which are between 30% (one server) and 50% (many) lower. To that end we compared the price of the S822L with a discounted DELL R730 system. The list below is not complete, as we only show the cost of the most important components. The idea is to focus on the total system price and show which components contribute the most to the total system cost.
| Xeon E7v3/POWER8 Price Comparison | ||||
| Feature | Dell R730 | IBM S822L | ||
| Type | Price | Type | Price | |
| Chassis | R730 | N/A | S822L | N/A |
| Processor | 2x E5-2697 | $5000 | 2x POWER8 3.42 | $3000 |
| RAM | 8x 16GB DDR4 DIMM |
$2150 | 8x 16 GB CDIMM (DDR3) | $8000 |
| PSU | 2x 1100W | $500 | 2x 1400W | $1000 |
| Disks | SATA or SSD | Starting at $200 |
SAS HD/SSD | +/- $450 |
| Total system price (approx.) | $10k | $15k | ||
With more or less comparable specs, the S822L was about 50% more expensive. However, it was almost impossible to make an apples-to-apples comparison. The biggest "price issue" are the CDIMMs, which are almost 4 times as expensive as "normal" RDIMMs. CDIMMs offer more as they include an L4-cache and some extra features (such as a redundant memory chip for each 9 chips). For most typical current Xeon E5 customers, the cost issue will be important. For a few, the extra redundancy and higher bandwidth will be interesting. Less important, but still significant is the fact that IBM uses SAS disks, which increase the cost of the storage system, especially if you want lots of them.
This cost issue will be much less important on most third party POWER8 systems. Tyan's "Habanero" system for example integrates the Centaur chips on the motherboard, making the motherboard more expensive but you can use standard registered DDR3L RDIMMs, which are much cheaper. Meanwhile the POWER8 processor tends to be very reasonably priced, at around $1500. That is what Dell would charge for an Intel Xeon E5-2670 (12 cores at 2.3-2.6 GHz, 120W). So while Intel's Xeon are much more power efficient than the POWER8 chips, the latter tends to be quite a bit cheaper.
Facebook
Twitter
RSS
146 Comments
View All Comments
psychobriggsy - Friday, November 6, 2015 - link
So you are complaining that your job's selection of hardware has made you earn twice as much?dgingeri - Friday, November 6, 2015 - link
No, because I don't earn twice as much. I'm not fully trained in AIX, so I have to muddle my way through dealing with the test machines we have. We don't use them for full production machines, just for testing software for our customers. (Which means I have to reinstall the OS on at least one of those machines about every month or so. That is a BIG pain in the behind due to the boot procedure. Where it takes a couple hours to reinstall Windows or Linux, it takes a full day to do it on an AIX machine.)I'm trying to advise people to NOT use AIX. It's an awful operating system. I'm also advising people NOT use IBM Power based machines because they are extremely aggravating to work on. Overall, it costs much more to run IBM Power machines, even if they aren't running AIX, than it does to run x86 machines. The up front cost might look competitive, but the maintenance costs are huge. Running AIX on them makes it an order of magnitude more expensive.
serpint - Friday, November 6, 2015 - link
I suggest reading the NIM A-Z handbook. It shouldn't take you more than 10 minutes to fully deploy an AIX system fully built and installed with software. As with Linux, it also shouldn't take more than about 10 minutes to install and fully deploy a server if you have any experience scripting installs.The developerworks community inside IBM is possibly the best free resource you could hope for. Also the redbooks.ibm.com site.
Compared to most *NIX flavors, AIX is UNIX for dummies.
agtcovert - Tuesday, November 10, 2015 - link
If you had a NIM server setup and were using LPARs, loading a functional image of AIX should take 10 minutes flat, on a 1G network.If you're loading AIX on a physical machine without using the virtualization, you're wasting the server.
agtcovert - Tuesday, November 10, 2015 - link
I've worked on AIX platforms extensively for about the same amount of time. First, most of these purchases go through a partner and yours must've sucked because we got great support from our IBM partner -- free training, access to experts, that sort of thing.Second, I always love the complaining about the cost of the hardware, etc. If you're buying big iron Power servers, the maintenance cost should be near irrelevant. And again, your partner should take care to negotiate that into the deal for 3-5 years ensuring you have access to updates.
The other thing no one ever talks about is *why* you buy these servers. Why do they take so long to boot? Well, for the frame it self, it's a deep POST. But then, mine were never rebooted in 4 years, and that's for firmware upgrades (online) and a couple of interface card swaps (also done online with no service disruption). Do that on x86. So reason #1 -- RAS, at the hardware level. Seriously, how often did you need to reboot the frame?
Reason #2 -- for large enterprises, you can do so much with these with relatively few cores they lead to huge licensing savings in Oracle, IBM software. For us, it was over $1m a year ongoing. And no, switching to other software was not an option. We could run an Oracle RAC on 4 cores of Power 7 (at the time) versus the 32 x86 it was on previously. That saves a lot of $.
The machine reviewed does not run AIX. It's Linux only. So the maintenance, etc. you mention isn't even relevant.
There are still things that are annoying I suppose. AIX is steeped in legacy to some degree, and certainly not as easy to manage as a Linux box. But there are a lot of guides out there for free -- it took me about a month to be fully productive. And the support costs you pay for -- well, if I ran into a wall, I just opened a PMR. IBM was always helpful
nils_ - Wednesday, November 11, 2015 - link
I'm mostly working in Linux Devops now, but I remember dreading to use all the "classic" Unix machines at my first "real" job 12 years ago. We ran a few IRIX and AIX boxes which were ancient along itself. Hell even the first thing I did on my work Macbook was to replace the BSD userland with GNU wherever possible.It's hard to find any information on them and any learning materials are expensive and usually on dead trees. They pretty much want to sell training, consulting etc. along with the often non-competitive Hardware prices since these companies don't actually WANT to sell hardware. They want to sell everything that surrounds it.
retrospooty - Friday, November 6, 2015 - link
The problem with server chips is that its about platform stability. IBM (and others) dropped off the face of the Earth and as mentioned above Intel now has 95% of the market. This chip looks great but will companies buy into it in mass? What if IBM makes another choice to drop off the face of the Earth again and your platform is dead ended? I would have to think long and hard about going with them at this point.FunBunny2 - Friday, November 6, 2015 - link
Not likely. the mainframe z machines are built using POWER blocks.Kevin G - Friday, November 6, 2015 - link
POWER and System Z are two different architectures. Case in point, POWER is a RISC design introduced in the 90's where as the System Z mainframes can trace their roots to a CISC design from the 1960's (and it is still possible to run some of that 1960's code unmodified).They do share a handful of common parts (think the CDIMMs) to cut down on support costs.
plonk420 - Friday, November 6, 2015 - link
can you run an x264 benchmark on it?? x)