Comparing Benchmarks: AT vs IBM

Before we close things out, let's spend a moment summarizing our results and comparing the performance we saw to the kind of performance advantages that IBM advertises POWER8 is capable of.

From a high level perspective, the S822L is more expensive and consumes a lot more power than a comparable Xeon system.

With limited optimization and with the current Ubuntu 15.04, the performance-per-watt ratio favors Intel even more as the POWER8 barely outperforms the very efficient 120W TDP Xeons. So there is no denying that the Intel systems offer a superior performance/watt ratio.

However, it would be unfair to base our judgement on our first attempt as we have to admit this our first real attempt to benchmark and test the POWER8 platform. It is very likely that we will manage to extract quite a bit more performance out of the system on our second attempt. IBM POWER8 also has a big advantage in memory bandwidth. But we did not manage to port OpenFOAM to the POWER platform, probably the most likely candidate for leveraging that advantage.

We are less convinced that the POWER8 platform has a huge "raw CPU compute advantage," contrary to what for example IBM's SPECJBB (85% faster ) and SAP (29% faster) results seem to suggest.

For example, IBM's own SPECjEnterprise®2010 benchmarking shows that:

SAP is "low IPC" (hard to run many instructions in parallel in one thread) software that benefits much from low latency caches. The massive L3-cache (12-cores, 96 MB) and huge thread count are probably giving the IBM POWER8 the edge. The RAM bandwidth also helps, but in a lesser degree. IBM clearly built POWER8 with this kind of software in mind. We had individual threadcount intensive benchmarks (LZMA decompression) and L3-cache sensitive benchmarks (ElasticSearch), but t o be fair to IBM, none of our benchmarks leveraged the three strongest points (threadcount, L3-cache size and memory bandwidth) all at once like SAP.

SPECJBB2013 has recently been discontinued as it was not reliable enough. We tend to trust the jEnterprise test a lot more. In any case, the best POWER8 has a 17% advantage there.

Considering that the POWER8 inside that S824 has 20% more cores and a 3% higher clockspeed, our 3.4 GHz 10-core CPU would probably be slightly behind the Xeon E5-2697 v3. We found out that the 10-core POWER8 is slightly faster than Xeon E5-2695 v3. The Xeon E5-2695 v3 is very similar to the E5-2697 v3, it is just running at a 10% lower clockspeed (All core turbo: 2.8GHz vs 3.1GHz). So all in all, our benchmarks seems to be close to the official benchmarks, albeit slightly lower.

Closing Thoughts: A Mix of Xeon "E5" and "E7"

So let's sum things up. The IBM S822L is definitely not a good choice for those looking to lower their energy bills or operate in a location with limited cooling. The pricing of the CDIMMs causes it to be more expensive than a comparable Xeon E5 based server. However, you get something in return: the CDIMMs should offer higher reliability and are more similar to the memory subsystem of the E7 than the E5. Also, PCIe adapters are hot-pluggable on the S822L and can be replaced without bringing down the system. With most Xeon E5 systems, only disks, fans and PSU are hot-pluggable.

In a number of ways then, the S822L is more a competitor to dual Xeon E7 systems than it is to dual Xeon E5 systems. In fact, a dual Xeon E7 server consumes in the 600-700W range, and in that scenario the power usage of S822L (700-800W) does not seem outrageous anymore.

The extra reliability is definitely a bonus when running real time data analytics or virtualization. A failing memory chip may cost a lot when you running fifty virtual machines on top of a server. Even in some HPC or batch data analytics applications where you have to wait for hours for a certain result that is being computed in an enormous amount of memory, the cost savings of being able to survive a failing memory chip might be considerable.

One more thing: for those who need full control, the fact that every layer in the software stack is open makes the S822L very attractive. For now, the available "OpenCompute" Xeon servers that are also "open" seem to mostly density optimized servers and the openess seems limited on several levels. Rackspace felt that the current OpenCompute servers are not "open enough", and went for OpenPOWER servers instead. In all those markets, the S822L is very interesting alternative to the dual Xeon E5 servers.

Ultimately however, the performance-per-dollar Xeon E5 competitors will most likely be OpenPOWER third party servers. Those servers do not use CDIMMS, but regular RDIMMs. Other components such as disks, networkcards and PSUs will probably be cheaper but potentially also slightly less reliable.

All in all, the arrival of OpenPOWER servers is much more exciting than most of us anticipated. Although the IBM POWER8 servers can not beat the performance/watt ratio of the Xeon, we now have a server processor that is not only cheaper than Intel's best Xeons, but that can also keep up with them. Combine that with the fact that IBM has lined up POWER8+ for next year and a whole range of server vendors is building their own POWER8 based servers, and we have a lot to look forward to!

Energy and Pricing
POST A COMMENT

146 Comments

View All Comments

  • Kevin G - Saturday, November 7, 2015 - link

    If all you do is just mount the network volume to use the data, then likely nothing at all. While binaries do have to be modified, the file systems themselves are written to store data in a single consistent manner. If you're wondering more if there would be some overhead in translating from LE to BE to work in memory, conceptually the answer is yes but I'd predict it be rather small and dwarfed by the time to transfer data over a network. I'd be curious to see the results.

    Ultiamtely I'd be more concerned with kernel modules for various peripherals when switching between LE and BE versions. Considering that POWER has been BE for a few generations and you did your initial testing using LE, availability shouldn't be an issue. You've been using the version which should have had the most problems in this regard.
    Reply
  • spikebike - Friday, November 6, 2015 - link

    So basically power is somewhat competitive with intel's WORST price/perf chips which also happen to have the worst memory bandwidth/CPU. Seems nowhere close for the more reasonable $400-$650 xeons like the D-1520/1540 or the E5-2620 and E5-2630. Sure IBM has better memory bandwidth than the worst intels, but if you want more memory bandwidth per $ or per core then get the E5-2620. Reply
  • JohanAnandtech - Saturday, November 7, 2015 - link

    It is definitely not an alternative for applications where performance/watt is important. As you mentioned, Intel offers a much better range of SKUs . But for transactional databases and data mining (traditional or unstructured), I see the POWER8 as very potent challenger. When you are handling a few hundreds of gigabytes of data, you want your memory to be reliable. Intel will then steer you to the E7 range, and that is where the POWER8 can make a difference: filling the niche between E5 and E7. Reply
  • nils_ - Wednesday, November 11, 2015 - link

    Especially if you're running software that doesn't easily scale out very well these are very competitive. And nowadays even MySQL will scale-up nicely to many, many cores. Reply
  • Gigaplex - Friday, November 6, 2015 - link

    "Less important, but still significant is the fact that IBM uses SAS disks, which increase the cost of the storage system, especially if you want lots of them."

    The Dell servers I've used had SAS controllers, and every SAS controller I've dealt with supported using SATA drives. I'm pretty sure SATA compatibility is in the SAS specification. In fact, the Dell R730 quoted in this review supports SAS drives. There shouldn't be anything stopping you from using the same drives in both servers.
    Reply
  • JohanAnandtech - Saturday, November 7, 2015 - link

    You are absolutely right about SATA drives being compatible with a SAS controller. However, afaik IBM gives you only the choice between their own rather expensive SAS drives and SSDs. And maybe I have looked over it, but in general DELL let you only chose between SATA and SSDs. And this has been the trend for a while: SATA if you want to keep costs low, SSDs for everything else. Reply
  • TomWomack - Sunday, November 8, 2015 - link

    And mounting a storage server made out of commodity hardware over a couple of lanes of 10Gbit Ethernet if you don't want to pay the exotic-hardware-supplier's markup on disc. Reply
  • Gunbuster - Friday, November 6, 2015 - link

    SAP and IBM AIX servers... I guess if you want to blow out your entire IT budget in once easy decision... Reply
  • Jake Hamby - Friday, November 6, 2015 - link

    I forgot to mention: VMX is better known as AltiVec (it's also called "Velocity Engine" by Apple). It's a very nice SIMD extension that was supported by Apple's G4 (Motorola/Freescale 7400/7450) and G5 (IBM PPC 970) Macs, as well as the PPC game consoles.

    It would be interesting to compare the Linux VMX crypto acceleration to code written to use the newer native AES & other instructions. In x86 terms, it'd be like SSE-optimized AES vs. the AES-NI instructions.
    Reply
  • Oxford Guy - Saturday, November 7, 2015 - link

    I had a dual 450 MHz G4 system and AltiVec was quite amazing in iTunes when doing encoding. Between the second processor and the AltiVec putting things into ALAC was very fast (in comparison with other machines at the time like the G3 and the AMD machines I had). Reply

Log in

Don't have an account? Sign up now