Assessing IBM's POWER8, Part 2: Server Applications on OpenPOWER
by Johan De Gelas on September 15, 2016 8:01 AM ESTSpark Benchmarking
Spark is wonderful framework, but you need some decent input data and some good coding skills to really test it. Speeding up Big Data applications is the top priority project at the lab I work for (Sizing Servers Lab of the University College of West-Flanders), so I was able to turn to the coding skills of Wannes De Smet to produce a benchmark that uses many of the Spark features and is based upon real world usage.
The test is described in the graph above. We first start with 300 GB of compressed data gathered from the CommonCrawl. These compressed files are a large amount of web archives. We decompress the data on the fly to avoid a long wait that is mostly storage related. We then extract the meaningful text data out of the archives by using the Java library "BoilerPipe". Using the Stanford CoreNLP Natural Language Processing Toolkit, we extract entities ("words that mean something") out of the text, and then count which URLs have the highest occurrence of these entities. The Alternating Least Square algorithm is then used to recommend which URLs are the most interesting for a certain subject.
We tested with Apache Spark 1.5 in standalone mode (non-clustered) as it took us a long time to make sure that the results were repetitive. For now, we keep version 1.5 to be able to compare with earlier results.
The POWER8 surprises here with excellent performance: it is able to keep with a 14 core Xeon E5 "Broadwell EP" and beats the midrange Xeon E5-2690 v3 by healthy margin. Remember, this is a midrange POWER8: there are SKUs that reach 3.4-3.8 GHz.
Facebook
Twitter
RSS
49 Comments
View All Comments
nils_ - Monday, September 26, 2016 - link
Isn't the limit slighty lower than 32 GiB? At some point the JVM switches to 64 bit pointers, which means you'll lose a lot of the available heap to larger pointers. I think you might want to lower your settings. I'm curious, what kind of GC times are you seeing with your heap size? I don't currently have access to Java running on non virtualised hardware so I would like to know if the overhead is significant (mostly running Elasticsearch here).CajunArson - Thursday, September 15, 2016 - link
All in all the Power chip isn't terrible but the power consumption coupled with the sheer amount of tuning that is required just to get it competitive with the Xeons isn't too encouraging. You could spend far less time tuning the Xeons and still have higher performance or go ahead with tuning to get even more performance out of those Xeons.On top of the fact that this isn't a supposedly "high end" model, the higher end power parts cost more and will burn through even more power, and that's an expense that needs to be considered for the types of real-world applications that use these servers.
dgingeri - Thursday, September 15, 2016 - link
That ad on the last page that claims lower equipment cost of course compares that to an HP DL380, the most overpriced Xeon E5 system out right now. (I know because I shopped them.) Comparing it to a comparable Dell R730 would show less expense, better support, and better expansion options.Morawka - Thursday, September 15, 2016 - link
you mean a company made a slide that uses the most extreme edge cases to make their product look good?!?! Shocking /sGondalf - Thursday, September 15, 2016 - link
Something is wrong is these power consumption data. The plataform idles at 221W and under full load only 260W?? the cpu is vanished?? Power 8 at over 3Ghz has an active power of only 40W??1) the idle value is wrong or 2) the under load value is wrong. All this is not consistent with IBM TDP official values.
IMO the energy consumption page of the article has to be rewrite.
JohanAnandtech - Thursday, September 15, 2016 - link
We have double checked those numbers. It is probably an indication that many of the power saving features do not work well under Linux right now.BTW, just to give you an idea: running c-ray (floating point) caused the consumption to go to 361W.
Kevin G - Thursday, September 15, 2016 - link
I presume that c-ray uses the 256 bit vector unit on POWER8?Also have you done any energy consumption testing that takes advantage of the hardware decimal unit?
mapesdhs - Thursday, September 15, 2016 - link
C-ray isn't that smart. :D It's a very simple code, brute force basically, and the smaller dataset can easily fit in a modern cache (actually the middling size test probably does too on CPUs like these). Hmm, I suppose it's possible one could optimise the compilation a bit to help, but I doubt anything except a full rewrite could make decent use of any vector tech, and I don't want to allow changes to the code, that would make comparisons to all other test results null. Compiler optimisations are ok, but not multi-pass optimisations that feed back info about the target data into the initial compile, that's cheating IMO (some people have done this to obtain what look like really silly run times, but I don't include them on my main C-ray page).Ian.
Gondalf - Tuesday, September 20, 2016 - link
Ummm so in short words the utilized sw don't stress at all the cpu, not even the hot caches near the memory banks. We need a bench with an high memory utilization and a balanced mix between integer and FP, more in line with real world utilizationI don't know if this test is enough to say POWER8 is power/perf competitive with haswell in 22nm.
In fact POWER market share is definitively at the historic minimum and 14nm Broadwell is pretty young, so this disaster it is not its fault.
jesperfrimann - Wednesday, September 21, 2016 - link
If you have a OPAL (Bare Metal system that cannot run POWERVM) then all the powersavings features are off by default AFAIR.Try to have a look at:
https://public.dhe.ibm.com/common/ssi/ecm/po/en/po...
Many of the features does have a performance impact, ranging from negative over neutral to positive for a single one.
But Again. I think your comparison with 'vanilla' software stacks are relevant. This is what people would see out of the box with an existing software stack.
It is 101% relevant to do that comparison as this is the marked that IBM is trying to break into with these servers.
But what could be fun to see was some tests where all the Bells and Whistles were utilized. As many have written here.. use of Hardware supported Decimal Floating Point. The Vector Execution unit, the ability to do hardware assisted Memory Compression etc. etc.
// Jesper