Intel's Benchmarks

Since time constraints meant that we were not able to run a ton of benchmarks ourselves, it's useful to check out Intel's own benchmarks as well. In our experience Intel's own benchmarking has a good track record for producing accurate numbers and documenting configuration details. Of course, you have to read all the benchmarking information carefully to make sure you understand just what is being tested.

The OLTP and virtualization benchmarks show that the new Xeon E7 v3 is about 25 to 39% faster than the previous Xeon E7 (v2). In some of those benchmarks, the new Xeon had twice as much memory, but it is safe to say that this will make only a small difference. We think it's reasonable to conclude that the Xeon E7 is 25 to 30% faster, which is also what we found in our integer benchmarks.

The increase in legacy FP application is much lower. For example Cinebench was 14% faster, SPECFP 9% and our own OpenFOAM was about 4% faster. Meanwhile linpack benchmarks are pretty useless to most of the HPC world, so we have more faith in our own benchmarking. Intel's own realistic HPC benchmarking showed at best a 19% increase, which is nothing to write home about.

The exciting part about this new Xeon E7 is that data analytics/mining happens a lot faster on the new Xeon E7 v3. The 72% faster SAS analytics number is not really accurate as part of the speedup was due to using P3700 SSDs instead of the S3700 SSD. Still, Intel claims that the replacing the E7 v2 with the v3 is good for a 55-58% speedup.

The most spectacular benchmark is of course SAP HANA. It is not 6x faster as Intel claims, but rather 3.3x (see our comments about TSX). That is still spectacular and the result of excellent software and hardware engineering.

Final Words: Comparing Xeon E7 v3 vs V2

For those of us running scale-up, reasonably priced HPC or database applications, it is hard to get excited about the Xeon E7 v3. The performance increases are small-but-tangible, however at the same time the new Xeon E7 costs a bit more. Meanwhile as far as our (HPC) energy measurements go, there is no tangible increase in performance per watt.

The Xeon E7 in its natural habitat: heavy heatsinks, hotpluggable memory

However organizations running SAP HANA will welcome the new Xeon E7 with open arms, they get massive speedups for a 0.1% or less budget increase. The rest of the data mining community with expensive software will benefit too, as the new Xeon E7 is at least 50% faster in those applications thanks to TSX.

Ultimately we wonder how the rest of us will fare. Will SAP/SAS speedups also be visible in open source Big Data software such as Hadoop and Elastic Search? Currently we are still struggling to get the full potential out of the 144 threads. Some of these tests run for a few days only to end with a very vague error message: big data benchmarking is hard.

Comparing Xeon E7 v3 and POWER8

Although the POWER8 is still a power gobbling monster, just like its older brother the POWER7, there is no denying that IBM has made enormous progress. Few people will be surprised that IBM's much more expensive enterprise systems beat Intel based offerings in the some high-end benchmarks like SAP's. But the fact that 24 POWER8 cores in a relatively reasonably priced IBM POWER8 server can beat 36 Intel Haswell cores by a considerable margin is new.

It is also interesting that our own integer benchmarking shows that the POWER8 core is capable of keeping up with Intel's best core at the same clockspeed (3.3-3.4 GHz). Well, at least as long as you feed it enough threads in IPC unfriendly code. But that last sentence is the exact description of many server workloads. It also means that the SAP benchmark is not an exception: the IBM POWER8 is definitely not the best CPU to run Crysis (not enough threads) but it is without a doubt a dangerous competitor for Xeon E7 when given enough threads to fill up the CPU.

Right now the threat to Intel is not dire, IBM still asks way too much for its best POWER8 systems and the Xeons have a much better performance-per-watt ratio. But once the OpenPOWER fondation partners start offering server solutions, there is a good chance that Intel will receive some very significant performance-per-dollar competition in the server market.

HPC Watts per Job
Comments Locked

146 Comments

View All Comments

  • PowerTrumps - Saturday, May 9, 2015 - link

    Ok, yes a data center like Verizon or ATT might not "qualify" but the point is accurate. I work with IBM's Power servers and have absolutely consolidated 5 racks of x86 into a single Power server - it was 54 Intel 2S & 4S servers into a single 64c Power7. Part of this is due to the "performance" of Power but most of the credit goes to the efficiency of the Power Hypervisor. PHYP can provide a QoS to each workload while weaving a greater amount of workloads onto fewer Power servers/cores than what the benchmarks imply.
  • newtrekemotion - Friday, May 8, 2015 - link

    I wouldn't discount Oracle so quickly. The T5 was a pretty big step forward from the T4 and the new M7 chip sounds like it could be quite the competitor with 2 TB of memory per socket and 32 cores, especially for highly threaded loads since an octo-socket system would have 2048 threads and support 16 TB of memory.. Hopefully this can bring some more competition to the market, though with only Oracle and Fujistu (maybe?) selling systems it won't have quite the impact that multiple POWER8 vendors could bring. Love them, hate them, or anywhere in between it seems Oracle is not ready to give up in this arena and it looks like they are putting more effort in than Sun was (or are at least executing on effort more than Sun did).

    Something else to note here is the process advantage that Intel has over everyone else. I might have missed it in the article, but especially for performance/watt this is important.

    In all I think the statement at the beginning of the article that this area is getting more exciting is very true. Just seems like it might be a 3 way race instead of a 2. The recent AMD announcement that they wanted to focus on HPC is interesting too though of the 4 (Intel, IBM, Oracle and AMD) they have the furthest to go and the fewest resources to do it with. The next few years are going to be very interesting and hopefully someone, or a combination can push Intel and drive the whole market forward.
  • JohanAnandtech - Friday, May 8, 2015 - link

    I was writing from a "who will be able to convert Intel Xeon people" point of view. As I wrote in the Xeon E7v2 article, Oracle's T processors have indeed vastly improved. That is all nice and well but there is no reason why someone considering a Xeon E7 would switch. Oracle's sales seems to mostly about people who are long time Oracle users. As far as I can see, OpenPOWER servers are the only real thread to Intel's server hegemony.
  • Kevin G - Saturday, May 9, 2015 - link

    Oracle does offer one reason to switch to SPARC: massive licensing discounts on Oracle software.

    If you're not using Oracle's software, then yeah, the SPARC platform is a very tough sell over x86 or POWER.
  • JohanAnandtech - Saturday, May 9, 2015 - link

    exactly. Good point.
  • PowerTrumps - Saturday, May 9, 2015 - link

    If you are running Oracle software you should know that IBM and Power are the largest platform which Oracle software runs on. Secondly, if running Oracle products licensed by the core, the only platform to control Oracle licensing is Power (not including Mainframe in this assertion). I have reduce Oracle licensing for customers anywhere from 4X to 10X. Do the math on that to appreciate those savings. Lastly, when I upgrade customers from one generation to another we talk about how much Oracle they can reduce. You don't hear that when upgrading from Sandy Bridge to Ivy Bridge to Haswell.
  • kgardas - Friday, May 8, 2015 - link

    I'm not sure about T5, but certainly latest Fujitsu's SPARC64-X+ is able to over-run POWER8 and by wide margin also older Xeon's. Just look for the spec. rate. It also won some SAP S&D 2-tier benchmark on absolute performance so I'm glad that SPARC is still competitive too...
  • Kevin G - Saturday, May 9, 2015 - link

    The top SPARC benchmarks I've seen are using far more sockets, cores, threads and memory to get to that top spot. It is nice that the system can scale to such high socket counts (40) but only if you can actually fund a project that needs that absolute performance. Drop down to 16 socket where you can get twice the performance from POWER than SPARC with the same licensing cost, what advantage does SPARC have to make people switch?

    Even then, a system like SGI's UV2000 would fall into the same niche due to its ability to scale to insane socket counts, software licensing fees be damned.
  • kgardas - Tuesday, May 12, 2015 - link

    Kevin G, actually you are right and I made an mistake. It was not intentional, I was misled by spec site claiming "24 cores, 4 chips, 6 cores/chip, 8 threads/core" for "IBM Power S824 (3.5 GHz, 24 core, RHEL)" so I've thought this is 4 socket setup and I compared it with Fujitsu M10-4 which won. Now, I've just found IBM is two socket which means it wins on socket/spec rate basis of course. Price-wise IBM is also way much cheaper than SPARC (if you don't run Oracle DB of course) so I keep my fingers crossed for OpenPOWER.
    Honestly, although this is really nice to see I still have kind of feeling that this is IBM hardware division swan's song. I would really like to be wrong here. Anyway, I still think that ARMv8 does have higher chances in getting into the Intel's business and be really a pain for Intel. On the other hand if OpenPOWER is successful in Chinese business, that would be good and some chance for us too to see lower-cost POWER machines...
  • PowerTrumps - Saturday, May 9, 2015 - link

    yes, take a look at those benchmark results and you see the Fuji M10-4S requires 640 & 512 cores. Even the Oracle M6-32 uses 384 cores. The Fuji 512c example had 33% higher SAPS with 2X the cores. The M6-32 has 50% more cores to get 21% higher SAPS. Further, looking at the SAP benchmark as a indicator of core, chip & server performance shows that SPARC & Intel are roughly 1600 - 2200 SAPS per core compared to Power8 which is 5451 SAPS for the 80 core E870. So you put this into context the 80 core Power8 has slightly less than 1/2 the SAPS of the 640 core Fujitsu M10-4S. Think of ALL the costs associated with 640 cores vs 80...ok, 160 if we want to get the SAPS roughly equal. 4X more cores to get less than 2X the results.

Log in

Don't have an account? Sign up now