Sizing Up Servers: Intel's Skylake-SP Xeon versus AMD's EPYC 7000 - The Server CPU Battle of the Decade?
by Johan De Gelas & Ian Cutress on July 11, 2017 12:15 PM EST- Posted in
- CPUs
- AMD
- Intel
- Xeon
- Enterprise
- Skylake
- Zen
- Naples
- Skylake-SP
- EPYC
Single Threaded Integer Performance: SPEC CPU2006
Even in the server market where high core count CPUs are ruling the roost, high single threaded performance is still very desirable. It makes sure that a certain level of performance is guaranteed in every situation, not just in "throughput situations" of "embarrassingly parallel" software.
SPEC CPU2017 has finally launched, but it did so while our testing was already under way. So SPEC CPU2006 was still our best option to evaluate single threaded performance. Even though SPEC CPU2006 is more HPC and workstation oriented, it contains a good variety of integer workloads.
It is our conviction that we should try to mimic how performance critical software is compiled instead of trying to achieve the highest scores. To that end, we:
- use 64 bit gcc : by far the most used compiler on linux for integer workloads, good all round compiler that does not try to "break" benchmarks (libquantum...) or favor a certain architecture
- use gcc version 5.4: standard compiler with Ubuntu 16.04 LTS. (Note that this is upgraded from 4.8.4 used in earlier articles)
- use -Ofast -fno-strict-aliasing optimization: a good balance between performance and keeping things simple
- added "-std=gnu89" to the portability settings to resolve the issue that some tests will not compile with gcc 5.x
- run one copy of the test
The ultimate objective is to measure performance in non-"aggressively optimized" applications where for some reason – as is frequently the case – a "multi-thread unfriendly" task keeps us waiting.
First the single threaded results. It is important to note that thanks to modern turbo technology, all CPUs will run at higher clock speeds than their base clock speed.
- The Xeon E5-2690 ("Sandy Bridge") is capable of boosting up to 3.8 GHz
- The Xeon E5-2690 v3 ("Haswell") is capable of boosting up to 3.5GHz
- The Xeon E5-2699 v4 ("Broadwell") is capable of boosting up to 3.6 GHz
- The Xeon 8176 ("Skylake-SP") is capable of boosting up to 3.8 GHz
- The EPYC 7601 ("Naples") is capable of boosting up to 3.2 GHz
First we look at the absolute numbers.
| Subtest | Application type | Xeon E5-2690 @ 3.8 |
Xeon E5-2690 v3 @ 3.5 |
Xeon E5-2699 v4 @ 3.6 |
EPYC 7601 @3.2 |
Xeon 8176 @3.8 |
| 400.perlbench | Spam filter | 35 | 41.6 | 43.4 | 31.1 | 50.1 |
| 401.bzip2 | Compression | 24.5 | 24.0 | 23.9 | 24.0 | 27.1 |
| 403.gcc | Compiling | 33.8 | 35.5 | 23.7 | 35.1 | 24.5 |
| 429.mcf | Vehicle scheduling | 43.5 | 42.1 | 44.6 | 40.1 | 43.3 |
| 445.gobmk | Game AI | 27.9 | 27.8 | 28.7 | 24.3 | 31.0 |
| 456.hmmer | Protein seq. analyses | 26.5 | 28.0 | 32.3 | 27.9 | 35.4 |
| 458.sjeng | Chess | 28.9 | 31.0 | 33.0 | 23.8 | 33.6 |
| 462.libquantum | Quantum sim | 55.5 | 65.0 | 97.3 | 69.2 | 102 |
| 464.h264ref | Video encoding | 50.7 | 53.7 | 58.0 | 50.3 | 67.0 |
| 471.omnetpp | Network sim | 23.3 | 31.3 | 44.5 | 23.0 | 40.8 |
| 473.astar | Pathfinding | 25.3 | 25.1 | 26.1 | 19.5 | 27.4 |
| 483.xalancbmk | XML processing | 41.8 | 46.1 | 64.9 | 35.4 | 67.3 |
As raw SPEC scores can be a bit much to deal with in a dense table, we've also broken out our scores on a percentage basis. Sandy Bridge EP (Xeon E5 v1) is about 5 years old, the servers based upon this CPU are going to get replaced by newer ones. So we've made "Single threaded Sandy Bridge-EP performance" our reference (100%) , and compare the single threaded performance of all other architectures accordingly.
| Subtest | Application type | Xeon E5-2690 @ 3.8 |
Xeon E5-2690 v3 @ 3.5 |
Xeon E5-2699 v4 @ 3.6 | EPYC 7601 @3.2 | Xeon 8176 @ 3.8 |
| 400.perlbench | Spam filter | 100% | 119% | 124% | 89% | 143% |
| 401.bzip2 | Compression | 100% | 98% | 98% | 98% | 111% |
| 403.gcc | Compiling | 100% | 105% | 70% | 104% | 72% |
| 429.mcf | Vehicle scheduling | 100% | 97% | 103% | 92% | 100% |
| 445.gobmk | Game AI | 100% | 100% | 103% | 87% | 111% |
| 456.hmmer | Protein seq. analyses | 100% | 106% | 122% | 105% | 134% |
| 458.sjeng | Chess | 100% | 107% | 114% | 82% | 116% |
| 462.libquantum | Quantum sim | 100% | 117% | 175% | 125% | 184% |
| 464.h264ref | Video encoding | 100% | 106% | 114% | 99% | 132% |
| 471.omnetpp | Network sim | 100% | 134% | 191% | 99% | 175% |
| 473.astar | Pathfinding | 100% | 99% | 103% | 77% | 108% |
| 483.xalancbmk | XML processing | 100% | 110% | 155% | 85% | 161% |
SPEC CPU2006 analysis is complicated, and with only a few days spend on the EPYC server, we must admit that what follows is mostly educated guessing.
First off, let's gauge the IPC efficiency of the different architectures. Considering that the EPYC core runs at 12-16% lower clockspeeds (3.2 vs 3.6/3.8 GHz), getting 90+% of the performance of the Intel architectures can be considered a "strong" (IPC) showing for the AMD "Zen" architecture.
As for Intel's latest CPU, pay attention to the effect of the much larger L2-cache of the Skylake-SP core (Xeon 8176) compared to the previous generation "Broadwell". Especially perlbench, gobmk, hmmer and h264ref (the instruction part) benefit.
Meanwhile with the new GCC 5.4 compiler, Intel's performance on the "403.gcc benchmark" seems to have regressed their newer rchitectures. While we previously saw the Xeon E5-2699v4 perform at 83-95% of the "Sandy Bridge" Xeon E5-2690, this has further regressed to 70%. The AMD Zen core, on the other hand, does exceptionally well when running GCC. The mix of a high percentage of (easy to predict) branches in the instruction mix, a relatively small footprint, and a heavy reliance on low latency (mostly L1/L2/8 MB L3) seems to work well. The workloads where the impact of branch prediction is higher (somewhat higher percentage of branch misses) - gobmk, sjeng, hmmer - perform quite well on "Zen" too, which has a much lower branch misprediction penalty than AMD's previous generation architecture thanks to the µop cache.
Otherwise the pointer chasing benchmarks – XML procesing and Path finding – which need a large L3-cache, are the worst performing on EPYC.
Also notice the fact that the low IPC omnetpp ("network sim") runs slower on Skylake-SP than on Broadwell, but still much faster than AMD's EPYC. Omnetpp is an application that benefited from the massive 55 MB L3-cache of Broadwell, and that is why performance has declined on Skylake. Of course, this also means that the fractured 8x8 MB L3 of AMD's EPYC processor causes it to perform much slower than the latest Intel server CPUs. In the video encoding benchmark "h264ref" this plays a role too, but that benchmark relies much more on DRAM bandwidth. The fact that the EPYC core has higher DRAM bandwidth available makes sure that the AMD chip does not fall too far behind the latest Intel cores.
All in all, we think we can conclude that the single threaded performance of the "Zen architecture" is excellent, but it somewhat let down by the lower turbo clock and the "smaller" 8x8 MB L3-cache.
Facebook
Twitter
RSS
219 Comments
View All Comments
twtech - Thursday, July 20, 2017 - link
I'd really like to see some compile-time benchmarks for these CPUs.For my own particular interests, time taken to do a full recompile of the Unreal 4 engine from source would be very useful. But even something more generic like the Linux kernel compiles per hour benchmark could serve as a useful point of reference.
szupek - Friday, July 21, 2017 - link
Meanwhile, the entire world still runs on IBM's DB2 for Datbases and IBM's Z/AS400 Mainframes. The fastest database in the world, by far...oh and the most secure (it's only hackable by standing in front of the console, seriously). Every single credit card transaction. Every single plain ticket. Most medical records and all of wall street. Yup. IBM still owns. So much that most of commenters probably have no idea just how big IBM truly is. If you care about Database speed & security, these processors shouldn't appeal to you.stevefan1999 - Saturday, July 22, 2017 - link
It's impossible for AMD to win completely.Remember kids, public cloud service providers such as Amazon(AWS), Google(GCP) and Joyent would still stick with Intel due to not only the compatibility issues like ecosystem and vendor inconsistency, but also the VM migration and security and module issues, all mentioned in the presentation slides presented by Intel. They are a very serious matter, as they, the public cloud services, are powering the Internet we use everyday, so being stable, consistent and be able to serve a good amount of SLA is vital to the public cloud, we wouldn't expect them to play with the new lad in the hood, the EPYC.
IIRC only the Microsoft(Azure) are using AMD server CPUs partially in some of their datacenters, running various Linux and Windows VMs using Hyper-V, and they have been performing quite well
The cloud services are exploding every year, but with what I've said, I doubt AMD could even kick in the first door at least for 3 to 4 years. This is still a big-win for Intel and what manipulations will Intel do I don't know.
On the other hand, Intel has failed to service the desktop market and they're figuring out how to hold their asses on the Internet infrastructure, never had them know the crusade of EPYC will come this fast.
The server market is quite a big meat, it's a 21 bil market, cool right? But that you will have guaranteed 'server upgrade' every year, is a bigger matter, as those server CPUs are designated to be disposed given the wattage and performance per dollar is lower on the newer CPUs. Those god-damn server operators will keen to replace their CPU (and therefore some serious metal pollution issues). Intel has been exploiting this and gained a big hurdle of money and therefore had their ecosystem grown. This is how Intel defends their platform by vendor lock-in, pathetic.
AMD is now being performance and cost competitive to Intel, but it's still dead in the High Performance Computing campaign unless AMD could provide higher frequencies. Well I have to say I know nothing about HPC, but I remembered the Bulldozer architecture of AMD is actually targeted and marketed for HPC! That's why AMD failed in general-purpose computing market and started the downfall of AMD/Domination of Intel 5 years ago. Even though we know the fate of Bulldozer, but hopefully AMD could still scrap some of the HPC goodies of Bulldozer out and benefits the mankind by accelerating researches such as finding the cures for cancer or solving some precise physics and mathematics.
Well, anyway the cloud, the HPC and the server market are the last resort for Intel and they will definitely hold their last ground. Good luck AMD on crushing the mean and obese Intel!
errorr - Sunday, July 23, 2017 - link
For all the talk about speed and efficiency the problem is about $$$. The sad fact is that what matters most isn't even the price of the cpus which is chump change in the grand scheme of things but how the software licensing costs are determined. Per core or per socket software pricing will matter a lot. The software companies will decide how successful EPYC is. I have a feeling they will be biased slightly toward AMD at the beginning as it is in their interest to foster competition for Intel, or if they are not forward looking enough the end customers might argue that the competition will benefit the SW companies in the long run by continuing to push competition.msroadkill612 - Thursday, July 27, 2017 - link
Whatever, its all pointless if the competition can read your secrets, which is a matter very close to the hearts of the cheque signers.AMD seem to have something very superior to offer in that department.
qweqwe - Tuesday, August 8, 2017 - link
we just did some heavy inhouse hpc-tests with epyc against diff. intel servers.the epyc is the clear winner in terms of performance and power consumption when it
comes to hand-tuned parallel-vector-code examples.
not bad amd !
readonly1 - Friday, October 27, 2017 - link
qweqwe, I totally agree with you. Our inhouse HPC tests get the similar conclusion, after comparing AMD Epyc 7351 (dual socket, 32 cores, 2400Mhz) and Intel SKylake 6154 (dual socket, 36 cores, 3000Mhz). I think AMD clearly wins in the memory bandwidth, which is extremely important for HPC computation.msroadkill612 - Monday, November 13, 2017 - link
7/11/2017 "Microsoft is already deploying AMD's EPYC in their Azure Cloud Datacenters."Interesting. As i have been theorising, a possible reason for the absence of retail epyc is not supply, but demand.
A single sale can soak up production runs.
If so tho, not much sign of big revenues from it yet, but there are other explanations for that. Contra processors for development work e.g.
q.epsilon.p - Sunday, June 10, 2018 - link
power consumption numbers with every benchmark would have been nice, because these parts are server benchmarks, Perf / Watt is one of the primary concerns. And where AMD kinda crush Intel, because it's isn't exactly being honest with it's TDP values nowadays when it comes to Data Centre and HEDT.TDP was traditionally the absolute maximum the CPU would put out as heat, now with a power consumption of 670W I am assuming that the heat being put out by the CPU is more than 165W.