AMD Rome Second Generation EPYC Review: 2x 64-core Benchmarked
by Johan De Gelas on August 7, 2019 7:00 PM ESTSingle-Thread SPEC CPU2006 Estimates
While it may have been superceded by SPEC2017, we have built up a lot of experience with SPEC CPU2006. Considering the trouble we experience with our datacenter infrastructure, it was our best first round option for raw performance analysis.
Single threaded performance continues to be very important, especially in maintainance and setup situations. These examples may include running a massive bash script, trying out a very complex SQL query, or configuring new software - there are lots of times where a user simply does not use all the cores.
Even though SPEC CPU2006 is more HPC and workstation oriented, it contains a good variety of integer workloads. It is our conviction that we should try to mimic how performance critical software is compiled instead of trying to achieve the highest scores. To that end, we:
- use 64 bit gcc : by far the most used compiler on linux for integer workloads, good all round compiler that does not try to "break" benchmarks (libquantum...) or favor a certain architecture
- use gcc version 7.4 and 8.3: standard compiler with Ubuntu 18.04 LTS and 19.04.
- use -Ofast -fno-strict-aliasing optimization: a good balance between performance and keeping things simple
- added "-std=gnu89" to the portability settings to resolve the issue that some tests will not compile
The ultimate objective is to measure performance in non-aggressively optimized"applications where for some reason – as is frequently the case – a multi-thread unfriendly task keeps us waiting. The disadvantage is there are still quite a few situations where gcc generates suboptimal code, which causes quite a stir when compared to ICC or AOCC results that are optimized to look for specific optimizations in SPEC code.
First the single threaded results. It is important to note that thanks to turbo technology, all CPUs will run at higher clock speeds than their base clock speed.
- The Xeon E5-2699 v4 ("Broadwell") is capable of boosting up to 3.6 GHz. Note: these are old results compiled w GCC 5.4
- The Xeon 8176 ("Skylake-SP") is capable of boosting up to 3.8 GHz.
- The EPYC 7601 ("Naples") is capable of boosting up to 3.2 GHz.
- The EPYC 7742 ("Rome") boosts to 3.4 GHz. Results are compiled with GCC 7.4 and 8.3
Unfortunately we could not test the Intel Xeon 8280 in time for this data. However, the Intel Xeon 8280 will deliver very similar results, the main difference being that it runs a 5% higher clock (4 GHz vs 3.8 GHz). So we basically expect the results to be 3-5% higher than the Xeon 8176.
As per SPEC licensing rules, as these results have not been officially submitted to the SPEC database, we have to declare them as Estimated Results.
| Subtest | Application Type | Xeon E5-2699 v4 |
EPYC 7601 |
Xeon 8176 |
EPYC 7742 |
EPYC 7742 |
| Frequency | 3.6 GHz | 3.2 GHz | 3.8 GHz | 3.4 GHz | 3.4 GHz | |
| Compiler | gcc 5.4 | gcc 7.4 | gcc 7.4 | gcc 7.4 | gcc 8.3 | |
| 400.perlbench | Spam filter | 43.4 | 31.1 | 46.4 | 41.3 | 43.7 |
| 401.bzip2 | Compression | 23.9 | 24.0 | 27.0 | 26.7 | 27.2 |
| 403.gcc | Compiling | 23.7 | 35.1 | 31.0 | 42.3 | 42.6 |
| 429.mcf | Vehicle scheduling | 44.6 | 40.1 | 40.6 | 39.5 | 39.6 |
| 445.gobmk | Game AI | 28.7 | 24.3 | 27.7 | 32.8 | 32.7 |
| 456.hmmer | Protein seq. | 32.3 | 27.9 | 35.6 | 30.3 | 60.5 |
| 458.sjeng | Chess | 33.0 | 23.8 | 32.8 | 27.7 | 27.6 |
| 462.libquantum | Quantum sim | 97.3 | 69.2 | 86.4 | 72.7 | 72.3 |
| 464.h264ref | Video encoding | 58.0 | 50.3 | 64.7 | 62.2 | 60.4 |
| 471.omnetpp | Network sim | 44.5 | 23.0 | 37.9 | 23.0 | 23.0 |
| 473.astar | Pathfinding | 26.1 | 19.5 | 24.7 | 25.4 | 25.4 |
| 483.xalancbmk | XML processing | 64.9 | 35.4 | 63.7 | 48.0 | 47.8 |
A SPEC CPU analysis is always complicated, being a mix of what kind of code the compiler produces and CPU architecture.
| Subtest | Application type | EPYC 7742 (2nd gen) vs 7601 (1st gen) |
EPYC 7742 vs Intel Xeon Scalable |
Gcc 8.3 |
| 400.perlbench | Spam filter | +33% | -11% | +6% |
| 401.bzip2 | Compression | +11% | -1% | +2% |
| 403.gcc | Compiling | +21% | +28% | +1% |
| 429.mcf | Vehicle scheduling | -1% | -3% | 0% |
| 445.gobmk | Game AI | +35% | +18% | +0% |
| 456.hmmer | Protein seq. analyses | +9% | -15% | +100% |
| 458.sjeng | Chess | +16% | -16% | -1% |
| 462.libquantum | Quantum sim | +5% | -16% | -1% |
| 464.h264ref | Video encoding | +24% | -4% | -3% |
| 471.omnetpp | Network sim | +0% | -39% | 0% |
| 473.astar | Pathfinding | +30% | +3% | 0% |
| 483.xalancbmk | XML processing | +36% | -25% | 0% |
First of all, the most interesting datapoint was the fact that the code generated by gcc 8 seems to have improved vastly for the EPYC processors. We repeated the single threaded test three times, and the rate numbers show the same thing: it is very consistent.
hmmer is one of the more branch intensive benchmarks, and the other two workloads where the impact of branch prediction is higher (somewhat higher percentage of branch misses) - gobmk, sjeng - perform consistingly better on the second generation EPYC with it's new TAGE predictor.
Why the low IPC omnetpp ("network sim") does not show any improvement is a mystery to us, we expected that the larger L3 cache would help. However this is a test that loves very large caches, as a result the Intel Xeons have the advantage (38.5 - 55 MB L3).
The video encoding benchmark "h264ref" also relies somewhat on the L3 cache, but that benchmark relies much more on DRAM bandwidth. The fact that the EPYC 7002 has higher DRAM bandwidth is clearly visible.
The pointer chasing benchmarks – XML procesing and Path finding – performed less than optimal on the previous EPYC generation (compared to the Xeons), but show very significant improvements on EPYC 7002.
Facebook
Twitter
RSS
180 Comments
View All Comments
AnonCPU - Friday, August 9, 2019 - link
The gain in hmmer on EPYC with GCC8 is not due to TAGE predictor.Hmmer gains a lot on EPYC only because of GCC8.
GCC8 vectorizer has been improved in GCC8 and hmmer gets vectorized heavily while it was not the case for GCC7. The same run on an Intel machine would have shown the same kind of improvement.
JohanAnandtech - Sunday, August 11, 2019 - link
Thanks, do you have a source for that? Interested in learning more!AnonCPU - Monday, August 12, 2019 - link
That should be due to the improvements on loop distribution:https://gcc.gnu.org/gcc-8/changes.html
"The classic loop nest optimization pass -ftree-loop-distribution has been improved and enabled by default at -O3 and above. It supports loop nest distribution in some restricted scenarios;"
There are also some references here in what was missing for hmmer vectorization in GCC some years ago:
https://gcc.gnu.org/ml/gcc/2017-03/msg00012.html
And a page where you can see that LLVM was missing (at least in 2015) a good loop distribution algo useful for hmmer:
https://www.phoronix.com/scan.php?page=news_item&a...
AnonCPU - Monday, August 12, 2019 - link
And more:https://community.arm.com/developer/tools-software...
just4U - Friday, August 9, 2019 - link
I guess the question to ask now is can they churn these puppies out like no tomorrow? Is the demand there? What about other Hardware? Motherboards and the like..Do they have 100 000 of these ready to go? The window of opportunity for AMD is always fleeting.. and if their going to capitalize on this they need to be able to put the product out there.
name99 - Friday, August 9, 2019 - link
No obvious reason why not. The chiplets are not large and TSMC ships 200 million Apple chips a year on essentially the same process. So yields should be there.Manufacturing the chiplet assembly also doesn't look any different from the Naples assembly (details differ, yes, but no new envelopes being pushed: no much higher frequency signals or denser traces -- the flip side to that is that there's scope there for some optimization come Milan...)
So it seems like there is nothing to obviously hold them back...
fallaha56 - Saturday, August 10, 2019 - link
Perhaps Hypertheading should be off on the Intel systems to better reflect eg Google’s reality / proper security standards now we know Intel isn’t secure?Targon - Monday, August 12, 2019 - link
That is why Google is going to be buying many Epyc based servers going forward. Mitigations do not mean a problem has been fixed.imaskar - Wednesday, August 14, 2019 - link
Why do you think AWS, GCP, Azure, etc. mitigated the vulnerabilities? They only patched Meltdown at most. All other things are too costly and hard to execute. They just don't care so much for your data. Too loose 2x cloud capacity for that? No way. And for security conscious serious customers they offer private clusters, so your workloads run on separate servers.ballsystemlord - Saturday, August 10, 2019 - link
Spelling and grammar errors:"This happened in almost every OS, and in some cases we saw reports that system administrators and others had to do quite a bit optimization work to get the best performance out of the EPYC 7001 series."
Missing "of":
"This happened in almost every OS, and in some cases we saw reports that system administrators and others had to do quite a bit of optimization work to get the best performance out of the EPYC 7001 series."
"...to us it is simply is ridiculous that Intel expect enterprise users to cough up another few thousand dollars per CPU for a model that supports 2 TB,..."
Excess "is" and missing "s":
"...to us it is simply ridiculous that Intel expects enterprise users to cough up another few thousand dollars per CPU for a model that supports 2 TB,..."
"Although the 225W TDP CPUs needs extra heatspipes and heatsinks, there are still running on air cooling..."
Excess "s" and incorrect "there",
"Although the 225W TDP CPUs need extra heatspipes and heatsinks, they're still running on air cooling..."
"The Intel L3-cache keeps latency consistingy low as long as you stay within the L3-cache."
"consistently" not "consistingy":
"The Intel L3-cache keeps latency consistently low as long as you stay within the L3-cache."
"For example keeping a large part of the index in the cache improve performance..."
Missing comma and missing "s" (you might also consider making cache plural, but you seem to be talking strictly about the L3):
"For example, keeping a large part of the index in the cache improves performance..."
"That is a real thing is shown by the fact that Intel states that the OLTP hammerDB runs 60% faster on a 28-core Intel Xeon 8280 than on EPYC 7601."
Missing "it":
"That it is a real thing is shown by the fact that Intel states that the OLTP hammerDB runs 60% faster on a 28-core Intel Xeon 8280 than on EPYC 7601."
In general, the beginning of the sentance appears quite poorly worded, how about:
"That L3 cache latency is a matter for concern is shown by the fact that Intel states that the OLTP hammerDB runs 60% faster on a 28-core Intel Xeon 8280 than on EPYC 7601."
"In NPS4, the NUMA domains are reported to software in such a way as it chiplets always access the near (2 channels) DRAM."
Missing "s":
"In NPS4, the NUMA domains are reported to software in such a way as its chiplets always access the near (2 channels) DRAM."
"The fact that the EPYC 7002 has higher DRAM bandwidth is clearly visible."
Wrong numbers (maybet you ment, series?):
"The fact that the EPYC 7742 has higher DRAM bandwidth is clearly visible."
"...but show very significant improvements on EPYC 7002."
Wrong numbers (maybet you ment, series?):
"...but show very significant improvements on EPYC 7742."
"Using older garbage collector because they happen to better at Specjbb"
Badly worded.
"Using an older garbage collector because it happens to be better at Specjbb"
"For those with little time: at the high end with socketed x86 CPUs, AMD offers you up to 50 to 100% higher performance while offering a 40% lower price."
"Up to" requires 1 metric, not 2. Try:
"For those with little time: at the high end with socketed x86 CPUs, AMD offers you from 50 up to 100% higher performance while offering a 40% lower price."