AMD Rome Second Generation EPYC Review: 2x 64-core Benchmarked
by Johan De Gelas on August 7, 2019 7:00 PM ESTMemory Subsystem: Latency
AMD chose to share a core design among mobile, desktop and server for scalability and economic reasons. The Core Complex (CCX) is still used in Rome like it was in the previous generation.
What has changed is that each CCX communicates with the central IO hub, instead of four dies communicating in 4 node NUMA layout (This option is still available to use via the NPS4 switch, keeping each CCD local to its quadrant of the sIOD as well as those local memory controllers, avoiding hops between sIOD quadrants which encour a slight latency penalty). So as the performance of modern CPUs depends heavily on the cache subsystem, we were more than curious what kind of latency a server thread would see as it accesses more and more pages in the cache hierarchy.
We're using our own in-house latency test. In particular what we're interested in publishing is the estimated structural latency of the processors, meaning we're trying to account for TLB misses and disregard them in these numbers, except for the DRAM latencies where latency measurements get a bit more complex between platforms, and we revert to full random figures.
| Mem Hierarchy |
AMD EPYC 7742 DDR4-3200 (ns @ 3.4GHz) |
AMD EPYC 7601 DDR4-2400 (ns @ 3.2GHz) |
Intel Xeon 8280 DDR-2666 (ns @ 2.7GHz) |
|
| L1 Cache | 32KB 4 cycles 1.18ns |
32KB 4 cycles 1.25ns |
32KB 4 cycles 1.48ns |
|
| L2 Cache | 512KB 13 cycles 3.86ns |
512KB 12 cycles 3.76ns |
1024KB 14 cycles 5.18ns |
|
| L3 Cache | 16MB / CCX (4C) 256MB Total ~34 cycles (avg) ~10.27 ns |
16MB / CCX (4C) 64MB Total |
38.5MB / (28C) Shared ~46 cycles (avg) ~17.5ns |
|
| DRAM 128MB Full Random |
~122ns (NPS1) ~113ns (NPS4) |
~116ns |
~89ns |
|
| DRAM 512MB Full Random |
~134ns (NPS1) ~125ns (NPS4) |
~109ns |
||
Update 2019/10/1: We've discovered inaccuracies with our originally published latency numbers, and have subsequently updated the article with more representative figures with a new testing tool.
Things get really interesting when starting to look at cache depths beyond the L2. Naturally Intel here this happens at 1MB while for AMD this is after 512KB, however AMD’s L2 has a speed advantage over Intel’s larger cache.
Where AMD has an ever more clearer speed advantage is in the L3 caches that are clearly significantly faster than Intel’s chips. The big difference here is that AMD’s L3’s here are only local to a CCX of 4 cores – for the EPYC 7742 this is now doubled to 16MB up from 8MB on the 7601.
Currently this is a two-edged sword for the AMD platforms: On one hand, the EPYC processors have significantly more total cache, coming in at a whopping 256MB for the 7742, quadruple the amount over the 64MB of the 7601, and a lot more than Intel’s platforms, which come in at 38.5MB for the Xeon 8180, 8176, 8280, and a larger 55MB for the Xeon E5-2699 v4.
The disadvantage for AMD is that while they have more cache, the EPYC 7742 rather consist of 16 CCX which all have a very fast 16 MB L3. Although the 64 cores are one big NUMA node now, the 64-core chip is basically 16x 4 cores, each with 16 MB L3-caches. Once you get beyond that 16 MB cache, the prefetchers can soften the blow, but you will be accessing the main DRAM.
A little bit weird is the fact that accessing data that resides at the same die (CCD) but is not within the same CCX is just as slow as accessing data is on a totally different die. This is because regardless of where the other CCX is, whether it is nearby on the same die or on the other side of the chip, the data access still has to go through the IF to the IO die and back again.
Is that necessarily a bad thing? The answer: most of the time it is not. First of all, in most applications only a low percentage of accesses must be answered by the L3 cache. Secondly, each core on the CCX has no less than 4 MB of L3 available, which is far more than the Intel cores have at their disposal (1.375 MB). The prefetchers have a lot more space to make sure that the data is there before it is needed.
But database performance might still suffer somewhat. For example, keeping a large part of the index in the cache improve performance, and especially OLTP accesses tend to quite random. Secondly the relatively slow communication over a central hub slow down synchronization communication. That is a real thing is shown by the fact that Intel states that the OLTP hammerDB runs 60% faster on a 28-core Intel Xeon 8280 than on EPYC 7601. We were not able to check it before the deadline, but it seems reasonable.
But for the vast majority of these high-end CPUs, they will be running many parallel applications, like running microservices, docker containers, virtual machines, map/reducing smaller chunks of data and parallel HPC Jobs. In almost all cases 16 MB L3 for 4 cores is more than enough.
Although come to think of it, when running an 8-core virtual machine there might be small corner cases where performance suffers a (little) bit.
In short, AMD leaves still a bit of performance on table by not using a larger 8-core CCX. We await to see what happens in future platforms.
Facebook
Twitter
RSS
180 Comments
View All Comments
AnonCPU - Friday, August 9, 2019 - link
The gain in hmmer on EPYC with GCC8 is not due to TAGE predictor.Hmmer gains a lot on EPYC only because of GCC8.
GCC8 vectorizer has been improved in GCC8 and hmmer gets vectorized heavily while it was not the case for GCC7. The same run on an Intel machine would have shown the same kind of improvement.
JohanAnandtech - Sunday, August 11, 2019 - link
Thanks, do you have a source for that? Interested in learning more!AnonCPU - Monday, August 12, 2019 - link
That should be due to the improvements on loop distribution:https://gcc.gnu.org/gcc-8/changes.html
"The classic loop nest optimization pass -ftree-loop-distribution has been improved and enabled by default at -O3 and above. It supports loop nest distribution in some restricted scenarios;"
There are also some references here in what was missing for hmmer vectorization in GCC some years ago:
https://gcc.gnu.org/ml/gcc/2017-03/msg00012.html
And a page where you can see that LLVM was missing (at least in 2015) a good loop distribution algo useful for hmmer:
https://www.phoronix.com/scan.php?page=news_item&a...
AnonCPU - Monday, August 12, 2019 - link
And more:https://community.arm.com/developer/tools-software...
just4U - Friday, August 9, 2019 - link
I guess the question to ask now is can they churn these puppies out like no tomorrow? Is the demand there? What about other Hardware? Motherboards and the like..Do they have 100 000 of these ready to go? The window of opportunity for AMD is always fleeting.. and if their going to capitalize on this they need to be able to put the product out there.
name99 - Friday, August 9, 2019 - link
No obvious reason why not. The chiplets are not large and TSMC ships 200 million Apple chips a year on essentially the same process. So yields should be there.Manufacturing the chiplet assembly also doesn't look any different from the Naples assembly (details differ, yes, but no new envelopes being pushed: no much higher frequency signals or denser traces -- the flip side to that is that there's scope there for some optimization come Milan...)
So it seems like there is nothing to obviously hold them back...
fallaha56 - Saturday, August 10, 2019 - link
Perhaps Hypertheading should be off on the Intel systems to better reflect eg Google’s reality / proper security standards now we know Intel isn’t secure?Targon - Monday, August 12, 2019 - link
That is why Google is going to be buying many Epyc based servers going forward. Mitigations do not mean a problem has been fixed.imaskar - Wednesday, August 14, 2019 - link
Why do you think AWS, GCP, Azure, etc. mitigated the vulnerabilities? They only patched Meltdown at most. All other things are too costly and hard to execute. They just don't care so much for your data. Too loose 2x cloud capacity for that? No way. And for security conscious serious customers they offer private clusters, so your workloads run on separate servers.ballsystemlord - Saturday, August 10, 2019 - link
Spelling and grammar errors:"This happened in almost every OS, and in some cases we saw reports that system administrators and others had to do quite a bit optimization work to get the best performance out of the EPYC 7001 series."
Missing "of":
"This happened in almost every OS, and in some cases we saw reports that system administrators and others had to do quite a bit of optimization work to get the best performance out of the EPYC 7001 series."
"...to us it is simply is ridiculous that Intel expect enterprise users to cough up another few thousand dollars per CPU for a model that supports 2 TB,..."
Excess "is" and missing "s":
"...to us it is simply ridiculous that Intel expects enterprise users to cough up another few thousand dollars per CPU for a model that supports 2 TB,..."
"Although the 225W TDP CPUs needs extra heatspipes and heatsinks, there are still running on air cooling..."
Excess "s" and incorrect "there",
"Although the 225W TDP CPUs need extra heatspipes and heatsinks, they're still running on air cooling..."
"The Intel L3-cache keeps latency consistingy low as long as you stay within the L3-cache."
"consistently" not "consistingy":
"The Intel L3-cache keeps latency consistently low as long as you stay within the L3-cache."
"For example keeping a large part of the index in the cache improve performance..."
Missing comma and missing "s" (you might also consider making cache plural, but you seem to be talking strictly about the L3):
"For example, keeping a large part of the index in the cache improves performance..."
"That is a real thing is shown by the fact that Intel states that the OLTP hammerDB runs 60% faster on a 28-core Intel Xeon 8280 than on EPYC 7601."
Missing "it":
"That it is a real thing is shown by the fact that Intel states that the OLTP hammerDB runs 60% faster on a 28-core Intel Xeon 8280 than on EPYC 7601."
In general, the beginning of the sentance appears quite poorly worded, how about:
"That L3 cache latency is a matter for concern is shown by the fact that Intel states that the OLTP hammerDB runs 60% faster on a 28-core Intel Xeon 8280 than on EPYC 7601."
"In NPS4, the NUMA domains are reported to software in such a way as it chiplets always access the near (2 channels) DRAM."
Missing "s":
"In NPS4, the NUMA domains are reported to software in such a way as its chiplets always access the near (2 channels) DRAM."
"The fact that the EPYC 7002 has higher DRAM bandwidth is clearly visible."
Wrong numbers (maybet you ment, series?):
"The fact that the EPYC 7742 has higher DRAM bandwidth is clearly visible."
"...but show very significant improvements on EPYC 7002."
Wrong numbers (maybet you ment, series?):
"...but show very significant improvements on EPYC 7742."
"Using older garbage collector because they happen to better at Specjbb"
Badly worded.
"Using an older garbage collector because it happens to be better at Specjbb"
"For those with little time: at the high end with socketed x86 CPUs, AMD offers you up to 50 to 100% higher performance while offering a 40% lower price."
"Up to" requires 1 metric, not 2. Try:
"For those with little time: at the high end with socketed x86 CPUs, AMD offers you from 50 up to 100% higher performance while offering a 40% lower price."