AMD Zen 2 Microarchitecture Analysis: Ryzen 3000 and EPYC Rome
by Dr. Ian Cutress on June 10, 2019 7:22 PM EST- Posted in
- CPUs
- AMD
- Ryzen
- EPYC
- Infinity Fabric
- PCIe 4.0
- Zen 2
- Rome
- Ryzen 3000
- Ryzen 3rd Gen
Cache and Infinity Fabric
If it hasn’t been hammered in already, the big change in the cache is the L1 instruction cache which has been reduced from 64 KB to 32 KB, but the associativity has increased from 4-way to 8-way. This change enabled AMD to increase the size of the micro-op cache from 2K entry to 4K entry, and AMD felt that this gave a better performance balance with how modern workloads are evolving.
The L1-D cache is still 32KB 8-way, while the L2 cache is still 512KB 8-way. The L3 cache, which is a non-inclusive cache (compared to the L2 inclusive cache), has now doubled in size to 16 MB per core complex, up from 8 MB. AMD manages its L3 by sharing a 16MB block per CCX, rather than enabling access to any L3 from any core.
Because of the increase in size of the L3, latency has increased slightly. L1 is still 4-cycle, L2 is still 12-cycle, but L3 has increased from ~35 cycle to ~40 cycle (this is a characteristic of larger caches, they end up being slightly slower latency; it’s an interesting trade off to measure). AMD has stated that it has increased the size of the queues handling L1 and L2 misses, although hasn’t elaborated as to how big they now are.
Infinity Fabric
With the move to Zen 2, we also move to the second generation of Infinity Fabric. One of the major updates with IF2 is the support of PCIe 4.0, and thus the increase of the bus width from 256-bit to 512-bit.
Overall efficiency of IF2 has improved 27% according to AMD, leading to a lower power per bit. As we move to more IF links in EPYC, this will become very important as data is transferred from chiplet to IO die.
One of the features of IF2 is that the clock has been decoupled from the main DRAM clock. In Zen and Zen+, the IF frequency was coupled to the DRAM frequency, which led to some interesting scenarios where the memory could go a lot faster but the limitations in the IF meant that they were both limited by the lock-step nature of the clock. For Zen 2, AMD has introduced ratios to the IF2, enabling a 1:1 normal ratio or a 2:1 ratio that reduces the IF2 clock in half.
This ratio should automatically come into play around DDR4-3600 or DDR4-3800, but it does mean that IF2 clock does reduce in half, which has a knock on effect with respect to bandwidth. It should be noted that even if the DRAM frequency is high, having a slower IF frequency will likely limit the raw performance gain from that faster memory. AMD recommends keeping the ratio at a 1:1 around DDR4-3600, and instead optimizing sub-timings at that speed.
Facebook
Twitter
RSS
216 Comments
View All Comments
wurizen - Friday, June 14, 2019 - link
flex^^^wurizen - Friday, June 14, 2019 - link
OMFG! I. Am. Not. Talking. About. Intel. Mesh.I. Am. Talking. About. Infinity. Fabric. High. Memory. Latency!
Now that I got that off my chest, let's proceed shall we...
OMFG!
L3 Cache? WTF!
Do you think you're so clever to talk about L3 cache to show off your knowledge as if to convince ppl here you know something? Nah, man!
WTF are you talking about L3 cache, dude? Come on, dude, get with the program.
The program is "Cross-CCX-High-Memory-Latency" with Infinity Fabric 1.0
And, games (BO3, BF1, BF4 from my testing) are what is affected by this high latency penalty in real-time. Imagine playing a game of BO3 while throughout the game, the game is "micro-pausing" "Micro-slow-motioning" repeatedly throughout the match? Yep, you got it, it makes it unplayeable.
In productive work like video editing, I would not see the high latency as an issue unless it affects "timeline editing" causing it to lag, as well.
I have heard some complain issues with it in audio editing with audio work. But I don't do that so I can't say.
As for "compute-intensive applications (y'know, real work)" --delatFx2
....
.....
......
You duh man, bruh! a real compute-intensive, man!
"This article mentions a Windows 10 patch to ensure that threads get assigned to the same CCX before going to the adjacent one." --deltaFx2
Uhhh... that won't fix it. Only AMD can fix it in Infinity Fabric 2.0 (Ryzen 2), if, indeed, AMD has fixed it. By making it faster! And/or, reducing that ~110ns latency to around 69ns.
Now, my question is, and you (deltaFx2) hasn't mentioned it in your wise-response to my comments is that SLIDE of "Raw Memory Performance" showing 69ns latency at 3200 Mhz RAM. Is that Raw memory performance Intra-CCX-Memory-Performance or Inter-core-Memory-Performance? Bada-boom, bish!
wurizen - Friday, June 14, 2019 - link
it's a problem ppl are having, if you search enough....Alistair - Wednesday, June 12, 2019 - link
those kinds of micro stutters are usually motherboard or most likely your windows installation causing it, reinstall windows, then try a different motherboard maybewurizen - Wednesday, June 12, 2019 - link
Wow, really? Re-install windows?I just wanna know (cough, cough Anand) what the Cross-CCX-Latency is for Ryzen 2 and Infinity Fabric 2.0.
If, it is still ~110ns like before.... well, guess what? 110 nano-effin-seconds is not fast enough. It's too HIGH a latency!
You can't update bios/motherboard or re-install windows, or get 6000 Mhz RAM (the price for that, tjo?) to fix it. (As shown in the graph for whatever "Raw Memory Latency" is for that 3200 Mhz RAM to 3600 Mhz stays at 69 ns and only at 37333 Mhz RAM does it drop to 67ns?).... This is the same result PCPER did with Ryzen IF 1.0 showing that getting Faster RAM at 3200 Mhz did not improve the Cross-CCX-Memory-Latency....
supdawgwtfd - Thursday, June 13, 2019 - link
O don't get any stutters with my 1600.As above. It's nothing to do with the CPU directly.
Something else is causing the problem.
deltaFx2 - Thursday, June 13, 2019 - link
How so you know for sure that the microstutter or whatever it is you think you are facing is due to the inter-ccx latency? Did you actually pin threads to CCXs to confirm this theory? Do you know when inter-ccx latency even comes into play? Inter-ccx latency ONLY matters for shared memory being modified by different threads; this should be a tiny fraction of your execution time, otherwise you are not much better going multithreaded. Moreover, Each CCX runs 8 threads so are you saying your game uses more than 8? That would be an interesting game indeed, given that intel's mainstream gaming CPUs don't have a problem on 4c8t.To me, you've just jumped the the gun and gone from "I have got some microstutter issues" to "I know PCPer ran some microbenchmark to find out the latency" to "that must be the problem". It does not follow.
FreckledTrout - Thursday, June 13, 2019 - link
I agree. If micro stutter from CCX latency was really occurring this would be a huge issue. These issues really have to be something unrelated.wurizen - Friday, June 14, 2019 - link
Another thing that was weird was GPU usage drop from 98% to like 0% in-game, midst-action, while I was playing... constantly, in a repeated pattern throughout the game... this is not a server or games hitching. we understand as gamers that a game will "hitch" once in a while. this is like "slow-motion" "micro-pause" thing happening through out the game. happens in single player (BF1) so I ruled out server-side. It's like the game goes in "slow-motion" for a second... not once or twice in a match, per se. But, throughout and in a repeated constant fashion... along with seeing GPU usage to accompany the effect dropping from 98% or so (normal) to 0% for split seconds (again, not once or twice in a match; but a constant, repeated pattern throughout the match)And, there are people having head-scratching issues similar to me with Ryzen CPU.
No one (cough, cough Anand; nor youtube tech tubers will address it) seems to address it tho.
But, I think that Ryzen 2 is coming out and if Cross-CCX-High-LAtency-Issue is the same, then we're bound to hear more. I'm sure.
I am thinking tech sites are giving AMD a chance... but not sure... doesn't matter tho. I got a 7700k (I wanted the 8-core thing when 1700x Ryzen came out) but its fine. Im not a fanboy. Just a techboy.... if anything...
wurizen - Friday, June 14, 2019 - link
The "micro-stutter" or "micro-pausing" is not once or twice (I get those with Intel, as well) but, a repeated, constant pattern throughout the match and round of game. The "micro-stutter" and "micro-pause" also "FEELS" different than what I felt with my prior 3700K CPU and current 7700K CPU. It's like a "micro-slow-motion." I am not making this up. I am not crazy!