Core-to-Core Latency

As the core count of modern CPUs is growing, we are reaching a time when the time to access each core from a different core is no longer a constant. Even before the advent of heterogeneous SoC designs, processors built on large rings or meshes can have different latencies to access the nearest core compared to the furthest core. This rings true especially in multi-socket server environments.

But modern CPUs, even desktop and consumer CPUs, can have variable access latency to get to another core. For example, in the first generation Threadripper CPUs, we had four chips on the package, each with 8 threads, and each with a different core-to-core latency depending on if it was on-die or off-die. This gets more complex with products like Lakefield, which has two different communication buses depending on which core is talking to which.

If you are a regular reader of AnandTech’s CPU reviews, you will recognize our Core-to-Core latency test. It’s a great way to show exactly how groups of cores are laid out on the silicon. This is a custom in-house test, and we know there are competing tests out there, but we feel ours is the most accurate to how quick an access between two cores can happen.


Click to enlarge (lots of cores and threads = lots of core pairings)

Comparing core to core latencies from Zen 4 (7950X) and Zen 3 (5950X), both are using a two CCX 8-core chiplet design, which is a marked improvement over the four CCX 16-core design featured on the Zen 2 microarchitecture, the Ryzen 9 3950X. The inter-core latencies within the L3 cache range from between 15 ns and 19 ns. The inter-core latencies between different cores within different parts of the CCD show a larger latency penalty of up to 79.5 ns, which is something AMD should work on going forward, but it's an overall improvement in cross CCX latencies compared to Zen 3. Any gain is still a gain.

Even though AMD has opted for a newer and more 'efficient' IOD which is based on TSMC's 6 nm node. It is around the same size physically as the previous AMD IOD on Zen 3 manufactured on GlobalFoundries 12 nm node, but with a much larger transistor count. Within the IOD is the newly integrated RDNA 2 graphics, although this isn't typical iGPU in the sense that an APU is. A lot of the room on the IOD is made up of the DDR5 memory controller or IMC, as well as the chips PCIe 5.0 lanes, and of course, connects to the logic through its primary interconnect named Infinity Fabric. All of these variables play a part on power, latency, and operation.


AMD Ryzen 9 5950X Core-to-Core Latency results

It's actually astounding how similar the latency performance of the Ryzen 9 7950X (Zen 4) is when compared directly to the Ryzen 9 5950X (Zen 3), despite being on the new 5 nm TSMC manufacturing process. Even with a change of IOD, but with the same interconnect, the inter-core latencies within the Ryzen 9 7950X are great in terms of cores within the same core complex; latency does degrade when pairing up with a core in another chiplet, but this works and AMD's Ryzen 5000 series proved that the overall penalty performance is negatable.

Test Bed and Setup SPEC2017 Single-Threaded Results
POST A COMMENT

205 Comments

View All Comments

  • Tom Sunday - Friday, September 30, 2022 - link

    Just today received a special sales notice from Micro Center giving away FREE 32GB DDR5 with any purchase of a Ryzen 7000 series CPU. I wonder if AMD is sponsering such a sales push and this early in the game? Giving away a $190 value is a big deal in the trying times of today! Reply
  • Castillan - Sunday, October 2, 2022 - link

    I suspect that's a Microcenter specific deal only. The RAM is 5600 at a fairly high latency (I think it was CAS40?). DDR5 prices have plummeted as well. The memory I picked up from Microcenter was 6600/CAS34 and marked down to 279 from 499.

    I'd guess that they have a surplus of a certain stock item that wasn't selling, and decided to use this promo to offload unwanted stock and still look good.
    Reply
  • imaskar - Friday, September 30, 2022 - link

    It would be really great to add code compilation tests: Java, Go, C++ (linux kernel), Rust. Reply
  • dizzynosed - Saturday, October 1, 2022 - link

    Si what shall I buy? Intel, amd, ??? Which cpu?? I only game. Reply
  • rocky12345 - Saturday, October 1, 2022 - link

    What's wrong with the gaming scores on the 7000 series there is no way a 5000 series should be able to match or beat a 7000 AMD CPU. I know this because I have a AMD Ryzen 5900x properly setup and tweaked. AMD is said to have sent DDR5 6000 with the test CPU's and asked the reviewers to use that to test with. Lets face it 97% of the people buying a new AMD Zen 4 setup or Intel 12th gen are not going to be using bargain basement low speed ram and if they do happen to buy cheaper ram most are more than likely to try and run it at the highest speed possible. did I read that right you used CL44 DDR5 5200Mhz talk about dead heading performance.

    Also maybe I missed it but what was the Intel test system setup? other than that it was a decent review. I never have seen Ryzen 5000 that close in gaming I guess using slow DDR5 knee jerks Ryzen 7000. My own ram is running at CL16 4000Mhz 2000IF and at the reported number in the review if I had the same video card I would be either faster or only slightly slower than the test results here for games and that would give me false hope that my Zen 3 was faster than it really is lol.
    Reply
  • Oxford Guy - Sunday, October 2, 2022 - link

    The only way you're going to see movement on this is if you lobby AMD to abandon JEDEC.

    This site sees JEDEC as all there is.
    Reply
  • GeoffreyA - Monday, October 3, 2022 - link

    I think it's about keeping a common baseline of memory speed, especially since Anandtech's database is about having parts directly comparable. Reply
  • Oxford Guy - Monday, October 10, 2022 - link

    That’s not the reason that has been given again and again and it’s a terrible one anyway. The parts are different. The memory that goes best with those parts differs. Reply
  • GeoffreyA - Tuesday, October 11, 2022 - link

    They should have set all the systems to DDR4 3200 and called it a day. Reply
  • byte99 - Sunday, October 2, 2022 - link

    I'm a bit confused. When Anandtech was doing their efficiency analysis, it seemed they were taking the 65W Eco mode label as the actual package power, instread of actually measuring it (as they usually do). When Ars Technica measured the package power of the 7950X and 7600X in 65w Eco Mode, they found it was 90W for both.

    [ https://arstechnica.com/gadgets/2022/09/ryzen-7600... ]

    Did Anandtech miss something obvious, or am I missing something?
    Reply

Log in

Don't have an account? Sign up now