Core-to-Core Latency

As the core count of modern CPUs is growing, we are reaching a time when the time to access each core from a different core is no longer a constant. Even before the advent of heterogeneous SoC designs, processors built on large rings or meshes can have different latencies to access the nearest core compared to the furthest core. This rings true especially in multi-socket server environments.

But modern CPUs, even desktop and consumer CPUs, can have variable access latency to get to another core. For example, in the first generation Threadripper CPUs, we had four chips on the package, each with 8 threads, and each with a different core-to-core latency depending on if it was on-die or off-die. This gets more complex with products like Lakefield, which has two different communication buses depending on which core is talking to which.

If you are a regular reader of AnandTech’s CPU reviews, you will recognize our Core-to-Core latency test. It’s a great way to show exactly how groups of cores are laid out on the silicon. This is a custom in-house test, and we know there are competing tests out there, but we feel ours is the most accurate to how quick an access between two cores can happen.


Click to enlarge (lots of cores and threads = lots of core pairings)

Comparing core to core latencies from Zen 4 (7950X) and Zen 3 (5950X), both are using a two CCX 8-core chiplet design, which is a marked improvement over the four CCX 16-core design featured on the Zen 2 microarchitecture, the Ryzen 9 3950X. The inter-core latencies within the L3 cache range from between 15 ns and 19 ns. The inter-core latencies between different cores within different parts of the CCD show a larger latency penalty of up to 79.5 ns, which is something AMD should work on going forward, but it's an overall improvement in cross CCX latencies compared to Zen 3. Any gain is still a gain.

Even though AMD has opted for a newer and more 'efficient' IOD which is based on TSMC's 6 nm node. It is around the same size physically as the previous AMD IOD on Zen 3 manufactured on GlobalFoundries 12 nm node, but with a much larger transistor count. Within the IOD is the newly integrated RDNA 2 graphics, although this isn't typical iGPU in the sense that an APU is. A lot of the room on the IOD is made up of the DDR5 memory controller or IMC, as well as the chips PCIe 5.0 lanes, and of course, connects to the logic through its primary interconnect named Infinity Fabric. All of these variables play a part on power, latency, and operation.


AMD Ryzen 9 5950X Core-to-Core Latency results

It's actually astounding how similar the latency performance of the Ryzen 9 7950X (Zen 4) is when compared directly to the Ryzen 9 5950X (Zen 3), despite being on the new 5 nm TSMC manufacturing process. Even with a change of IOD, but with the same interconnect, the inter-core latencies within the Ryzen 9 7950X are great in terms of cores within the same core complex; latency does degrade when pairing up with a core in another chiplet, but this works and AMD's Ryzen 5000 series proved that the overall penalty performance is negatable.

Test Bed and Setup SPEC2017 Single-Threaded Results
POST A COMMENT

205 Comments

View All Comments

  • vortmax2 - Sunday, October 16, 2022 - link

    Some people don't want to limit TDP themselves. Nothing wrong with that. Reply
  • Techie2 - Tuesday, September 27, 2022 - link

    What a screwed up launch of Ryzen 7000 CPUs and AM5 mobos by e-tailers. DDR5 EXPO DRAM showed up online a few days ago. On 9-27-22 it looks like e-tailers are actually hiding the four Ryzen 7000 CPUs to sell older stock. The AM5 mobos which have been sitting in inventory for weeks were not posted online until early morning instead of at midnight as in the past. You'd think by now they could figure out how to do a proper launch of a new CPU or platform but evidently not when it's AMD.

    No consumer grade Gen 5 SSDs listed by e-tailers that I could find. Are PC builders suppose to just wait until Nov. to see if they actually show up? AMD's partners may be cooperating with AMD but the purchasing experience is a piss poor sales methodology being employed IMNHO.

    YMMV
    Reply
  • nandnandnand - Tuesday, September 27, 2022 - link

    AMD said weeks ago that PCIe 5.0 SSDs would be coming in November.

    Nobody should be buying this stuff on day 1 unless they like being findom'd by corporations.
    Reply
  • yhselp - Tuesday, September 27, 2022 - link

    First-draft-copy issues aside, this article is written exceptionally well. The information is excellent and extensive as usual, but I feel like there's been a step-up in the way its presented/explained. Kudos and thank you. Reply
  • yhselp - Tuesday, September 27, 2022 - link

    it's* goddamit Reply
  • vortmax2 - Sunday, October 16, 2022 - link

    Great post. So many grammar police on here that can't help but criticize and take away from the actual purpose of the article. Reply
  • HardwareDufus - Tuesday, September 27, 2022 - link

    Clearly both manufacturers are producing vey compelling products this time around. At the $600-$700 mark we have two CPUs trading blows; R9-7950X (Zen4) & just announced i9-13900K (RaptorLake) We will have updates of both lines, with AMD adding 3D Cache and Intel increasing Boost Speeds. Probably can't go wrong with either choice, neither one dominates the other completely.

    I think I'll bite this time around. Yeah I know Zen5 will be a new architecture and Intel will adopt chiplets and all of the benefits that accompany them... But, I think either one of these chips, probably available in volume in 1st quarter 2023, will serve most folks well.

    In the case of Intel, you can continue to use DD4 and 600 series chipsets. However really take advantage of the capabilities of the chip, DDR5 and motherboards featureing 700 series chipsets will be available, on par with Zen 4 requiring a DDR5 and otherboards featuring the new AM5 socket and 600 series chipsets. Apples to Apples when comparing the requirements to go ALL In on performance.
    Reply
  • HardwareDufus - Tuesday, September 27, 2022 - link

    Dang, that was some awesome typing I just did there... Reply
  • nandnandnand - Tuesday, September 27, 2022 - link

    3D cache will dominate over a couple hundred extra MHz in frequency (13900KS). Reply
  • Hifihedgehog - Tuesday, September 27, 2022 - link

    @nandnandnand: That may well be true, but that's the future and months out yet. Ryzen 7000 non X3D has to sell between then and now or AMD is not going to be posting a pretty quarter. The 13900K is far cheaper in platform and unit price and will meet or exceed the 7950X for now. DDR5 and AM5 motherboards coupled with a higher price will be Ryzen 7000 series undoing, and good too. AMD needs to realize people purchase because of intrinsic quality, not brand loyalty. Reply

Log in

Don't have an account? Sign up now