Core-to-Core Latency

As the core count of modern CPUs is growing, we are reaching a time when the time to access each core from a different core is no longer a constant. Even before the advent of heterogeneous SoC designs, processors built on large rings or meshes can have different latencies to access the nearest core compared to the furthest core. This rings true especially in multi-socket server environments.

But modern CPUs, even desktop and consumer CPUs, can have variable access latency to get to another core. For example, in the first generation Threadripper CPUs, we had four chips on the package, each with 8 threads, and each with a different core-to-core latency depending on if it was on-die or off-die. This gets more complex with products like Lakefield, which has two different communication buses depending on which core is talking to which.

If you are a regular reader of AnandTech’s CPU reviews, you will recognize our Core-to-Core latency test. It’s a great way to show exactly how groups of cores are laid out on the silicon. This is a custom in-house test, and we know there are competing tests out there, but we feel ours is the most accurate to how quick an access between two cores can happen.


Click to enlarge (lots of cores and threads = lots of core pairings)

Comparing core to core latencies from Zen 4 (7950X) and Zen 3 (5950X), both are using a two CCX 8-core chiplet design, which is a marked improvement over the four CCX 16-core design featured on the Zen 2 microarchitecture, the Ryzen 9 3950X. The inter-core latencies within the L3 cache range from between 15 ns and 19 ns. The inter-core latencies between different cores within different parts of the CCD show a larger latency penalty of up to 79.5 ns, which is something AMD should work on going forward, but it's an overall improvement in cross CCX latencies compared to Zen 3. Any gain is still a gain.

Even though AMD has opted for a newer and more 'efficient' IOD which is based on TSMC's 6 nm node. It is around the same size physically as the previous AMD IOD on Zen 3 manufactured on GlobalFoundries 12 nm node, but with a much larger transistor count. Within the IOD is the newly integrated RDNA 2 graphics, although this isn't typical iGPU in the sense that an APU is. A lot of the room on the IOD is made up of the DDR5 memory controller or IMC, as well as the chips PCIe 5.0 lanes, and of course, connects to the logic through its primary interconnect named Infinity Fabric. All of these variables play a part on power, latency, and operation.


AMD Ryzen 9 5950X Core-to-Core Latency results

It's actually astounding how similar the latency performance of the Ryzen 9 7950X (Zen 4) is when compared directly to the Ryzen 9 5950X (Zen 3), despite being on the new 5 nm TSMC manufacturing process. Even with a change of IOD, but with the same interconnect, the inter-core latencies within the Ryzen 9 7950X are great in terms of cores within the same core complex; latency does degrade when pairing up with a core in another chiplet, but this works and AMD's Ryzen 5000 series proved that the overall penalty performance is negatable.

Test Bed and Setup SPEC2017 Single-Threaded Results
POST A COMMENT

205 Comments

View All Comments

  • Otritus - Wednesday, September 28, 2022 - link

    Anandtech has almost always tested at the manufacturer listed specification rather than a recommended overclock. Back in the day this methodology was disadvantageous to Intel as AMD could use DDR3-2133 while Intel was limited to DDR3-1600. If AMD was really so confident that Ryzen 7000 could run at DDR5-6000 cl36, then they should list that. Not say that 2DPC is only able to support DDR5-3600, and 1DPC can only guarantee DDR5-5200. As for the 2080Ti that’s just because the GPU lab burned down, so Anandtech doesn’t do GPU reviews anymore, and companies don’t want to send faster GPUs to not get reviewed. Reply
  • misan - Tuesday, September 27, 2022 - link

    Could you update the article to include the SPEC geomean totals (for int and fp) like in previous reviews (makes comparisons easier). Also, what is the actual observed package power when running CB23 in the 65W TDP mode? Reply
  • Blastdoor - Tuesday, September 27, 2022 - link

    Great idea!

    I’d also like to see more benchmarks — at least SPEC if nothing else — in ECO mode
    Reply
  • haplo602 - Tuesday, September 27, 2022 - link

    I hope there will be a dedicated article exploring the ECO Mode, its limitations and what effects it has on perf/power/temps. This looks like the best feature of the generation so far. You can on demand change personality of the same CPU from energy efficient to maximum performance without having to buy different models. Reply
  • Foeketijn - Tuesday, September 27, 2022 - link

    +1 Reply
  • Blastdoor - Tuesday, September 27, 2022 - link

    Agreed. I understand that AMD feels they have to compete with Intel for the absolute performance crown, even if that means insane temperatures. But I bet there are plenty of folks who would view 80% of the performance for just 40% of the watts as a good tradeoff. Reply
  • Gigaplex - Wednesday, September 28, 2022 - link

    Feature of the generation? ECO mode exists on older Ryzen systems too. Reply
  • Blastdoor - Wednesday, September 28, 2022 - link

    I guess the space heater race with Intel has just made it more relevant than ever before Reply
  • amon.akira - Tuesday, September 27, 2022 - link

    "the reality is that under sustained load, depending on the aggressiveness of the cooling, is more around 5.4 GHz."

    5.4 is allcore or ST sustained?
    Reply
  • TEAMSWITCHER - Tuesday, September 27, 2022 - link

    If you are already gaming at 4K, then these CPU upgrades are nearly worthless. Yet AMD (and soon Intel) is heavily targeting GAMERS in all their promotions on these new products. WHY ON EARTH DO THEY EVEN TRY?

    NVidia was gang beaten by You Tubers last week, but they put out new 4000 series products that will do far more for gaming and productivity at the same time.. Blender runs so much faster using Nvidia Optix and an RTX GPU. This industry is broken, when it attacks companies delivering REAL performance games, and then praises companies giving consumers a pittance.

    HEDTis the platform I want. Nvidia is the GPU that delivers. The rest is just garbage.
    Reply

Log in

Don't have an account? Sign up now