Core-to-Core Latency

As the core count of modern CPUs is growing, we are reaching a time when the time to access each core from a different core is no longer a constant. Even before the advent of heterogeneous SoC designs, processors built on large rings or meshes can have different latencies to access the nearest core compared to the furthest core. This rings true especially in multi-socket server environments.

But modern CPUs, even desktop and consumer CPUs, can have variable access latency to get to another core. For example, in the first generation Threadripper CPUs, we had four chips on the package, each with 8 threads, and each with a different core-to-core latency depending on if it was on-die or off-die. This gets more complex with products like Lakefield, which has two different communication buses depending on which core is talking to which.

If you are a regular reader of AnandTech’s CPU reviews, you will recognize our Core-to-Core latency test. It’s a great way to show exactly how groups of cores are laid out on the silicon. This is a custom in-house test, and we know there are competing tests out there, but we feel ours is the most accurate to how quick an access between two cores can happen.


Click to enlarge (lots of cores and threads = lots of core pairings)

Comparing core to core latencies from Zen 4 (7950X) and Zen 3 (5950X), both are using a two CCX 8-core chiplet design, which is a marked improvement over the four CCX 16-core design featured on the Zen 2 microarchitecture, the Ryzen 9 3950X. The inter-core latencies within the L3 cache range from between 15 ns and 19 ns. The inter-core latencies between different cores within different parts of the CCD show a larger latency penalty of up to 79.5 ns, which is something AMD should work on going forward, but it's an overall improvement in cross CCX latencies compared to Zen 3. Any gain is still a gain.

Even though AMD has opted for a newer and more 'efficient' IOD which is based on TSMC's 6 nm node. It is around the same size physically as the previous AMD IOD on Zen 3 manufactured on GlobalFoundries 12 nm node, but with a much larger transistor count. Within the IOD is the newly integrated RDNA 2 graphics, although this isn't typical iGPU in the sense that an APU is. A lot of the room on the IOD is made up of the DDR5 memory controller or IMC, as well as the chips PCIe 5.0 lanes, and of course, connects to the logic through its primary interconnect named Infinity Fabric. All of these variables play a part on power, latency, and operation.


AMD Ryzen 9 5950X Core-to-Core Latency results

It's actually astounding how similar the latency performance of the Ryzen 9 7950X (Zen 4) is when compared directly to the Ryzen 9 5950X (Zen 3), despite being on the new 5 nm TSMC manufacturing process. Even with a change of IOD, but with the same interconnect, the inter-core latencies within the Ryzen 9 7950X are great in terms of cores within the same core complex; latency does degrade when pairing up with a core in another chiplet, but this works and AMD's Ryzen 5000 series proved that the overall penalty performance is negatable.

Test Bed and Setup SPEC2017 Single-Threaded Results
POST A COMMENT

205 Comments

View All Comments

  • Silver5urfer - Tuesday, September 27, 2022 - link

    Intel won't sell new mobos. They already have Z690 saturation. Barely anyone will get Z790. AMD on the other hand will continue to sell new boards, the quarter is not based on the Client only. It will include the HPC. Intel lost money there, and AMD won't be losing because Genoa is on track and SPR XEON is delayed.

    AMD AM5 is not just hey this thing is fast and just for gaming. It will be a socket that is going to last until Intel Nova Lake launches that is next 2 Intel sockets. That is a huge advantage for a small price for paying customers now.

    Also why is everyone chanting same BS that GN Steve did with AMD boards are too expensive, did you see how Z690 was at when it launched same thing it was expensive ? And DDR4 boards are worse quality and features than the premium cut DDR5. Then Intel launched B660 and AMD's B650/E is also coming. So nope that BS argument about Mobo pricing is too much thrown around. Once the B650 launches by that time 13th gen will hit Retail market and new GPUs as well. And it's November season and in America the Black Friday sales will kick in and see price cut for all products we are seeing now.

    So ultimately AMD is not going to lose money.

    The biggest BS from a smart customer pov is with Intel LGA1700 EOL and the whole socket bending crap, it's like AM4's unreliable IMC and poor IODie with it's issues. AM5 needs to prove itself but given how they removed the IF from memory clocks I can bet it won't have the issues from AM4.

    X3D is a niche market it won't be chart topper for sales at-least if it's again 7800X3D single SKU. Same for KS bin. It depends on how AMD will execute, idk why every single AMD fan says X3D is going to do something if AMD can clock it this high and also allow tuning then it will be a true gen refresh to compete vs Meteor Lake else it will be just a Gaming Juggernaut.
    Reply
  • nandnandnand - Tuesday, September 27, 2022 - link

    @Silver5urfer rumored to be 3 SKUs, including a 7900X3D, and +30% average performance instead of 15%. I guess that would be a result of improved latency, bandwidth, no voltage/clock decreases, etc. Reply
  • Silver5urfer - Wednesday, September 28, 2022 - link

    A 7950X3D means it will have extreme high heat because not only single cache stack you are adding 2 stacks atop the CCDs, how will AMD able to remove that ? Unless the way Cowos TSMC Stacking is technically changed OR they have to lap out the IHS internally to reduce the thickness and compensate the high heat transfer. The current IHS is thick due to many reasons one can assume - The LGA1718 stability, Chiplet integrity with high heat and pressure of HS and cooler compat and it causes the heat density increase, which is why 95C.

    I really think a 7800X3D is the only way for AMD even though rumors mention 3SKUs because a total SKU refresh totally cannibalize the entire 7000 lineup, because a 7600X is to get best gaming out of AM5 with cheaper option almost at more than 1/2 the price reduction vs a top end R9. And R7 7900X is basically an all rounder like 5900X best for gaming and production now you add the Cache block it would have to fight with 7900X.

    Voltage reduction was done on Zen 3 because AMD shoved 1.4v through all Ryzen 5000 processors, insanely high and IODie was also on high voltage, causing all that instability add the 1.3v bin silicon, everything gets better including the heat density. Zen 4 TSMC 5N is much better because it's just 1.2v now at high clock rate. The voltage is not an issue anymore, the design of the Zen 4 itself is like this, how AMD intended to breathe fire at 95C even for 7600X is the hint.
    Reply
  • nandnandnand - Wednesday, September 28, 2022 - link

    Heat was never the problem for the 5800X3D. It was only voltage, due to using an immature 3D (2.5D) chiplet technology that could not be run at the higher voltages. So I don't think the 7950X3D can't happen. If they have to drop voltages and clocks again, then hopefully the cache has improved.

    I think AMD should do at least a 7950X3D and 7800X3D. They can prevent cannibalization by giving it a healthy price bump. Probably +$100 to the 7950X3D, +$50 to the 7800X3D, and let the 7700X price drift lower. 7900X3D doesn't make sense, and people would love a 7600X3D but AMD would not.
    Reply
  • nandnandnand - Tuesday, September 27, 2022 - link

    @Hifihedgehog OP compared 7000X3D to the 13900KS, that's what I addressed. Reply
  • Hifihedgehog - Tuesday, September 27, 2022 - link

    Wrong: the i9-13900K is less than $600. The 7950X is going to have to have its price lowered, especially with the price of DDR5 and the motherboards simply off the charts. And good too: Lisa Su needs to be running a price war and not pretend that her company has more market share. Reply
  • The Von Matrices - Tuesday, September 27, 2022 - link

    A price war doesn't benefit AMD when they are supply constrained by TSMC and selling every chip they can manufacture. There's a reason that AMD doesn't offer any products in the <=$100 CPU market right now and it isn't because they don't want to make money. Reply
  • Hifihedgehog - Tuesday, September 27, 2022 - link

    https://download.intel.com/newsroom/2022/2022innov... Reply
  • dwade123 - Tuesday, September 27, 2022 - link

    Overheated and overpriced. Don't let those scumbags tell you that "95C is normal" because it's not. Avoid at all cost! Reply
  • Thanny - Tuesday, September 27, 2022 - link

    Running the memory at JEDEC speeds is definitely the wrong choice for a review. While it may be true that most people don't set the memory profile in the BIOS, none of those people read CPU reviews. Essentially every person who would read this reviews will be setting memory to the XMP/EXPO settings.

    So you're essentially invalidating your test results for the only people who see them.
    Reply

Log in

Don't have an account? Sign up now