Core-to-Core Latency

As the core count of modern CPUs is growing, we are reaching a time when the time to access each core from a different core is no longer a constant. Even before the advent of heterogeneous SoC designs, processors built on large rings or meshes can have different latencies to access the nearest core compared to the furthest core. This rings true especially in multi-socket server environments.

But modern CPUs, even desktop and consumer CPUs, can have variable access latency to get to another core. For example, in the first generation Threadripper CPUs, we had four chips on the package, each with 8 threads, and each with a different core-to-core latency depending on if it was on-die or off-die. This gets more complex with products like Lakefield, which has two different communication buses depending on which core is talking to which.

If you are a regular reader of AnandTech’s CPU reviews, you will recognize our Core-to-Core latency test. It’s a great way to show exactly how groups of cores are laid out on the silicon. This is a custom in-house test, and we know there are competing tests out there, but we feel ours is the most accurate to how quick an access between two cores can happen.


Click to enlarge (lots of cores and threads = lots of core pairings)

Comparing core to core latencies from Zen 4 (7950X) and Zen 3 (5950X), both are using a two CCX 8-core chiplet design, which is a marked improvement over the four CCX 16-core design featured on the Zen 2 microarchitecture, the Ryzen 9 3950X. The inter-core latencies within the L3 cache range from between 15 ns and 19 ns. The inter-core latencies between different cores within different parts of the CCD show a larger latency penalty of up to 79.5 ns, which is something AMD should work on going forward, but it's an overall improvement in cross CCX latencies compared to Zen 3. Any gain is still a gain.

Even though AMD has opted for a newer and more 'efficient' IOD which is based on TSMC's 6 nm node. It is around the same size physically as the previous AMD IOD on Zen 3 manufactured on GlobalFoundries 12 nm node, but with a much larger transistor count. Within the IOD is the newly integrated RDNA 2 graphics, although this isn't typical iGPU in the sense that an APU is. A lot of the room on the IOD is made up of the DDR5 memory controller or IMC, as well as the chips PCIe 5.0 lanes, and of course, connects to the logic through its primary interconnect named Infinity Fabric. All of these variables play a part on power, latency, and operation.


AMD Ryzen 9 5950X Core-to-Core Latency results

It's actually astounding how similar the latency performance of the Ryzen 9 7950X (Zen 4) is when compared directly to the Ryzen 9 5950X (Zen 3), despite being on the new 5 nm TSMC manufacturing process. Even with a change of IOD, but with the same interconnect, the inter-core latencies within the Ryzen 9 7950X are great in terms of cores within the same core complex; latency does degrade when pairing up with a core in another chiplet, but this works and AMD's Ryzen 5000 series proved that the overall penalty performance is negatable.

Test Bed and Setup SPEC2017 Single-Threaded Results
POST A COMMENT

205 Comments

View All Comments

  • RestChem - Wednesday, October 5, 2022 - link

    Meh, time will out the ultimate price-points and all that, but as it emerges I really wonder what kind of users are looking to drop this kind of dollarses on high-end AMD builds. My gut is that they've priced themselves out of their primary demographic, and max TDP is right up there too, same as with their GPUs. When it comes down to a difference of a couple hundred bucks per build (assuming people build these with the pricey DDR5-6000 there's scant mobo support for through whatever AMD's integrated mem-OC profile scheme is) are there going to be enough users who just root hard enough for the underdog to build on these platforms, contra even high-end Alder Lake or (however much extra, reamins at time of writing to be seen) Raptor Lake builds? Before the announcements I was expecting AMD to get in cheap again, promise at least like performance for a bit of a discount, but it seems even those days are over and they want to play head-to-head. I wish them the best but I don't see them scoring well in that fight. Reply
  • tvdang7 - Thursday, October 6, 2022 - link

    " I have a 1440p 144Hz monitor and I play at 1080p just because that's what I'm used to."
    Is this some kind of joke. We are supposed to listen to reviewers that are stuck in 2010
    Reply
  • Hresna - Sunday, October 9, 2022 - link

    I’m curious as to whether there’s any appreciable difference to a consumer as to whether a particular PCIe lane or USB port is provisioned by the CPU or the Chipset…. Like, is there a reliability, performance, or some other metric difference?

    I’m just curious why it’s a design consideration to even include them in the CPU design to begin with, unless it has to do with how the CPU lanes are multiplexed in/out of the CPU and somehow some of the lanes can talk inter-device via the chipset without involving the cpu…
    Reply
  • bigtree - Monday, October 10, 2022 - link

    Where is octa channel memory? dual channel memory is a $300 CPU.
    Where is native Thunderbolt 4 support?
    (mac minis have had thunderbolt 3 for over 5 years).
    Cant even find one X670 Motherboard with 4x Thunderbolt 4 ports. And you want $300? Thunderbolt 4 should be standard on the cheapest boards. Its a $20 chip.
    Reply
  • Oxford Guy - Monday, October 10, 2022 - link

    The mission of corporations is to extract profit for shareholders and protect the lavish lifestyles of the rich. It is not to provide value to the plebs. Do the absolute minimum is the mantra. Reply
  • RedGreenBlue - Tuesday, October 11, 2022 - link

    That must be why Intel made Thunderbolt royalty-free and it’s now built into USB 4. Reply
  • Oxford Guy - Wednesday, October 12, 2022 - link

    It probably can afford to since states like Ohio are willing to bankroll half of the cost of its fabs. Reply
  • RedGreenBlue - Tuesday, October 11, 2022 - link

    It’s built into USB 4 now. Just make sure it’s functional already because it might need a driver, AMD did that on the 600 series. Aside from that important fact, I don’t care if there aren't many boards with it. The thunderbolt ecosystem has been crap since the beginning. Peripheral makers didn’t take advantage of it because USB was a more common approach and intel didn’t make thunderbolt cheap to implement. The Mac Minis have it because Apple made a big bet on it when it came out. These days it’s nice to have but it’s a throw-away feature unless you have a niche product that needs it. It’s for niche purposes and that would have been a waste of pci lanes. I would’ve liked it for external GPU’s but intel effectively shut that down and I don’t know if they’ve opened the door to it again. USB is way more convenient. Reply
  • RedGreenBlue - Tuesday, October 11, 2022 - link

    And 8 channel memory, like, this sounds like a joke. That’s for server or workstation cpus because of how many layers it takes for the wiring on the board and the pins on the socket. That’s part of why server and workstation boards are so expensive. If you need that much bandwidth you’re in the wrong market segment. Look at Threadripper chips. Reply
  • RedGreenBlue - Tuesday, October 11, 2022 - link

    It would be appreciated if architecture reviews had the pipeline differences in a chart to compare across generations. Anandtech used to have that included and it gave a good comparison of different generations and competitor architectures. I can understand not including it in the product review but I don’t remember a chart being in the previous Zen 4 overview article. Reply

Log in

Don't have an account? Sign up now