AMD Threadripper Pro Review: An Upgrade Over Regular Threadripper?
by Dr. Ian Cutress on July 14, 2021 9:00 AM EST- Posted in
- CPUs
- AMD
- ThreadRipper
- Threadripper Pro
- 3995WX
CPU Tests: Microbenchmarks
Core-to-Core Latency
As the core count of modern CPUs is growing, we are reaching a time when the time to access each core from a different core is no longer a constant. Even before the advent of heterogeneous SoC designs, processors built on large rings or meshes can have different latencies to access the nearest core compared to the furthest core. This rings true especially in multi-socket server environments.
But modern CPUs, even desktop and consumer CPUs, can have variable access latency to get to another core. For example, in the first generation Threadripper CPUs, we had four chips on the package, each with 8 threads, and each with a different core-to-core latency depending on if it was on-die or off-die. This gets more complex with products like Lakefield, which has two different communication buses depending on which core is talking to which.
If you are a regular reader of AnandTech’s CPU reviews, you will recognize our Core-to-Core latency test. It’s a great way to show exactly how groups of cores are laid out on the silicon. This is a custom in-house test built by Andrei, and we know there are competing tests out there, but we feel ours is the most accurate to how quick an access between two cores can happen.
On all our Threadripper Pro CPUs, we saw:
- a thread-to-thread latency of 7ns,
- a core-to-core in the same CCX latency as 17-18 nanoseconds,
- a core-to-core in a different CCX scale from 80 ns with no IO die hops to 113 with 3 IO die hops
Here we can distinuguish how long it takes for threads to ping back and forth with cores that are different hops across the IO die.
A y-Cruncher Sprint
The y-cruncher website has a large about of benchmark data showing how different CPUs perform to calculate specific values of pi. Below these there are a few CPUs where it shows the time to compute moving from 25 million digits to 50 million, 100 million, 250 million, and all the way up to 10 billion, to showcase how the performance scales with digits (assuming everything is in memory). This range of results, from 25 million to 250 billion, is something I’ve dubbed a ‘sprint’.
I have written some code in order to perform a sprint on every CPU we test. It detects the DRAM, works out the biggest value that can be calculated with that amount of memory, and works up from 25 million digits. For the tests that go up to the ~25 billion digits, it only adds an extra 15 minutes to the suite for an 8-core Ryzen CPU.
With this test, we can see the effect of increasing memory requirements on the workload and the scaling factor for a workload such as this. We're plotting milllions of digits calculated per second.
The 64C/64T processor obtains the peak efficiency here, although as more digits are calculated, the memory requirements come into play.
Facebook
Twitter
RSS
98 Comments
View All Comments
Spunjji - Friday, July 16, 2021 - link
Having seen how modern processors behave with insufficient cooling, Threska's right that it won't get "fried", but you're correct to infer that it would result in unpredictably sub-optimal performance.Anecdotally, I had a friend with a Sandy Bridge system with a cooling issue that he only noticed when he bought a new GPU and ran 3DMark and got unexpectedly low results. The "cooling issue" was that the stock heatsink wasn't even making contact with the CPU heat-spreader; he'd been gaming with the system for 3 years by that point. 😬
serpretetsky - Friday, July 16, 2021 - link
I had to do some thermal shutdown testing on some consumer intel cpu. I forgot which one. Maybe i5/i7 8000 series?With server CPUs this was usually pretty easy, remove fan, and wait for shutdown. With the consumer CPU it kept running. So i completely removed the heatsink, the thing simply downclocked to 800 MHz, and continued running happily with no heatsink. Booted to linux, ran everything great, and no heatsink (actually once it booted to linux I think it even started clocking back up once in a while). I had get a hot-air soldering gun to heat it up till shutdown.
mode_13h - Saturday, July 17, 2021 - link
5-10 years ago, there was a heatsink gasket where you have to get near 100 degrees C to melt the material so it fuses with the heatsink and CPU. I forget the name, but I'm wondering if it's even possible to do that any more.skaurus - Wednesday, July 14, 2021 - link
That's great analysis.Threska - Wednesday, July 14, 2021 - link
It would be nice to see how these MBs do with VFIO since that has considerations most users don't.mode_13h - Wednesday, July 14, 2021 - link
Ian, is the source code for your 3DPM benchmark published anywhere? If not, it would be nice if we could see it and compare the AVX2 path with the AVX-512 one. Also, maybe someone could add support for ARM NEON or SVE.techguymaxc - Wednesday, July 14, 2021 - link
I'm slightly confused by the concluding remarks."Performance between Threadripper Pro and Threadripper came in three stages. Either (a) the results between similar processors was practically identical, (b) Threadripper beat TR Pro by a small margin due to slightly higher frequencies, or (c) TR Pro thrashed Threadripper due to memory bandwidth availability. That last point, (c), only really kicks in for the 32c and 64c processors it should be noted. Our 16c TR Pro had the same memory bandwidth results as TR, most likely due to only having two chiplets in its design."
A and B are observable, but C only proves true in synthetic benchmarks (and Pi calculation). Is there a real-world use-case for the additional memory bandwidth, outside of calculating Pi?
Blastdoor - Wednesday, July 14, 2021 - link
The advantage shows up with multi-threaded SPEC. SPEC is essentially a composite of a suite of real-world tasks. I guess you could call it 'synthetic' due to it being a composite, but the individual tasks don't strike me as 'synthetic.' For example, here's a description of namd: https://www.spec.org/cpu2017/Docs/benchmarks/508.n...techguymaxc - Wednesday, July 14, 2021 - link
Thanks for that info. It would be nice to see the breakdown of individual test results from the SPEC suite.arashi - Saturday, July 17, 2021 - link
Bench