The AMD Threadripper 2 CPU Review: The 24-Core 2970WX and 12-Core 2920X Tested
by Ian Cutress on October 29, 2018 9:00 AM ESTOur New Testing Suite for 2018 and 2019
Spectre and Meltdown Hardened
In order to keep up to date with our testing, we have to update our software every so often to stay relevant. In our updates we typically implement the latest operating system, the latest patches, the latest software revisions, the newest graphics drivers, as well as add new tests or remove old ones. As regular readers will know, our CPU testing revolves an automated test suite, and depending on how the newest software works, the suite either needs to change, be updated, have tests removed, or be rewritten completely. Last time we did a full re-write, it took the best part of a month, including regression testing (testing older processors).
One of the key elements of our testing update for 2018 (and 2019) is the fact that our scripts and systems are designed to be hardened for Spectre and Meltdown. This means making sure that all of our BIOSes are updated with the latest microcode, and all the steps are in place with our operating system with updates. In this case we are using Windows 10 x64 Enterprise 1709 with April security updates which enforces Smeltdown (our combined name) mitigations. Uses might ask why we are not running Windows 10 x64 RS4, the latest major update – this is due to some new features which are giving uneven results. Rather than spend a few weeks learning to disable them, we’re going ahead with RS3 which has been widely used.
Our previous benchmark suite was split into several segments depending on how the test is usually perceived. Our new test suite follows similar lines, and we run the tests based on:
- Power
- Memory
- Office
- System
- Render
- Encoding
- Web
- Legacy
- Integrated Gaming
- CPU Gaming
Depending on the focus of the review, the order of these benchmarks might change, or some left out of the main review. All of our data will reside in our benchmark database, Bench, for which there is a new ‘CPU 2019’ section for all of our new tests.
Within each section, we will have the following tests:
Power
Our power tests consist of running a substantial workload for every thread in the system, and then probing the power registers on the chip to find out details such as core power, package power, DRAM power, IO power, and per-core power. This all depends on how much information is given by the manufacturer of the chip: sometimes a lot, sometimes not at all.
We are currently running POV-Ray as our main test for Power, as it seems to hit deep into the system and is very consistent. In order to limit the number of cores for power, we use an affinity mask driven from the command line.
Memory
These tests involve disabling all turbo modes in the system, forcing it to run at base frequency, and them implementing both a memory latency checker (Intel’s Memory Latency Checker works equally well for both platforms) and AIDA64 to probe cache bandwidth.
Office
- Chromium Compile: Windows VC++ Compile of Chrome 56 (same as 2017)
- PCMark10: Primary data will be the overview results – subtest results will be in Bench
- 3DMark Physics: We test every physics sub-test for Bench, and report the major ones (new)
- GeekBench4: By request (new)
- SYSmark 2018: Recently released by BAPCo, currently automating it into our suite (new, when feasible)
System
- Application Load: Time to load GIMP 2.10.4 (new)
- FCAT: Time to process a 90 second ROTR 1440p recording (same as 2017)
- 3D Particle Movement: Particle distribution test (same as 2017) – we also have AVX2 and AVX512 versions of this, which may be added later
- Dolphin 5.0: Console emulation test (same as 2017)
- DigiCortex: Sea Slug Brain simulation (same as 2017)
- y-Cruncher v0.7.6: Pi calculation with optimized instruction sets for new CPUs (new)
- Agisoft Photoscan 1.3.3: 2D image to 3D modelling tool (updated)
Render
- Corona 1.3: Performance renderer for 3dsMax, Cinema4D (same as 2017)
- Blender 2.79b: Render of bmw27 on CPU (updated to 2.79b)
- LuxMark v3.1 C++ and OpenCL: Test of different rendering code paths (same as 2017)
- POV-Ray 3.7.1: Built-in benchmark (updated)
- CineBench R15: Older Cinema4D test, will likely remain in Bench (same as 2017)
Encoding
- 7-zip 1805: Built-in benchmark (updated to v1805)
- WinRAR 5.60b3: Compression test of directory with video and web files (updated to 5.60b3)
- AES Encryption: In-memory AES performance. Slightly older test. (same as 2017)
- Handbrake 1.1.0: Logitech C920 1080p60 input file, transcoded into three formats for streaming/storage:
- 720p60, x264, 6000 kbps CBR, Fast, High Profile
- 1080p60, x264, 3500 kbps CBR, Faster, Main Profile
- 1080p60, HEVC, 3500 kbps VBR, Fast, 2-Pass Main Profile
Web
- WebXPRT3: The latest WebXPRT test (updated)
- WebXPRT15: Similar to 3, but slightly older. (same as 2017)
- Speedometer2: Javascript Framework test (new)
- Google Octane 2.0: Depreciated but popular web test (same as 2017)
- Mozilla Kraken 1.1: Depreciated but popular web test (same as 2017)
Legacy (same as 2017)
- 3DPM v1: Older version of 3DPM, very naïve code
- x264 HD 3.0: Older transcode benchmark
- Cinebench R11.5 and R10: Representative of different coding methodologies
Linux (when feasible)
When in full swing, we wish to return to running LinuxBench 1.0. This was in our 2016 test, but was ditched in 2017 as it added an extra complication layer to our automation. By popular request, we are going to run it again.
Integrated and CPU Gaming
We have recently automated around a dozen games at four different performance levels. A good number of games will have frame time data, however due to automation complications, some will not. The idea is that we get a good overview of a number of different genres and engines for testing. So far we have the following games automated:
AnandTech CPU Gaming 2019 Game List | ||||||||
Game | Genre | Release Date | API | IGP | Low | Med | High | |
World of Tanks enCore | Driving / Action | Feb 2018 |
DX11 | 768p Minimum |
1080p Medium |
1080p Ultra |
4K Ultra |
|
Final Fantasy XV | JRPG | Mar 2018 |
DX11 | 720p Standard |
1080p Standard |
4K Standard |
8K Standard |
|
Shadow of War | Action / RPG | Sep 2017 |
DX11 | 720p Ultra |
1080p Ultra |
4K High |
8K High |
|
F1 2018 | Racing | Aug 2018 |
DX11 | 720p Low |
1080p Med |
4K High |
4K Ultra |
|
Civilization VI | RTS | Oct 2016 |
DX12 | 1080p Ultra |
4K Ultra |
8K Ultra |
16K Low |
|
Car Mechanic Simulator '18 | Simulation / Racing | July 2017 |
DX11 | 720p Low |
1080p Medium |
1440p High |
4K Ultra |
|
Ashes: Classic | RTS | Mar 2016 |
DX12 | 720p Standard |
1080p Standard |
1440p Standard |
4K Standard |
|
Strange Brigade* | FPS | Aug 2018 |
DX12 Vulkan |
720p Low |
1080p Medium |
1440p High |
4K Ultra |
|
Shadow of the Tomb Raider | Action | Sep 2018 |
DX12 | 720p Low |
1080p Medium |
1440p High |
4K Highest |
|
Grand Theft Auto V | Open World | Apr 2015 |
DX11 | 720p Low |
1080p High |
1440p Very High |
4K Ultra |
|
Far Cry 5 | FPS | Mar 2018 |
DX11 | 720p Low |
1080p Normal |
1440p High |
4K Ultra |
|
*Strange Brigade is run in DX12 and Vulkan modes |
For our CPU Gaming tests, we will be running on an NVIDIA GTX 1080. For the CPU benchmarks, we use an RX460 as we now have several units for concurrent testing.
In previous years we tested multiple GPUs on a small number of games – this time around, due to a Twitter poll I did which turned out exactly 50:50, we are doing it the other way around: more games, fewer GPUs.
Scale Up vs Scale Out: Benefits of Automation
One comment we get every now and again is that automation isn’t the best way of testing – there’s a higher barrier to entry, and it limits the tests that can be done. From our perspective, despite taking a little while to program properly (and get it right), automation means we can do several things:
- Guarantee consistent breaks between tests for cooldown to occur, rather than variable cooldown times based on ‘if I’m looking at the screen’
- It allows us to simultaneously test several systems at once. I currently run five systems in my office (limited by the number of 4K monitors, and space) which means we can process more hardware at the same time
- We can leave tests to run overnight, very useful for a deadline
- With a good enough script, tests can be added very easily
Our benchmark suite collates all the results and spits out data as the tests are running to a central storage platform, which I can probe mid-run to update data as it comes through. This also acts as a mental check in case any of the data might be abnormal.
We do have one major limitation, and that rests on the side of our gaming tests. We are running multiple tests through one Steam account, some of which (like GTA) are online only. As Steam only lets one system play on an account at once, our gaming script probes Steam’s own APIs to determine if we are ‘online’ or not, and to run offline tests until the account is free to be logged in on that system. Depending on the number of games we test that absolutely require online mode, it can be a bit of a bottleneck.
Benchmark Suite Updates
As always, we do take requests. It helps us understand the workloads that everyone is running and plan accordingly.
A side note on software packages: we have had requests for tests on software such as ANSYS, or other professional grade software. The downside of testing this software is licensing and scale. Most of these companies do not particularly care about us running tests, and state it’s not part of their goals. Others, like Agisoft, are more than willing to help. If you are involved in these software packages, the best way to see us benchmark them is to reach out. We have special versions of software for some of our tests, and if we can get something that works, and relevant to the audience, then we shouldn’t have too much difficulty adding it to the suite.
69 Comments
View All Comments
schujj07 - Monday, October 29, 2018 - link
You would be far to limited with RAM to run 60 VMs on that system. I've got 80 on dual Dell 7425's with dual 24 Core Epycs and 512GB RAM and I'm already getting RAM limited.Again I wouldn't install ESXi on these. Use Win 10 and Workstation for your test/dev and you will have a more agile system. If you don't need it for testing that day you still have Windows. FYI I'm VMware Admin.
Ratman6161 - Monday, October 29, 2018 - link
All depends...in my home lab environment (which lets me test things at will and do whatever I want as opposed to at work where even the lab is more locked down) . For me, the Threadrippers would be great...but extreme overkill. I actually use old FX8320's which I bought when they were dirt cheap and DDR3 RAM was cheap too. The free version of ESXi works fine for me too. For my purposes the threadrippers would be really cool but more expensive than they would be worth.Icehawk - Monday, October 29, 2018 - link
I would love one of these high cores boxes for our test lab, using W10 and VM on my desktop is very limiting for me (work rig is 7700 & 32gb) - one of these would let me put plenty of resources onboard. Currently my lab runs off a G6 Dell server which is totally fine but if I could get myself a new, personal, lab I'd want a TR rig since it can host a lot more RAM than Intel's option.odrade - Tuesday, October 30, 2018 - link
Hi I completely agree with you.With security enhancement moving to sandbox/VM (Application Guard, Sandboxed Defender in 19H1) virtualization scenario will be more prevalent beyond developper or test scenarios.
One major disappointment is that after 12+ months since GA there is no support for nested virtualization for TR/TR2 ?, Ryzen ? Epyc ?.
This issue seems to be general and not limited to hyper-v (KVM, etc..).
This is strange since EPYC made is way through Azure or Oracle Cloud catalog.
During Ignite 2018 there was a demo with an EPYC box (VM or Server).
Regards G.
GreenReaper - Wednesday, October 31, 2018 - link
You could ask for HyperV over here:https://windowsserver.uservoice.com/forums/295047-...
But such features are often buggy in their initial implementations:
http://www.os2museum.com/wp/vme-broken-on-amd-ryze...
https://www.reddit.com/r/Amd/comments/8ljgph/has_t...
It wouldn't surprise me if they ran into too many problems to want to push out a solution. And Intel has had issues here too - most recently L1 Terminal Fault relating to EPT:
https://www.redhat.com/en/blog/understanding-l1-te...
If people buy enough of them, and there is a performance benefit or it otherwise becomes a feature differentiator, support will doubtless be developed. Chicken and egg, I know.
odrade - Monday, November 5, 2018 - link
Hi,Thanks four your inputs.
This feature is handy if you want to build advanced lab scenarios while preserving your work environment or avoid the hassle to use dual boot.
Maybe this feature will be enabled with the 2019 Epyc / TR iteration.
And if the the socket and compatibility promises is kept by AMD refreshing
my setup will do it and put those extra pcie lanes to use (upgrading storage as well).
At least the 7mm process will help to kept the power compatibility in line.
Regards G.
Blindsay - Monday, October 29, 2018 - link
For the chart on the last page, the "12-core Battle" it would be interesting to see a "similar price battle" of like the 9900k vs 7820X vs 2920X. I suspect the 9900k would hold up rather well especially once it returns to its SRPmapesdhs - Monday, October 29, 2018 - link
A battle for what? If it's gaming, get the far cheaper 2700X and using the difference to buy a better GPU, giving better gaming results by default (some niche cases at 1080p, but in general the 9900K is a poor value option for gaming, except for those who've gone the NPC route into high refresh displays from which there's no way back, ironic now NVIDIA has decided to move backwards to sub-60Hz 1080p with RTX).Blindsay - Monday, October 29, 2018 - link
Definitely not for gaming lol. It is for a home server (unraid)PeachNCream - Tuesday, October 30, 2018 - link
That's a lot of compute for a home server. Home servers (outside of those used for the development of professional skills or to test software outside of a setting where there are office usage policies) serve very limited useful purposes. They're mainly a solution looking for a problem or just fun to mess around with. I have an old C2D E8400-powered desktop PC with 8GB of RAM that I just recently put online as a local file, media, and internal web server connected via a cheap TPLink PCI (non-e) wifi card. There's nothing that the kids and I have done to it yet that brings it anywhere close to its knees. Even streaming videos from it to three other systems at once is a non-issue and all of those files are stored on a single 1TB 5400 RPM 2.5 inch mechanical HDD. TR is extreme overkill for a toy server at home. Literally any old scavenged desktop or laptop can act as a home server.