Test Bed and Setup

As per our processor testing policy, we take a premium category motherboard suitable for the socket, and equip the system with a suitable amount of memory running at the manufacturer's maximum supported frequency. This is also typically run at JEDEC subtimings where possible.

It is noted that some users are not keen on this policy, stating that sometimes the maximum supported frequency is quite low, or faster memory is available at a similar price, or that the JEDEC speeds can be prohibitive for performance. While these comments make sense, ultimately very few users apply memory profiles (either XMP or other) as they require interaction with the BIOS, and most users will fall back on JEDEC supported speeds - this includes home users as well as industry who might want to shave off a cent or two from the cost or stay within the margins set by the manufacturer. Where possible, we will extend out testing to include faster memory modules either at the same time as the review or a later date.

Test Setup
AMD 3000*1 R9 3900X
R7 3700X
MSI MEG X570
Ace
7C35v12

7C35v11*2
Wraith Prism G.Skill TridentZ
4x8 GB
DDR4-3200
CL16
16-16-16-36
AMD 2000 R7 2700X
R5 2600X
R5 2500X
ASRock X370
Gaming K4
P4.80 Wraith Max* G.Skill SniperX
2x8 GB
DDR4-2933
AMD 1000 R7 1800X ASRock X370
Gaming K4
P4.80 Wraith Max* G.Skill SniperX
2x8 GB
DDR4-2666
AMD TR4 TR 1920X ASUS ROG
X399 Zenith
0078 Enermax
Liqtech TR4
G.Skill FlareX
4x8GB
DDR4-2666
Intel 9th Gen i9-9900K
i7-9700K
i5-9600K
ASRock Z370
Gaming i7**
P1.70 TRUE
Copper
Crucial Ballistix
4x8GB
DDR4-2666
Intel 8th Gen i7-8086K
i7-8700K
i5-8600K
ASRock Z370
Gaming i7
P1.70 TRUE
Copper
Crucial Ballistix
4x8GB
DDR4-2666
Intel 7th Gen i7-7700K
i5-7600K
GIGABYTE X170
ECC Extreme
F21e Silverstone
AR10-115XS
G.Skill RipjawsV
2x16GB
DDR4-2400
Intel 6th Gen i7-6700K
i5-6600K
GIGABYTE X170
ECC Extreme
F21e Silverstone
AR10-115XS
G.Skill RipjawsV
2x16GB
DDR4-2133
Intel HEDT i9-7900X
i7-7820X
i7-7800X
ASRock X299
OC Formula
P1.40 TRUE
Copper
Crucial Ballistix
4x8GB
DDR4-2666
GPU Sapphire RX 460 2GB (CPU Tests)
MSI GTX 1080 Gaming 8G (Gaming Tests)
PSU Corsair AX860i
Corsair AX1200i
SSD Crucial MX200 1TB

**Crucial MX300 1TB
OS Windows 10 x64 RS3 1709
Spectre and Meltdown Patched


**Windows 10 x64 1903
Spectre and Meltdown Patched
*1 Ryzen 3000 series has been tested in a different environment.

*2 Initial Review BIOS - Graphs results are marked with **
 

We must thank the following companies for kindly providing hardware for our multiple test beds. Some of this hardware is not in this test bed specifically, but is used in other testing.

Hardware Providers
Sapphire RX 460 Nitro MSI GTX 1080 Gaming X OC Crucial MX200 +
MX500 SSDs
Corsair AX860i +
AX1200i PSUs
G.Skill RipjawsV,
SniperX, FlareX
Crucial Ballistix
DDR4
Silverstone
Coolers
Silverstone
Fans

Security Mitigrations

The systems have applied the latest Spectre and Meltdown mitigation patches where applicable. Meanwhile we should note that while the ZombieLoad exploit was announced earlier this year as well, the patches for that have not been released yet. We'll be looking at those later on once they hit.

Article Testing Methodology Update (July 8th):

We ran our original review numbers with the latest available firmware for the MSI MEG X570 ACE motherboard last week (Version  7C35v11). On Saturday the 6th MSI had shared with us a notice about a new version coming out, which became available to download to us on Sunday the 7th, the launch day and date of publication of the review.

We’ve had more time to investigate the new firmware, and have discovered extremely large changes in the behaviour of the frequency boosting algorithm. The new firmware (Version 7C35v12) for the motherboard contains AMD’s new ComboPI1.0.0.3.a (AGESA) firmware.

We discovered the following direct measurable effects between the two firmware versions:

(Note: This is a custom test that uses a fine-grained looping timed fixed instruction chain to derive frequency; it showcases single-core frequency)

We notice a significant change in the CPU’s boosting behaviour, now boosting to higher frequencies, and particularly at a faster rate from idle, more correctly matching AMD’s described intended boost behaviour and latency.

We’re currently in the process of re-running all our suite numbers and updating the article where necessary to reflect the new frequency behaviour.

Article Testing Methodology Update (July 9th):

We've updated the article benchmark numbers on the Ryzen 9 3900X. We've seen 3-9% improvements in exclusive ST workloads. MT workloads have remained unchanged, Gaming had both benefits and negatives. We continue to work on getting updated 3700X numbers and filling out the missing pieces.

Original BIOS results are as of first publication are marked with ** in the graphs.

Article Testing Methodology Update (July 10th):

We've also updated our Ryzen 7 3700X results now. Ultimately our conclusions haven't changed, but AMD does narrow the gap a bit more. For a full summary of our findings, please check out this article.

Benchmarking Setup: Windows 1903 SPEC2006 & 2017: Industry Standard - ST Performance & IPC
Comments Locked

447 Comments

View All Comments

  • Irata - Monday, July 8, 2019 - link

    Thanks for your reply Ryan. I did not intend to be rude when saying "lazy" but rather show that I do not think this is something that was done by intent.

    Like I said - mention these things and it helps clear up misunderstandings.

    It is definitely very positive that you test the Ryzen CPU with the latest builds though.
    I also like that you mention if prices include an HSF or not, but it would have been nice to mention the price of HSF used for Intel systems (when not boxed), as e.g. the Thermalright True Copper is a rather expensive CPU cooler.

    I think you already addressed not using a faster nVME drive (a PCIe 4 version would have been ideal if available - this would also have given an indication of potentially increased system power use for the Ryzen with PCIe 4 drives) on Twitter.

    Those are little nitpicks, so not intended to be a criticism of the overall article. It is just that people tend to be rather sensitive when it comes to Intel vs. AMD CPU comparisons, given Intel's history of things they are willing to do to keep mind- and marketshare.
  • Daeros - Monday, July 15, 2019 - link

    Whether or not it is intentional, AT has had an increasing Intel bias over the last several years. Watch to see how long it takes for an AMD article to get pushed down by rumors or vaporware from Intel or Nvidia.
  • rarson - Monday, July 8, 2019 - link

    I think Ryan brings up several salient points, and whether or not you think that they did or did not have the time to do what you wanted (they were also a man down without Dr. Cuttress), the fact of the matter is that AMD dropped a bunch of CPUs and GPUs all at once and literally everyone was scrambling to do what they could in order to cover these launches.

    I don't think it's coincidence that even in the tech Youtube space, if you watch 10 different reviews you'll largely see 10 different testing methodologies and 10 (somewhat) different results. Every single reviewer I've talked to said that this was one of, if not the most, difficult launch windows they've ever dealt with. Additionally, launching on a weekend with all of its associated complications (not just on reviewers' ends, but partners as well) is a bitch, with everyone scrambling at the last minute on their days off getting in last-minute updates and whatnot.

    When AMD tells you at the last minute, "Oh, the brand new Windows 10 update includes something new" you don't necessarily have time to go back and redo all the benchmarks you had already done on the Intel platform.

    TL;DR while there may have been flaws in some of the testing, take the details with a grain of salt and compare them to the myriad of other reviews out there for a better overall picture if necessary.
  • Irata - Tuesday, July 9, 2019 - link

    You are making a good point and unfortunately this was an - unfortunately - typical AMD CPU launch with things still being beta. I would assume testers are none too happy about having to re-do their tests.

    What I don't get from AMD (even if (and that's a capital IF) it's not their fault, it's their responsibility) is how they cannot see how this makes their products appear in a less favorable light. Let's say the buggy bios cost them 5%, the conclusion with a 5% better performance would have been even more in Ryzen 3000's favor.

    It's a bit like showing up to a job interview wearing the clothes you wore for the previous day's physical activity.
  • Daeros - Monday, July 15, 2019 - link

    Lazy isn't in it. Intentionally misleading is more like it. On one page, where AMD wins more than it looses in the charts, out of 21 paragraphs, 2 had something positive to say about AMD or Ryzen 3k without following up with something along the lines of "but we know Intel's got new tech coming, too"
  • Ryan Smith - Monday, July 8, 2019 - link

    To be sure, they're still valid. The patches for Fallout and ZombieLoad are not out yet (I only mention them because the vulnerabilities have already been announced).
  • RSAUser - Monday, July 8, 2019 - link

    They've been out since 14 May, what are you talking about?
  • djayjp - Monday, July 8, 2019 - link

    Don't forget RIDL
  • Meteor2 - Sunday, July 14, 2019 - link

    RIDL and Zombieload are the same thing.

    Yes, the Intel CPUs should have been re-benchmarked on 1903, updated after 14 May when the OS-side fixes for the new MDS-class flaws were released. That's only fair and it's quite reasonable to expect that users will apply security updates, not leave their systems unpatched and vulnerable for perhaps a percent or two of performance.
  • FireSnake - Monday, July 8, 2019 - link

    Ryan: how is this not explained in the article? I am reading this site for more then a decade and I trust you most. and I trust you will provide such information. I would expect, you update the article with this info.

Log in

Don't have an account? Sign up now