Test Bed and Setup

As per our processor testing policy, we take a premium category motherboard suitable for the socket, and equip the system with a suitable amount of memory running at the manufacturer's maximum supported frequency. This is also typically run at JEDEC subtimings where possible.

It is noted that some users are not keen on this policy, stating that sometimes the maximum supported frequency is quite low, or faster memory is available at a similar price, or that the JEDEC speeds can be prohibitive for performance. While these comments make sense, ultimately very few users apply memory profiles (either XMP or other) as they require interaction with the BIOS, and most users will fall back on JEDEC supported speeds - this includes home users as well as industry who might want to shave off a cent or two from the cost or stay within the margins set by the manufacturer. Where possible, we will extend out testing to include faster memory modules either at the same time as the review or a later date.

Test Setup
AMD 3000*1 R9 3900X
R7 3700X
MSI MEG X570
Ace
7C35v12

7C35v11*2
Wraith Prism G.Skill TridentZ
4x8 GB
DDR4-3200
CL16
16-16-16-36
AMD 2000 R7 2700X
R5 2600X
R5 2500X
ASRock X370
Gaming K4
P4.80 Wraith Max* G.Skill SniperX
2x8 GB
DDR4-2933
AMD 1000 R7 1800X ASRock X370
Gaming K4
P4.80 Wraith Max* G.Skill SniperX
2x8 GB
DDR4-2666
AMD TR4 TR 1920X ASUS ROG
X399 Zenith
0078 Enermax
Liqtech TR4
G.Skill FlareX
4x8GB
DDR4-2666
Intel 9th Gen i9-9900K
i7-9700K
i5-9600K
ASRock Z370
Gaming i7**
P1.70 TRUE
Copper
Crucial Ballistix
4x8GB
DDR4-2666
Intel 8th Gen i7-8086K
i7-8700K
i5-8600K
ASRock Z370
Gaming i7
P1.70 TRUE
Copper
Crucial Ballistix
4x8GB
DDR4-2666
Intel 7th Gen i7-7700K
i5-7600K
GIGABYTE X170
ECC Extreme
F21e Silverstone
AR10-115XS
G.Skill RipjawsV
2x16GB
DDR4-2400
Intel 6th Gen i7-6700K
i5-6600K
GIGABYTE X170
ECC Extreme
F21e Silverstone
AR10-115XS
G.Skill RipjawsV
2x16GB
DDR4-2133
Intel HEDT i9-7900X
i7-7820X
i7-7800X
ASRock X299
OC Formula
P1.40 TRUE
Copper
Crucial Ballistix
4x8GB
DDR4-2666
GPU Sapphire RX 460 2GB (CPU Tests)
MSI GTX 1080 Gaming 8G (Gaming Tests)
PSU Corsair AX860i
Corsair AX1200i
SSD Crucial MX200 1TB

**Crucial MX300 1TB
OS Windows 10 x64 RS3 1709
Spectre and Meltdown Patched


**Windows 10 x64 1903
Spectre and Meltdown Patched
*1 Ryzen 3000 series has been tested in a different environment.

*2 Initial Review BIOS - Graphs results are marked with **
 

We must thank the following companies for kindly providing hardware for our multiple test beds. Some of this hardware is not in this test bed specifically, but is used in other testing.

Hardware Providers
Sapphire RX 460 Nitro MSI GTX 1080 Gaming X OC Crucial MX200 +
MX500 SSDs
Corsair AX860i +
AX1200i PSUs
G.Skill RipjawsV,
SniperX, FlareX
Crucial Ballistix
DDR4
Silverstone
Coolers
Silverstone
Fans

Security Mitigrations

The systems have applied the latest Spectre and Meltdown mitigation patches where applicable. Meanwhile we should note that while the ZombieLoad exploit was announced earlier this year as well, the patches for that have not been released yet. We'll be looking at those later on once they hit.

Article Testing Methodology Update (July 8th):

We ran our original review numbers with the latest available firmware for the MSI MEG X570 ACE motherboard last week (Version  7C35v11). On Saturday the 6th MSI had shared with us a notice about a new version coming out, which became available to download to us on Sunday the 7th, the launch day and date of publication of the review.

We’ve had more time to investigate the new firmware, and have discovered extremely large changes in the behaviour of the frequency boosting algorithm. The new firmware (Version 7C35v12) for the motherboard contains AMD’s new ComboPI1.0.0.3.a (AGESA) firmware.

We discovered the following direct measurable effects between the two firmware versions:

(Note: This is a custom test that uses a fine-grained looping timed fixed instruction chain to derive frequency; it showcases single-core frequency)

We notice a significant change in the CPU’s boosting behaviour, now boosting to higher frequencies, and particularly at a faster rate from idle, more correctly matching AMD’s described intended boost behaviour and latency.

We’re currently in the process of re-running all our suite numbers and updating the article where necessary to reflect the new frequency behaviour.

Article Testing Methodology Update (July 9th):

We've updated the article benchmark numbers on the Ryzen 9 3900X. We've seen 3-9% improvements in exclusive ST workloads. MT workloads have remained unchanged, Gaming had both benefits and negatives. We continue to work on getting updated 3700X numbers and filling out the missing pieces.

Original BIOS results are as of first publication are marked with ** in the graphs.

Article Testing Methodology Update (July 10th):

We've also updated our Ryzen 7 3700X results now. Ultimately our conclusions haven't changed, but AMD does narrow the gap a bit more. For a full summary of our findings, please check out this article.

Benchmarking Setup: Windows 1903 SPEC2006 & 2017: Industry Standard - ST Performance & IPC
Comments Locked

447 Comments

View All Comments

  • djayjp - Sunday, July 7, 2019 - link

    So results for Intel chips are completely invalid then.
  • futrtrubl - Sunday, July 7, 2019 - link

    You will need to have to explain that then. Comparing Intel with mitigations vs AMD with mitigations.
  • djayjp - Sunday, July 7, 2019 - link

    No. Fallout/ZombieLoad does not affect AMD chips.
  • djayjp - Sunday, July 7, 2019 - link

    Intel performance will suffer whereas AMD's won't be affected.
  • WaltC - Sunday, July 7, 2019 - link

    Ha-ha...;) So because AMD has a newer architecture without most of the vulnerabilities that plague Intel's ancient CPU architectures--it should be held against AMD? Rubbish...;) Look, what is unfair about testing both architectures/cpus with all the mitigations that each *requires*? I can't see a thing wrong with it--it's perfect, in fact.
  • extide - Sunday, July 7, 2019 - link

    They tested Intel WITHOUT Fallout/ZombieLoad which would affect them. Probably not by much, though, honestly.
  • RSAUser - Monday, July 8, 2019 - link

    Well the results are close enough for a lot of tests to be error margin, that the mitigation would put AMD in the lead.

    The tests should reflect real world as of when the article is published, using old results without declaring that Intel doesn't have mitigation applied on every page is the equivalent of falsifying the results as people will buy based on these tests.
  • mkaibear - Monday, July 8, 2019 - link

    "using old results without declaring that Intel doesn't have mitigation applied on every page is the equivalent of falsifying the results as people will buy based on these tests."

    Oh, that's just inane. They quite openly state the exact test specification on the "Test Bed and Setup" page, including which mitigations are applied. Arguing that not putting one particular piece of information on every page means it's the equivalent of falsifying the results is completely ridiculous.
  • RSAUser - Tuesday, July 9, 2019 - link

    How many go through the test bed set up page?
  • Meteor2 - Sunday, July 14, 2019 - link

    Pretty much everyone reading such an in-depth review, I should think.

Log in

Don't have an account? Sign up now