A Timely Discovery: Examining Our AMD 2nd Gen Ryzen Results
by Ian Cutress & Ryan Smith on April 25, 2018 11:15 AM EST
Last week, we published our AMD 2nd Gen Ryzen Deep Dive, covering our testing and analysis of the latest generation of processors to come out from AMD. Highlights of the new products included better cache latencies, faster memory support, an increase in IPC, an overall performance gain over the first generation products, new power management methods for turbo frequencies, and very competitive pricing.
In our review, we had a change in some of the testing. The big differences in our testing for this review was two-fold: the jump from Windows 10 Pro RS2 to Windows 10 Pro RS3, and the inclusion of the Spectre and Meltdown patches to mitigate the potential security issues. These patches are still being rolled out by motherboard manufacturers, with the latest platforms being first in that queue. For our review, we tested the new processors with the latest OS updates and microcode updates, as well as re-testing the Intel Coffee Lake processors as well. Due to time restrictions, the older Ryzen 1000-series results were used.
Due to the tight deadline of our testing and results, we pushed both our CPU and gaming tests live without as much formal analysis as we typically like to do. All the parts were competitive, however it quickly became clear that some of our results were not aligned with those from other media. Initially we were under the impression that this was as a result of the Spectre and Meltdown (or Smeltdown) updates, as we were one of the few media outlets to go back and perform retesting under the new standard.
Nonetheless, we decided to take an extensive internal audit of our testing to ensure that our results were accurate and completely reproducible. Or, failing that, understanding why our results differed. No stone was left un-turned: hardware, software, firmware, tweaks, and code. As a result of that process we believe we have found the reason for our testing being so different from the results of others, and interestingly it opened a sizable can of worms we were not expecting.
An extract from our Power testing script
What our testing identified is that the source of the issue is actually down to timers. Windows uses timers for many things, such as synchronization or ensuring linearity, and there are sets of software relating to monitoring and overclocking that require the timer with the most granularity - specifically they often require the High Precision Event Timer (HPET). HPET is very important, especially when it comes to determining if 'one second' of PC time is the equivalent to 'one second' of real-world time - the way that Windows 8 and Windows 10 implements their timing strategy, compared to Windows 7, means that in rare circumstances the system time can be liable to clock shift over time. This is often highly dependent on how the motherboard manufacturer implements certain settings. HPET is a motherboard-level timer that, as the name implies, offers a very high level of timer precision beyond what other PC timers can provide, and can mitigate this issue. This timer has been shipping in PCs for over a decade, and under normal circumstances it should not be anything but a boon to Windows.
However, it sadly appears that reality diverges from theory – sometimes extensively so – and that our CPU benchmarks for the Ryzen 2000-series review were caught in the middle. Instead of being a benefit to testing, what our investigation found is that when HPET is forced as the sole system timer, it can sometimes a hindrance to system performance, particularly gaming performance. Worse, because HPET is implemented differently on different platforms, the actual impact of enabling it isn't even consistent across vendors. Meaning that the effects of using HPET can vary from system to system, as well as the implementation.
And that brings us to the state HPET, our Ryzen 2000-series review, and CPU benchmarking in general. As we'll cover in the next few pages, HPET plays a very necessary and often very beneficial role in system timer accuracy; a role important enough that it's not desirable to completely disable HPET – and indeed in many systems this isn't even possible – all the while certain classes of software such as overclocking & monitoring software may even require it. However for a few different reasons it can also be a drain on system performance, and as a result HPET shouldn't always be used. So let's dive into the subject of hardware timers, precision, Smeltdown, and how it all came together to make a perfect storm of volatility for our Ryzen 2000-series review.
Facebook
Twitter
RSS
242 Comments
View All Comments
Spunjji - Thursday, April 26, 2018 - link
Is there any other kind? Either you're at the budget end where everything is GPU limited or at the high-end where not spending a decent amount on a monitor to go with your £500 GPU is a crying shame.There's a niche where Intel has a clear win, and that's people running 240Hz 1080p rigs. For most folks with the money to spend, 2560x1440 (or an ultra-wide equivalent) @ 144hz is where it's at for the ideal compromise between picture quality, smoothness and cost. There are a lot of monitors hitting those specs right now.
eva02langley - Wednesday, April 25, 2018 - link
I was mentioning in the review that 1080p benchmarks need to go... now it is even more true with HPET.Kudos on this guys, it is really interesting to read.
DanNeely - Wednesday, April 25, 2018 - link
>93% of Steam gamers main display is at 1080p or lower.If the new review suit split what GPUs were run at what resolutions, dropping 1080p from the high end card section might be reasonable. OTOH with 240hz 1080p screens a thing there's still an enthusiast market for 1080p combined with a flagship GPU.
IndianaKrom - Wednesday, April 25, 2018 - link
* Raises hand, that's me, someone with a GTX 1080 and 240Hz 1920x1080 display.The industry seems obsessed with throwing higher and higher spacial resolution at gamers when what I really want is better temporal resolution.
eva02langley - Thursday, April 26, 2018 - link
1080p @ 60Hz which is a non issue because we are talking about RX 580/1060 GTX or below. At that point the GPU is the bottleneck.It only affect 1080p @ 144Hz with a 1080 GTX/Vega 64 minimum which is really < 2%.
You are really the exception, however the 1080p CPU bottleneck focus on you entirely without even taking in consideration other Use Cases.
Holliday75 - Thursday, April 26, 2018 - link
I am willing to bet that 95%+ of Steam users have no clue what we are talking about and don't care.mapesdhs - Sunday, May 6, 2018 - link
IndianaKrom, are you aware that using high(er) frequency monitors retrains your brain's vision system so that you become tuned to that higher refresh rate? New Scientist had an article about this recently; gamers who use high frequency monitors can't use normal monitors anymore, even if previously they would not have found 60Hz bothersome at all. In other words, you're chasing goalposts that will simply keep moving by virtue of using ever higher refresh rates. I mean blimey, 240Hz is higher than the typical "analogue" vision refresh of a bird. :DIMO these high frequency monitors are bad for gaming in general, because they're changing product review conclusion via authors accepting that huge fps numbers are normal (even though the audience that would care is minimal). Meanwhile, game devs are not going to create significantly more complex worlds if it risks new titles showing more typical frame rates in the 30s to 80s as authors would then refer to that as slow, perhaps criticise the 3D engine, moan that gamers with HF monitors will be disappointed, and I doubt GPU vendors would like it either. We're creating a marketing catch22 with all this, doubly so as VR imposes some similar pressures.
I don't mind FPS fans wanting HF monitors in order to be on the cutting edge of competitiveness, but it shouldn't mean reviews become biased towards that particular market in the way they discuss the data (especially at 1080p), and it's bad if it's having a detrimental effect on new game development (I could be wrong about the latter btw, but I strongly suspect it's true from all I've read and heard).
We need a sanity check with frame rates in GPU reviews: if a game is doing more than 80 or 90fps at 1080p, then the conclusion emphasis should be that said GPU is more than enough for most users at that resolution; if it's well over 100fps then it's overkill. Just look at the way 8700K 1080p results are described in recent reviews, much is made of differences between various CPUs when the frame rates are already enormous. Competitive FPS gamers with HF monitors might care, but for the vast majority of gamers the differences are meaningless.
Luckz - Monday, May 14, 2018 - link
So the real question is if someone first exposed to 100/120/144 Hz immediately squirms in delight, or if they only vomit in disgust months later when they see a 60 Hz screen again. That should be the decider.Spunjji - Thursday, April 26, 2018 - link
1080p is popular in the Steam survey where, incidentally, so is low-end GPU and CPU hardware. Most of those displays are 60hz and an awful lot of them are in laptops. Pointing at the Steam surveys to indicate where high-end CPU reviews should focus their stats is misguided.I'm still not certain that testing CPUs in a way that artificially amplifies their differences in a non-CPU-reliant workload is really the way to go.
ElvenLemming - Wednesday, April 25, 2018 - link
You can just ignore the 1080p benchmarks if you don't think they're meaningful. As DanNeely said, 93% of surveyed Steam users are 1080p or lower, so I'd be shocked if more than a handful of review sites get rid of it.