A Timely Discovery: Examining Our AMD 2nd Gen Ryzen Results
by Ian Cutress & Ryan Smith on April 25, 2018 11:15 AM EST
Last week, we published our AMD 2nd Gen Ryzen Deep Dive, covering our testing and analysis of the latest generation of processors to come out from AMD. Highlights of the new products included better cache latencies, faster memory support, an increase in IPC, an overall performance gain over the first generation products, new power management methods for turbo frequencies, and very competitive pricing.
In our review, we had a change in some of the testing. The big differences in our testing for this review was two-fold: the jump from Windows 10 Pro RS2 to Windows 10 Pro RS3, and the inclusion of the Spectre and Meltdown patches to mitigate the potential security issues. These patches are still being rolled out by motherboard manufacturers, with the latest platforms being first in that queue. For our review, we tested the new processors with the latest OS updates and microcode updates, as well as re-testing the Intel Coffee Lake processors as well. Due to time restrictions, the older Ryzen 1000-series results were used.
Due to the tight deadline of our testing and results, we pushed both our CPU and gaming tests live without as much formal analysis as we typically like to do. All the parts were competitive, however it quickly became clear that some of our results were not aligned with those from other media. Initially we were under the impression that this was as a result of the Spectre and Meltdown (or Smeltdown) updates, as we were one of the few media outlets to go back and perform retesting under the new standard.
Nonetheless, we decided to take an extensive internal audit of our testing to ensure that our results were accurate and completely reproducible. Or, failing that, understanding why our results differed. No stone was left un-turned: hardware, software, firmware, tweaks, and code. As a result of that process we believe we have found the reason for our testing being so different from the results of others, and interestingly it opened a sizable can of worms we were not expecting.
An extract from our Power testing script
What our testing identified is that the source of the issue is actually down to timers. Windows uses timers for many things, such as synchronization or ensuring linearity, and there are sets of software relating to monitoring and overclocking that require the timer with the most granularity - specifically they often require the High Precision Event Timer (HPET). HPET is very important, especially when it comes to determining if 'one second' of PC time is the equivalent to 'one second' of real-world time - the way that Windows 8 and Windows 10 implements their timing strategy, compared to Windows 7, means that in rare circumstances the system time can be liable to clock shift over time. This is often highly dependent on how the motherboard manufacturer implements certain settings. HPET is a motherboard-level timer that, as the name implies, offers a very high level of timer precision beyond what other PC timers can provide, and can mitigate this issue. This timer has been shipping in PCs for over a decade, and under normal circumstances it should not be anything but a boon to Windows.
However, it sadly appears that reality diverges from theory – sometimes extensively so – and that our CPU benchmarks for the Ryzen 2000-series review were caught in the middle. Instead of being a benefit to testing, what our investigation found is that when HPET is forced as the sole system timer, it can sometimes a hindrance to system performance, particularly gaming performance. Worse, because HPET is implemented differently on different platforms, the actual impact of enabling it isn't even consistent across vendors. Meaning that the effects of using HPET can vary from system to system, as well as the implementation.
And that brings us to the state HPET, our Ryzen 2000-series review, and CPU benchmarking in general. As we'll cover in the next few pages, HPET plays a very necessary and often very beneficial role in system timer accuracy; a role important enough that it's not desirable to completely disable HPET – and indeed in many systems this isn't even possible – all the while certain classes of software such as overclocking & monitoring software may even require it. However for a few different reasons it can also be a drain on system performance, and as a result HPET shouldn't always be used. So let's dive into the subject of hardware timers, precision, Smeltdown, and how it all came together to make a perfect storm of volatility for our Ryzen 2000-series review.
Facebook
Twitter
RSS
242 Comments
View All Comments
bbertram - Wednesday, April 25, 2018 - link
Well this is interesting! This could have serious implications.Googled HPET really quick and found this: https://www.reddit.com/r/Planetside/comments/416ns...
and then I found this link from that thread....a little ironic.
https://forums.anandtech.com/threads/do-you-have-h...
bbertram - Wednesday, April 25, 2018 - link
An interesting article that talks more about the issue. They look to even have a benchmark to show the impact. The video is also very interesting. The more I research this problem the more i see its been know for a very long time now.https://tinyurl.com/yd8qsh7w
bbertram - Wednesday, April 25, 2018 - link
ohhh...more nice info: https://tinyurl.com/yd39zw8c_mat - Wednesday, April 25, 2018 - link
Very thorough article. I like to point out a few things though, that may add some information to this.AMD and especially Intel have swept this problem under the rug since the launch of Skylake X. I noticed this problem while benching for a review and initially thought that my OS installation was the cause. After some testing I finally found the same root of evil as Ian did. At that time I made a video and called it the "Intel X299 HPET" bug (can't post a link, it was already mentioned in the comments here).
I tried to talk to PR and engineers at Intel for quite a while and they heard about my bug report but refused to comment. Time went by and Threadripper and Coffee Lake were born, both inheriting the same slow HPET QPC timer calls. I informed Intel repeatedly, still no comment.
During that time I wrote the following benchmark that sheds some light on the whole QPC and timer business on Windows. It shows your Windows timer configuration, gives recommendations for precision and performance, provides a way to bench your QPC timer in a synthetic and a game test and gives easy access to change TSC to HPET and vice versa.
As I am not able to post a link here, please search for "TimerBench", you will be able to download it.
I am also the author of GPUPI, one of those benchmarks for overclockers mentioned in the article that enforced HPET timers for secure timing a while back. Since discovering the HPET bug I have pulled back on this restriction. Since Skylake HPET is no longer necessary to avoid BCLK skewing, iTSC is just fine. AMD is still affected though, possibly Ryzen 2 as well (Threadripper and Ryzen 1 was).
bbertram - Wednesday, April 25, 2018 - link
Link to download: https://tinyurl.com/y7w6tg36Link to article: https://tinyurl.com/yd8qsh7w
Arbie - Thursday, April 26, 2018 - link
Wow! Google translator is amazing when going from German to English!mapesdhs - Sunday, May 6, 2018 - link
Might be because English has its roots in Germanic languages. :D Old English sounds a lot like common words in Dutch, and there's a region in Germany where the way German is spoken can sound to other Germans to be rather like English (according to a German guy I know). It's all those pesky Saxons, Angles, etc. :DTrackSmart - Wednesday, April 25, 2018 - link
Thank you _mat! Hopefully your comment gets attention here at Anandtech, and in turn, this article and your work get some attention from Intel. On the AMD side, it sounds like enabling HPET has only a small penalty in most cases, but those differences on the Intel side are very troubling. At the very least we should be forewarned!Dark_wizzie - Wednesday, April 25, 2018 - link
What software causes HPET to be forced on in Windows? I have multiple software installed but it still appears off.Dec666 - Wednesday, April 25, 2018 - link
Hi, AT.First of all, I wanted to thank you for an extreme effort you put in your reviews and analysis.
My thinking on the subject is that if you disable HPET in OS, this may make your numbers and review conclusion be irrelevant to the real world scenarios. As you have said, many programs (like video streaming, monitoring/overclocking, and potentially motherboard software (not to say about Ryzen Master)) require HPET to be enabled in OS and they will force it during the installation process and most likely won’t inform you about this. That means, that if you’ve installed all the software you going to use on fresh OS (and/or fresh PC), it is very possible that some of that software will have HPET forced and you won’t know about it.
To my mind, most of people, who read CPU reviews, are enthusiasts and/or those, who want to make a decision on CPU purchase by themselves. The majority of people will just buy PC based on others’ opinion or consultant’s advise. So those, for whom 10% difference in performance matters, and/or those, who bought expensive GPU like 1080/1080ti, will probably use monitoring software like HWinfo or Afterburner. That means, that HPET will be forced on their systems. That means, that they will have real world numbers close to what you’ve got in the original Ryzen 2000 review.
Another thing is that by disabling HPET in OS, while doing tests for a new review, you will hide the problem with it on Intel systems. People will not consider this as a potential performance hit or disadvantage of Intel platform in general.
Moreover, I suspect that in future more programs and, probably, next-gen games will require HPET (in order to better synchronize even more threads). Since most of people buy CPU for more than one year, they will have potentially worse experience with Intel CPUs in future, compared to AMD CPUs.
So it looks more logical to me to test CPUs with HPET forced (for all software), but have additional tests with HPET disabled for just games in order to have games tested with HPET both on and off. That will emphasize the problem. For me this is the same reason why it is important to test hardware with all Smeltdown patches and BIOS updates installed.
Thanks.