Reaching for Turbo: Aligning Perception with AMD’s Frequency Metricsby Dr. Ian Cutress on September 17, 2019 10:00 AM EST
Detecting Turbo: Microseconds vs. Milliseconds
One of the biggest issues with obtaining frequency data is the actual process of monitoring. While there are some basic OS commands to obtain the frequency, it isn’t just as simple as reading a number.
How To Read A Frequency
As an outlay, we have to differentiate between the frequency of a processor vs. the frequency of a core. On modern PC processors, each core can act independently of each other in terms of frequency and voltage, and so each core can report different numbers. In order to read the value for each core, that core has to be targeted using an affinity mask that binds the reading to a particular core. If a simple ‘what’s the frequency’ request goes out to a processor without an affinity mask, it will return the value of the core to which that thread ends up being assigned. Typically this is the fastest core, but if there is already work being performed on a chip, that thread might end up on an idle core. If a request to find out ‘what is the current frequency of the processor’ is made, users could end up with a number of values: the frequency on a specific core, the frequency of the fastest core, or an average frequency of all the cores. To add more confusion to the matter, if the load on a core is taken into account, depending on the way the request is made, a core running at ‘50%’ load at peak frequency might end up returning a value of half frequency.
There are a multitude of programs that report frequency. Several of the most popular include:
- Intel XTU
- Intel Power Gadget
- Ryzen Master
Some of these use similar methods to access frequency values, others have more intricate methods, and then the reporting and logging of each frequency value can have different effects on the system being tested.
I asked one of the main developers of these monitoring tools how they detect the frequency of a core. They gave me a brief overview – it’s not as simple as it turns out.
- Know the BCLK (~100 MHz) precisely. Normally this is done my measuring the APIC clock, but on modern systems that use internal clock references (Win 10 1803+) this causes additional interrupt bandwidth, and so often this value is polled rarely and cached.
- Detect the CPU Core multiplier by reading a single Model Specific Register based on the CPU. This has to be done in kernel mode, so there is additional overhead switching from user mode to kernel mode and back.
- This has to be repeated for each core by using an affinity mask, using a standard Win32 API call of SetCurrentThreadAffinityMask. As this is an API call, there is again additional overhead.
So the frequency of a single core here is measured by the base clock / BCLK and multiplying it by the Core Multiplier as defined in the registers for that core, all through an affinity mask. Typically BCLK is the same across all cores, but even that has some drift and fluctuations over time, so it will depend on how frequently you request that data.
Another alternative method is to apply a simple load – a known array of consistent instructions and to measure the number of cycles / length of time it takes to compute that small array. This method might be considered more accurate by some, but it still requires the appropriate affinity mask to be put in place, and actually puts in additional load to the system, which could cause erroneous readings.
How Quick Can Turbo Occur
Modern processors typically Turbo anywhere from 4 GHz to 5 GHz, or four to five billion cycles a second. That means each cycle at 5 GHz is equal to 0.2 nanoseconds, or 0.2 x 10-9 seconds. These processors don’t stay at that frequency – they adjust the frequency up or down based on the load requests, which helps manage power and heat. How quickly a processor can respond to these requests for a higher frequency has become a battleground in recent years.
How a processor manages its frequency all comes down to how it interacts with the operating system. In a pre-Skylake world, a processor would have a number of pre-defined ACPI power states, relating to performance (P), Device (D), and processor (C), based on if the processor was on, in sleep, or needed high frequency. P-states relied on a voltage-frequency scaling, and the OS could control P0 to P1 to P2 and beyond, with P1 being the guaranteed base frequency and any higher P number being OS controlled. The OS could request P0, which enabled the processor to enter boost mode. All of this would go through a set of OS drivers relating to power and frequency control; this came to be known as SpeedStep for Intel, and Cool’n’Quiet for AMD.
As defined in the ACPI specifications, with the introduction of UEFI control came CPPC, or Collaborative Processor Performance Control. Requiring CPU and OS support, with Skylake we saw Intel and Microsoft introduced a new ‘Speed Shift’ feature that put the control of the frequency modes of the processor back in the hands of the processor – the CPU could directly respond to the instruction density coming into the core and modify the frequency directly without additional commands. The end result of CPPC, and Speed Shift for Intel, was a much faster frequency response mechanism.
With Speed Shift in Skylake, on Windows, Intel was promoting that before Speed Shift they were changing frequency anywhere up to 100 milliseconds (0.1 s) after the request was made. With Speed Shift, that had come down to the 35 millisecond mark, around a 50-66% improvement. With subsequent updates to the Skylake architecture and the driver stack, Intel states that this has improved further.
Users can detect to see if CPPC is enabled on their Intel system very easily. By going to the Event Viewer, selecting Window Logs -> System, and then going to a time stamp where the machine was last rebooted, we can see ACPI CPPC listed under the Kernel-Processor-Power source.
For my Core i7-8565U Whiskey Lake CPU, it shows that APCI CPPC is enabled, and that my CPU Core 5 is running at 2.0 GHz base with a 230% peak turbo, or 4.6 GHz, which relates to the single-core turbo frequency of my processor.
For AMD, with Zen 2, the company announced the use of CPPC2 in collaboration with Microsoft. This is CPPC but with a few extra additional tweaks to the driver stack for when an AMD processor is detected.
Here AMD is claiming that they can change frequency, when using the Windows 10 May 2019 update or newer, on the scale of 1-2 ms, compared to 30 ms with the standard CPPC interface. This comes down to how AMD has implemented its ‘CPPC2’ model, with a series of shim drivers in place to help speed the process along. If we go back to how we can detect that CPPC mode similar to Intel, we see a subtle difference:
Notice here it doesn’t say CPPC2, just CPPC. What does display is the 3600 MHz base frequency of our 3700X, and a maximum performance percentage of 145%, which would make the peak turbo of this processor somewhere near 5220 MHz. Clearly that isn’t the peak turbo of this CPU (which would be 4400 MHz), which means that AMD is using this artificially high value combined with its CPPC driver updates to help drive a faster frequency response time.
The Observer Effect
Depending on the software being used, and the way it calculates the current frequency of any given core/processor, we could end up artificially loading the system, because as explained above it is not as simple as just reading a number – extra calculations have to be made or API calls have to be driven. The more frequently the user tries to detect the frequency, the more artificial load is generated on the core, and at some point the system will detect this as requiring a different frequency, making the readings change.
This is called the observer effect. And it is quite easy to see it in action.
For any tool that allows the user to change the polling frequency, as the user changes that frequency from once per second to ten times per second, then 100 times per second, or 1000 times per second, even on a completely idle system, some spikes will be drawn – more if the results are being logged to memory or a data file.
Therein lies the crutch of frequency reporting. Ultimately we need the polling frequency to be quick enough to capture all the different frequency changes, but we don’t want it interfering with the measurement. Combined with CPPC, this can make detecting certain peak frequencies particularly annoying.
Let’s go back to our time scales for instructions and frequency changes. At 4 GHz, we can break down the following:
|Time Scales at 4 GHz|
|Simple Loop (1000 cycles)||
|CPPC Frequency Change (AMD)||
Note that a frequency change is the equivalent to losing around 800,000 cycles at 4 GHz, so the CPU has to gauge to what point the frequency change is worth it based on the instructions flowing into the core.
But what this does tell is one of the inherent flaws in frequency monitoring – if a CPU can change frequency as quickly as every 1-2 ms, but we can only poll at around 50-100 ms, then we can miss some turbo values. If a processor quickly fires up to a peak turbo, processes a few instructions, and then drops down almost immediately due to power/frequency requirements not being met for the incoming instruction stream, it won’t ever be seen by the frequency polling software. If the requirements are met of course, then we do see the turbo frequency – the value we end up seeing is the one that the system feels is more long-term suitable.
With an attempt at sub-1ms polling time, we can see this in effect. The blue line shows the Ryzen processor in a balanced power configuration, and at around 3.6 milliseconds the 3700X jumps up to 4350-4400 MHz, bouncing around between the two. But by 4.6 milliseconds, we have already jumped down to 4.3 GHz, then at 5.2 milliseconds we are at 4.2 GHz.
We were able to obtain this data using Windows Subsystem for Linux, using an add-dependency chain from which we derive the frequency based on the throughput. There is no observer effect here because it is the workload – not something that can be done when an external workload is used. It gives us a resolution of around 40 microseconds, and relies on the scheduler automatically assigning the thread to the best core.
But simply put, unless a user is polling this quick, the user will not see the momentary peaks in turbo frequency if they are on the boundary of supporting it. The downside of this is that polling this quick puts an artificial load on the system, and means any concurrent running benchmark will be inadequate.
(For users wondering what that orange line is, that would be the processor in ‘performance mode’, which gives a higher tolerance for turbo.)
It all leads to a question – if a core hits a turbo frequency but you are unable to detect it, does that count?
Ultimately, by opting for a more aggressive binning strategy so close to silicon limits, AMD has reached a point where, depending on the workload and the environment, a desktop CPU might only sustain a top Turbo bins momentarily. Like Turbo itself, this is not a bad thing, as it extracts more performance from their processors that would otherwise be left on the table by lower clockspeeds. But compared to Intel’s processors and what we’re used to, these highest bins require more platform management to ensure that the processor is indeed reaching its full potential.
Post Your CommentPlease log in or sign up to comment.
View All Comments
StrangerGuy - Wednesday, September 18, 2019 - linkPretty much the only people left OCing CPUs are epeen wavers with more money than sense.
"$300 mobo and 360mm AIO for that Intel 8 core <10% OC at >200W...Look at all that *free* performance! Amirite?"
Korguz - Wednesday, September 18, 2019 - link" Pretty much the only people left OCing CPUs are epeen wavers with more money than sense" that a pretty bold statement. i know quite a few people who overclock their cpus, because intel charged too much for the higher end ones, so they had to get a lower tier chip. with zen, thats not the case as much any more as they have switched over to amd, because by this time, they would of had to get a new mobo any way, because intels upgrade paths are only 2, maybe 3 years if they are lucky.
dont " need " and AIO, as there are some pretty good air coolers out there, and some, dont like the idea of water, or liquid in their comps :-)
Xyler94 - Thursday, September 19, 2019 - linkSorry, here's where I'll have to disagree with you.
You'll never overclock an i3 to i5/i7 levels. If my choices were between an overclocking i3, with a Z series board, or a locked i5 with an H series board, I'd choose the i5 in a heartbeat, as that's just generally better. Overclocking will never make up the lack of physical cores.
So I agree, mostly these days overclocking is reserved for A: People with e-peens, and B:, people who genuinely need 5ghz on a single core... which are fewer than those who can utilize the multi-threaded horsepower of Ryzen... so yeah,
evernessince - Tuesday, September 17, 2019 - linkDoes it deliver well? I see plenty of people on the Intel reddit not hitting advertised turbo speeds. That's considering they are using $50+ CPU coolers as well.
"Pretty impressive to see a server cpu with 20% lower ST performance only because the
low power process utilized is unable to deliver a clock speed near 4Ghz, absurd thing considering
that Intel 14nm LP gives 4GHz at 1V without struggles."
What CPU are you talking about? Even AMD's 64 core monster has damn near the same IPC as the Intel Xeon 8280 (thanks to Zen 2 IPC improvements) and that CPU has LESS THEN HALF THE CORES and only consumes 20w more. The Intel CPU also costs almost twice as much. Only a moron brings up single threaded performance in a server chip conversation anyways, it's one of the least important metric for server chips. AMD's new EPYC chip crushes Intel in Core count, TCO, power consumption, and security. Everything that is important to server.
yankeeDDL - Wednesday, September 18, 2019 - linkYou do realize that the clock speed does not depend only by the process, right? Your comment sounds like that that of a disgruntled Intel fanboy trying to put AMD in under a bar light. For 25MHz.
Spunjji - Monday, September 23, 2019 - linkAbsolute codswallop. AMD are getting 100-300Mhz more on their peak clock speeds for Zen2 with this first-gen 7nm process tech than they were seeing with Zen+ 12nm (and nearly 400Mhz more than Zen on 14nm), so nothing about that implies that 7nm is slower than 14nm. Intel's architecture and process tech are not remotely comparable to AMD's, and we don't know what is the primary limiting factor on Zen clockspeeds.
Not sure why you're claiming lower ST performance on the server parts either - Rome is better in every single regard than its predecessors, and it's better pound-for-pound than anything Intel will be able to offer in the next 12-18 months.
PeachNCream - Tuesday, September 17, 2019 - linkI see a tempest in a teapot on the stove of a person who is busy splitting hairs at the kitchen table. It would be more interesting to calculate how much energy and time was expended on the issue to see if the performance uplift from the fix will offset the global societal cost of all the clamoring this generated. For that, I suppose you'd have to know how many Ryzen chips are actually doing something productive as opposed to crunching at video games.
The idea of buying cheaper hardware for non-work needs sticks here. Less investment means less worry about maximizing your return on your super-happy-fun-box and less heartburn over a little bit of clockspeed on a component that plays second fiddle to GPU performance when it comes to entertainment anyway.
psychobriggsy - Tuesday, September 17, 2019 - linkUltimately in the end it isn't MHz that counts, it is observed performance in the software that you care about. That's why we read reviews, and why the review industry is so large.
If performance X was good enough, does it matter if it was achieved at 4.5GHz or 4.425GHz? Not really. But if the CPU manufacturer is using it as a primary competitive comparison metric (rather than a comparative metric with their other SKUs) then it has to be considered, like in this article.
It is sad that MHz is still a major metric in the industry, although now Intel IPC is similar to AMD IPC, it is actually kinda relevant.
What I'd like is better CPU power draw measurements versus what the manufacturer says. Because TDP advertising seems to be even more fraught with lies/marketing than MHz marketing! Obviously most users don't care about 10 or 20% extra power draw at a CPU level, as at a system level it will be a tiny change, but it's when it is 100% that it matters.
IMO, I'd like TDPs to be reported at all core sustained turbo, not base clocks. Sure, have a typical TDP measurement as well as the more information the better, but don't hide your 200W CPU under a 95W TDP.
TechnicallyLogic - Tuesday, September 17, 2019 - linkPersonally, I feel that AMD should have 2 numbers for the max frequency of the CPU; "Boost Clock" and "Burst Clock". Assuming that you have adequete cooling and power delivery, the boost clock would be sustainable indefinitely on a single core, while the burst clock would be the peak frequency that a single core on the CPU can reach, even if it's just for a few ms.
fatweeb - Tuesday, September 17, 2019 - linkI could see them eventually going in this direction considering Navi already has three clocks: Base, Gaming, and Boost. The first two would be guaranteed, the last not so much.