Thread Director: Windows 11 Does It Best

Every operating system runs what is called a scheduler – a low-level program that dictates where workloads should be on the processor depending on factors like performance, thermals, and priority. A naïve scheduler that only has to deal with a single core or a homogenous design has it pretty easy, managing only power and thermals. Since those single core days though, schedulers have grown more complex.

One of the first issues that schedulers faced in monolithic silicon designs was multi-threading, whereby a core could run more than one thread simultaneously. We usually consider that running two threads on a core usually improves performance, but it is not a linear relationship. One thread on a core might be running at 100%, but two threads on a single core, while overall throughput might increase to 140%, it might mean that each thread is only running at 70%. As a result, schedulers had to distinguish between threads and hyperthreads, prioritizing new software to execute on a new core before filling up the hyperthreads. If there is software that doesn’t need all the performance and is happy to be background-related, then if the scheduler knows enough about the workload, it might put it on a hyperthread. This is, at a simple level, what Windows 10 does today.

This way of doing things maximizes performance, but could have a negative effect on efficiency, as ‘waking up’ a core to run a workload on it may incur extra static power costs. Going beyond that, this simple view assumes each core and thread has the same performance and efficiency profile. When we move to a hybrid system, that is no longer the case.

Alder Lake has two sets of cores (P-cores and E-cores), but it actually has three levels of performance and efficiency: P-cores, E-Cores, and hyperthreads on P-cores. In order to ensure that the cores are used to their maximum, Intel had to work with Microsoft to implement a new hybrid-aware scheduler, and this one interacts with an on-board microcontroller on the CPU for more information about what is actually going on.

The microcontroller on the CPU is what we call Intel Thread Director. It has a full scope view of the whole processor – what is running where, what instructions are running, and what appears to be the most important. It monitors the instructions at the nanosecond level, and communicates with the OS on the microsecond level. It takes into account thermals, power settings, and identifies which threads can be promoted to higher performance modes, or those that can be bumped if something higher priority comes along. It can also adjust recommendations based on frequency, power, thermals, and additional sensory data not immediately available to the scheduler at that resolution. All of that gets fed to the operating system.

The scheduler is Microsoft’s part of the arrangement, and as it lives in software, it’s the one that ultimately makes the decisions. The scheduler takes all of the information from Thread Director, constantly, as a guide. So if a user comes in with a more important workload, Thread Director tells the scheduler which cores are free, or which threads to demote. The scheduler can override the Thread Director, especially if the user has a specific request, such as making background tasks a higher priority.

What makes Windows 11 better than Windows 10 in this regard is that Windows 10 focuses more on the power of certain cores, whereas Windows 11 expands that to efficiency as well. While Windows 10 considers the E-cores as lower performance than P-cores, it doesn’t know how well each core does at a given frequency with a workload, whereas Windows 11 does. Combine that with an instruction prioritization model, and Intel states that under Windows 11, users should expect a lot better consistency in performance when it comes to hybrid CPU designs.

Under the hood, Thread Director is running a pre-trained algorithm based on millions of hours of data gathered during the development of the feature. It identifies the effective IPC of a given workflow, and applies that to the performance/efficiency metrics of each core variation. If there’s an obvious potential for better IPC or better efficiency, then it suggests the thread is moved. Workloads are broadly split into four classes:

  • Class 3: Bottleneck is not in the compute, e.g. IO or busy loops that don’t scale
  • Class 0: Most Applications
  • Class 1: Workloads using AVX/AVX2 instructions
  • Class 2: Workloads using AVX-VNNI instructions

Anything in Class 3 is recommended for E-cores. Anything in Class 1 or 2 is recommended for P cores, with Class 2 having higher priority. Everything else fits in Class 0, with frequency adjustments to optimize for IPC and efficiency if placed on the P-cores. The OS may force any class of workload onto any core, depending on the user.

There was some confusion in the press briefing as to whether Thread Director can ‘learn’ during operation, and how long it would take – to be clear, Thread Director doesn’t learn, it already knows from the pre-trained algorithm. It analyzes the instruction flow coming into a core, identifies the class as listed above, calculates where it is best placed (which takes microseconds), and communicates that to the OS. I think the confusion came with the difference in the words ‘learning’ and ‘analyzing’. In this case, it’s ‘learning’ what the instruction mix to apply to the algorithm, but the algorithm itself isn’t updated in the way that it is ‘learning’ and adjusting the classes. Ultimately even if you wanted to make the algorithm self-learn your workflow, the algorithm can’t actually see which thread relates to which program or utility – that’s something on the operating system level, and down to Microsoft. Ultimately, Thread Director could suggest a series of things, and the operating system can choose to ignore them all. That’s unlikely to happen in normal operation though.

One of the situations where this might rear its head is to do with in-focus operation. As showcased by Intel, the default behavior of Windows changes depending on whether on the power plan.

When a user is on the balanced power plan, Microsoft will move any software or window that is in focus (i.e. selected) onto the P-cores. Conversely, if you click away from one window to another, the thread for that first window will move to an E-core, and the new window now gets P-core priority. This makes perfect sense for the user that has a million windows and tabs open, and doesn’t want them taking immediate performance away.

However, this way of doing things might be a bit of a concern, or at least it is for me. The demonstration that Intel performed was where a user was exporting video content in one application, and then moved to another to do image processing. When the user moved to the image processing application, the video editing threads were moved to the E-cores, allowing the image editor to use the P-cores as needed.

Now usually when I’m dealing with video exports, it’s the video throughput that is my limiting factor. I need the video to complete, regardless of what I’m doing in the interim. By defocusing the video export window, it now moves to the slower E-cores. If I want to keep it on the P-cores in this mode, I have to keep the window in focus and not do anything else. The way that this is described also means that if you use any software that’s fronted by a GUI, but spawns a background process to do the actual work, unless the background process gets focus (which it can never do in normal operation), then it will stay on the E-cores.

In my mind, this is a bad oversight. I was told that this is explicitly Microsoft’s choice on how to do things.

The solution, in my mind, is for some sort of software to exist where a user can highlight programs to the OS that they want to keep on the high-performance track. Intel technically made something similar when it first introduced Turbo Max 3.0, however it was unclear if this was something that had to come from Intel or from Microsoft to work properly. I assume the latter, given the OS has ultimate control here.

I was however told that if the user changes the Windows Power Plan to high-performance, this behavior stops. In my mind this isn’t a proper fix, but it means that we might see some users/reviews of the hardware with lower performance if the workload doing the work is background, and the reviewer is using the default Balanced Power Plan as installed. If the same policy is going to apply to Laptops, that’s a bigger issue.

Cache and Hybrid Designs DDR5: Detailed Support, XMP, Memory Boost
POST A COMMENT

395 Comments

View All Comments

  • shabby - Wednesday, October 27, 2021 - link

    Typo in the 12600 charts... 20 threads or 16? Reply
  • nandnandnand - Wednesday, October 27, 2021 - link

    16 threads.
    2p + e = 2*6 + 4

    Thread count typos are going to happen a lot with Alder Lake, Raptor Lake...
    Reply
  • at_clucks - Friday, October 29, 2021 - link

    20-30 years ago it was really easy to know everything important about a CPU (frequency more or less, maybe multiplier/FSB) just by looking at the model name. Intel made it really confusing for the past few generations even for the relatively mundane setup with multi-identical-cores at a certain frequency. Now with the different types of cores, different base and turbo frequencies, different thread count, etc. you'll have to search for the table every time.

    And we're in the easy part now, just high end, all models ending in "00". Can't wait for the "12672KSF" which has random tweaks here and there depending on what they could get out of the wafer.
    Reply
  • yeeeeman - Friday, October 29, 2021 - link

    Neah, if you went to school at least you can count. Reply
  • Hifihedgehog - Wednesday, October 27, 2021 - link

    AMDead… until Zen 4. Zen 3+ is dead to me, because Zen 3+ is a band-aid solution to hobble along until they can get Zen 4 is out the door. Packing on a cache block is not going to beat this. Only a fresh and fast microarchitectural update can regain performance for AMD. And that update is Zen 4. Reply
  • Hifihedgehog - Wednesday, October 27, 2021 - link

    Also, timing is crucial and hitting the moving target. Zen 4 may beat Alder Lake but can it beat Raptor Lake which will likely come out late next year? If not, AMDead will be dead even with Zen 4, even if Zen 4 is a huge improvement over Zen 3. AMD has to compete against Intel’s best at time of release, not against itself. Reply
  • schujj07 - Wednesday, October 27, 2021 - link

    Yes because we have independent benchmarks out already from real world usage and not Geekbench. Don't forget to look at the power usage on the new Intel CPUs. They very well could be faster than AMD, however, if they need double the power to be 15% faster that really isn't a win. Reply
  • shabby - Wednesday, October 27, 2021 - link

    Don't think intel cares about losing the efficiency crown on the desktop, they just want to be at the top of the charts at all costs. Now in the server department with dozens of cores then it starts to bite them in the ass. Reply
  • Hifihedgehog - Wednesday, October 27, 2021 - link

    Most desktop users could care less about power efficiency. If this were a laptop or notebook question, that would be a totally different story. That is an interesting point you bring up, though! It's likely why Intel released desktop BEFORE mobile. Historically, they release first to mobile and then desktop. Power efficiency may be very well why they opted to avoid pushing it to mobile first. Or not at all. They may just be trying to counter AMD where it hurts most, in their current Ryzen stronghold. Reply
  • schujj07 - Wednesday, October 27, 2021 - link

    Power efficiency does make a slight difference on the desktop. If you need use a huge AIO or air cooler compared to a mid range air cooler that hurts your pocket book. That can also make your system louder than the comparable other system. Not to mention it will affect your summertime electric bill if it keeps heating up the room so your AC is kicking on. Reply

Log in

Don't have an account? Sign up now