Over the last 5 years the mobile space has seen a dramatic change in terms of performance of smartphone and tablet SoCs. The industry has seen a move from single-core to dual-core to quad-core processors to today’s heterogeneous 6-10 core designs. This was a natural evolution similar to what the PC space has seen in the last decade, but only in a much more accelerated pace. While ILP (Instruction-level parallelism) has certainly also gone up with each new processor architecture, with designs such as ARM’s Cortex A15 or Apple’s Cyclone processor cores brining significant single-threaded performance boosts, it’s the increase of CPU cores that has brought the most simple way of increasing overall computing power.

This increasing of CPU cores brought up many discussions about just how much sense such designs make in real-world usages. I can still remember when the first quad-cores were introduced that users were arguing the benefit of 4 cores in mobile workloads and that these increases were just done for the sake of marketing. I can draw parallels between those discussions from a few years ago and today’s arguments about 6 to 10-core SoCs based on big.LITTLE.

While there have been some attempts to analyse the core-count debate, I was never really satisfied with the methodology and results of these pieces. The existing tools for monitoring CPUs just don’t cut it when it comes to accurately analysing the fine-grained events that dictate the management of multi-core and heterogeneous CPUs. To try to finally have a proper analysis of the situation, for this article, I’ve tried to approach this issue from the ground up in an orderly and correct manner, and not relying on any third-party tools.

Methodology Explained

I should start with a disclaimer that because the tools required for such an analysis rely heavily on the Linux kernel, that this analysis is constrained to the behaviour of Android devices and doesn't necessarily represent the behaviour of devices on other operating systems, in particular Apple's iOS. As such, any comparisons between such SoCs should be limited to purely to theoretical scenarios where a given CPU configuration would be running Android.

The Basics: Frequency

Traditionally when wanting to log what the CPU is doing, most users would think of looking at the frequency which it is currently running at. Usually this gives a rough idea to see if there is some load on the CPU and when it kicks into high gear. The issue with this is the way one captures the frequency: the readout sample will always be a single discrete value at a given point in time. To be able to accurately get a good representation of the frequency one would need to have a sample rate of at least twice as fast as the CPU’s DVFS mechanism. Mobile SoCs now can switch frequency at intervals of down to 10-20ms, and even have unpredictable finer-grained switches which can be caused by QoS (Quality of Service) requests.

Sampling at anything under half the DVFS switching speeds can lead to inaccurate data. For example this can happen in periodic short high bursts. Take a given sample rate of 1s: Imagine that we read frequency out at 0.1s and 1.1s in time. Frequency at both these readouts would be either at a high or low frequency. What happens in-between though is not captured, and due to the switching speed being so high, we can miss out on 90%+ of the true frequency behaviour of the CPU.

Instead of going the route of logging the discrete frequency at a very high rate, we can do something far more accurate: Log the cumulative residency time for each frequency on each readout. Since Android devices run on the Linux kernel, we have easy access to this statistic provided by the CPUFreq framework. The time-in-state statistics are always accurate because they are incremented by the kernel driver asynchronously at each frequency change. So by calculating the deltas between each readout, we end up with an accurate frequency distribution within the period between our readouts.

What we end up is a stacked time distribution graph such as this:

The Y-axis of the graph is a stacked percentage of each CPU’s frequency state. The X-axis represents the distribution in time, always depending on the scenario’s length. For readability’s sake in this article, I chose an effective ~200ms sample period (Due to overhead on scripting and time-keeping mechanisms, this is just a rough target) which should give enough resolution for a good graphical representation of the CPU’s frequency behaviour.

With this, we now have the first part of our tools to accurately analyse the SoC’s behaviour: frequency.

The Details: Power States

While frequency is one of the first metrics that comes to mind when trying to monitor a CPU’s behaviour, there’s a whole other hidden layer that rarely gets exposure: CPU idle states. For readers looking for a more in-depth explanation of how CPUIdle works, I’ve touched upon it and power management of modern SoCs in general work in our deep dive of the Exynos 7420. These explanations are valid for basically all of today's SoCs based on ARM CPU IP, so it applies to SoCs from MediaTek and ARM-based Qualcomm chipsets as well.

To keep things short, a simplified explanation is that beyond frequency, modern CPUs are able to save power by entering idle states that either turn off the clock or the power to the individual CPU cores. At this point we’re talking about switching times of ~500µs to +5ms. It is rare to find SoC vendors expose APIs for live readout of the power states of the CPUs, so this is a statistic one couldn’t even realistically log via discrete readouts. Luckily CPU idle states are still arbitrated by the kernel, which again, similarly to the CPUFreq framework, provides us aggregate time-in-state statistics for each power state on each CPU.

This is an important distinction to make in today’s ARM CPU cores as (except for Qualcomm’s Krait architecture) all CPUs within a cluster run on the same synchronous frequency plane. So while one CPU can be reported to be running at a high frequency, this doesn’t really tell us what it’s doing and could as well be fully power-gated while sitting idle.

Using the same method as for frequency logging, we end up with an idle power-state stacked time-distribution graph for all cores within a cluster. I’ve labelled the states as “Clock-gated”, “Power-gated” and “Active” which in technical terms they represent the WFI (Wait-For-Interrupt) C1, power-collapse C2 idle states, as well as the difference in time to the wall-clock which represents the “active” time in which the CPU isn’t in any power-saving state.

The Intricacies: Scheduler Run-Queue Depths

One metric I don’t think that was ever discussed in the context of mobile is the depth of the CPU’s run-queue. In the Linux kernel scheduler the run-queue is a list of processes (The actual implementation involves a red-black tree) currently residing on that CPU. This is at the core of the preemptive scheduling nature of the CFS (Completely Fair Scheduler) process scheduler in the Linux kernel. When multiple processes run on the same CPU the scheduler is in charge to fairly distribute processing time between each thread based on time-slices and process priority.

The kernel and Android are able to sort of expose information on the run-queue through one of the kernel’s sysfs nodes. On Android this can be enabled through the “Show CPU Usage” option in the developer options. This gives you three numerical parameters as well as a list of the read-out active processes. The numerical value is the so-called “load average” of the scheduler. It represents the load of the whole system – and it can be used to read how many threads in a system are used. The three values represent averages for different time-windows: 1 minute, 5 minutes and 15 minutes. The actual value is a percentage – so for example 2.85 represents 285%. How this is meant to be interpreted is that if we were to consolidate all processes in as little CPUs as possible we theoretically have two CPUs whose load is 100% (summing up to 200%) as well as a third up to 85% load.

Now this is very odd, how can the phone be fully using almost 3 cores while I was doing nothing more than idling on the screen with the CPU statistics on? Sadly the kernel scheduler suffers from the same sampling rate issue as explained in our frequency logging methodology. Truth is that the load average statistic is only a snapshot of the scheduler’s run-queues which is updated only in 5-second intervals and the represented value is a calculated load based on the time between snapshots. Unfortunately this statistic is extremely misleading and in no way represents the actual situation of the run-queues. On Qualcomm devices this statistic is even more misleading as it can show load-averages of up to 12 in idle situations. Ultimately, this means it’s basically impossible to get accurate RQ-depth statistics on stock devices.

Luckily, I stumbled upon the same issue a few years ago and was aware of a patch that I previously used in the past and which was authored by Nvidia which introduces detailed rq-depth statistics. This tracks the run-queues accurately and atomically each time a process enters or leaves a run-queue, enabling it to expose a sliding-window average of the run-queue depth of each CPU over the period of 134ms.

Now we have a live pollable average for the scheduler’s run-queues and we can fully log the exact amount of threads run on the system.

Again, the X-axis throughout the graphs represent the time in milliseconds. This time the Y-axis represents the rq-depth of each CPU. I also included the sum of the rq-depths of all CPUs in a cluster as well the sum of both clusters for the system total in a separate graph.

The values can be interpreted similarly to the load-average metrics, only this time we have a separate value for each CPU. A run-queue depth of 1 means the CPU is loaded 100% of the time, 0.2 means the CPU is loaded by only 20%. Now the interesting metric comes for values above 1: For anything above a rq-depth of 1 it means that the CPU is preempting between multiple processes which cumulatively exceed the processing power of that CPU. For example in the above graph we have some per-CPU peaks of ~2. It means the CPU has at least two threads on that CPU and they each share 50% of the compute-time of that CPU, i.e. they’re running at half speed.

The Data And The Goals

On the following pages we’ll have a look at about 20 different real-world often encountered use-cases where we monitor CPU frequency, power states and scheduler run-queues. What we are looking for specifically is the run-queue depth spikes for each scenario to see just how many threads are spawned during the various scenarios.

The tests are run on Samsung's Galaxy S6 with the Exynos 7420 (4x Cortex A57 @ 2.1GHz + 4x Cortex A53 @ 1.5GHz) which should serve well as a representation of similar flagship devices sold in 2015 and beyond.

Depending on the use-cases, we'll see just how many of the cores on today's many-core big.LITTLE systems are used. Together with having power management data on both clusters, we'll also see just how much sense heterogeneous processing makes and just how much benefit one can gain from it.

Browser: S-Browser - AnandTech Article
POST A COMMENT

155 Comments

View All Comments

  • R0H1T - Tuesday, September 01, 2015 - link

    Seems like Android has Windows' number as far as "multi-threading" is concerned, kudos to Google for this & seems like the tired old argument of developers getting a free pass (for poor MT implementation on desktops) needs to change asap! Reply
  • Impulses - Tuesday, September 01, 2015 - link

    Ehh, I think you're ignoring some key differences in clock speed and single threaded performance, not to mention how easily Intel can ramp clock speed up and back down, and then there's Hyper Threading which allows you to span more threads per core.

    Laptops might be the outlier, but I dunno what benefit a desktop (which have commonly run quads for years) would see from a lower powered core cluster. Development just works very differently by nature of the environment.

    Also things that benefit a ton from parallelization on the desktop often end up using the GPU instead... And/or specialized instructions that aren't available at all on mobile. It's not even apples and oranges IMO, it's apples and watermelons.
    Reply
  • R0H1T - Tuesday, September 01, 2015 - link

    You're missing the point, which is that Google & Android have shown (even with the vast number of SoC's it runs on) that MT & load management, when implemented properly, on the supported hardware & complementing software, makes great use of x number of cores even in a highly constrained environment like a smartphone.

    On desktops we ought to have had affordable octa cores available for the masses by now, but since Intel has no real competition & they price their products through the roof, we're seeing what or how windows & the x86 platform has stagnated. Granted that more people are moving to small, portable computing devices but there's no reason why the OS & the platform as a whole has to slow down, also the clock speed, IPC argument is getting old now. If anything DX12, Mantle, Vulkan et al have shown us is that if there's good hardware & the willingness to push it to its limits developers, with the right tools at hand, will make use of it. Not to mention giving them a free pass for badly coded programs, remember the "ST performance is king" argument, is the wrong way to go as it not only wastes the (great) potential of desktops but it also slows down the progress of PC as a platform.

    Now I know MT isn't a cakewalk especially on modern systems but if anything it should be more widespread because desktops & notebooks give a lot of thermal headroom, as compared to tablets & smartphones, besides the 30+ years of history behind this particular industry should make the task easier. Also not all compute tasks can be offloaded to GPU, that's why it's even more imperative that the users push developers to make use of more cores & not get the free ride that GPGPU has been giving them over the last few years, as it is the GPU industry is also slowing down massively & then we'll eventually be back to square one & zero growth.
    Reply
  • metafor - Tuesday, September 01, 2015 - link

    Yes and no. Google and Android are able to show that things like app updates, web page loads and general system upkeep is able to take advantage of multiple threads. But that's been true for a while. In a smartphone, those happen to be the performance dominating tasks. On a desktop, those tasks are noise.

    Desktop workloads that actually stress the CPU (and users care about performing well) are very different. That's not to say they're not threadable, but they may not be as threadable as Chrome, which basically eats RAM and processes.

    That being said, heterogenous MT could make a lot of sense for laptop processors as well. Having threadable workloads run on smaller Atoms instead of big Sky Lakes would probably improve efficiency. But it may not be as dramatic depending on the perf/W of Sky Lake at lower frequencies.
    Reply
  • niva - Tuesday, September 01, 2015 - link

    OK can we talk about this for a bit. I for one found the webpage CPU usage extremely disturbing. I'm running an old phone, Galaxy Nexus, and browsing has become by far the task my phone struggles with the most. Why is that? What is it about modern websites that causes them to be so CPU heavy? Is that acceptable? It does seem that much of the internet is filled with websites running shady scripts in the background and automatically playing video or sound which is annoying at the very least, but detrimental to performance always. Whatever happened to website optimization for minimizing data usage and actually making websites accessible?

    Secondly, what is the actual throughput of CPUs in desktops compared to the latest state of the line arm APUs? Just because desktop workloads might be different, does that mean that a mobile APU cannot handle it or is that simply due to the usage mode of the device in question? What I'm seeing out of mobile/phone chips is that they are extremely capable, to the point I'm starting to wonder if I'll ever need another desktop rig to replace my old Phenom X2 machine.
    Reply
  • metafor - Tuesday, September 01, 2015 - link

    I would guess that websites are just more complicated nowadays. Think about a dynamic website like Twitter, which has to have live menus and notifications/updates. That's basically a program more than just a web page. We've slowly migrated what used to be stand-alone programs to load-on-demand web programs. And added many many inefficient layers of script interpreters in between. Reply
  • emn13 - Thursday, September 03, 2015 - link

    Somewhat ironically, the more modern a web-page the *less* friendly it is likely to be to multithreading. After all, modern features tend to include heavy javascript usage (which is almost purely single-threaded), and a CPU usage that is bottlenecked by a path through javascript (typically layout not actually the JS itself, but that layout affects JS and hence needs fine-grained interaction). Reply
  • Jaybus - Tuesday, September 01, 2015 - link

    It is the more extensive use of client-side processing, in a nutshell, JavaScript and JSON. On older websites, they dynamic stuff was processed server-side and the client simply did page reloads. The modern sites require less bandwidth, but at the expense of increasing CPU usage.

    Also, modern sites are higher res and more image intensive, or in other words more GPU heavy as well. Some of the Nexus struggle can be attributed to GPU load.
    Reply
  • mkozakewich - Wednesday, September 02, 2015 - link

    Most of it has to do with using multiple JavaScript libraries. It's not strange to need to download over 50 different files on a website today. Anandtech.com took 123 requests over four seconds to load. Mostly fonts, ads, and Twitter stuff, but it adds up. Reply
  • name99 - Tuesday, September 01, 2015 - link

    You are totally misinterpreting these results.
    The mere existence of a large number of runnable threads does not mean that the cores are being usefully used. Knowing that there are frequently four runnable threads is NOT the same thing as knowing that four cores are useful, because it is quite possible that those threads are low priority, and that sliding them so as run consecutively rather than simultaneously would have no effect on perceived user performance.

    There is plenty of evidence to suggest that this interpretation is correct.
    Within the AnandTech data, the fact that these threads are usually on the LITTLE cores, and running those at low frequency, suggests they are not high priority threads.

    This paper from MS research confirms the hypothesis:
    http://research.microsoft.com:8082/en-us/um/people...

    Now there is a whole lot of tribalism going on in this thread. I'm not interested in that; I'm interested in the facts. What the MS paper states (confirmed, IMHO) by these AnandTech results, is that there is a reasonable (around 20%) throughput improvement in going from one to two threads, along with a small (around 10%) energy drop, and that going from two to three or four cores buys you only very slight further energy and performance boosts.
    In one sense this means there's no harm in having octacores around --- they don't seem to burning energy, and in principle they could deliver extra snappiness (though the lousiness of the scheduling in these AnandTech results suggests that's more a hope than a reality). But there's a world of difference between the claim "doesn't hurt energy, may occasionally be slightly useful" and the claim "pretty much always useful because apps are so deeply threaded these days".
    Reply

Log in

Don't have an account? Sign up now