System Performance

To see how the new CPUs and memory subsystem translate into more real system performance, we move onto more representative tests such as PCMark. PCMark’s performance is affected by several factors: not only does raw performance of the hardware count but also we need to consider the individual system’s software stack. We’ve seen large differences between Android OS major versions where the improvements of the Android Runtime can be directly visible in subtests such as the Writing test. Also a SoC’s DVFS schemes and schedulers can have huge impacts on “performance-latency”, meaning how fast the CPUs can ramp up a workload. This directly translates in a lot more performance in several of PCMark’s subtests as in the default settings none of the tests actually represent the pure performance of the CPU if it were locked at maximum frequency on the performance cores. The results of the tests are also overall a good representation of “snappiness” of a device.

PCMark Work 2.0 - Web Browsing 2.0

In the web browsing test the Snapdragon 845 QRD manages to outpace the Pixel 2 XL by 20%. Here we’re also looking at performance across devices with different OS versions. The Google devices are running Android 8.1 while the Samsung devices were tested with Android 7.0. The Mate 10 Pro runs Android 8.0 while the Mate 9 still had 7.0. The Qualcomm QRD we tested ran Android 8.0.

Again the performance increase over Snapdragon 835 devices isn’t all that great. DynamiQ allows for far more efficient thread transitions between the CPU cores and subsequently I expected Qualcomm to take advantage of this through more aggressive scheduling resulting in more than just a 20% increase. The difference between the Mate 9 and Mate 10 here is a good example of what a software configuration change can bring in terms of performance (both devices employ same performance CPU configurations). Samsung’s Exynos’ SoCs still use GTS scheduling and have non-optimal performance-latency resulting in bad scores, amplified by the fact that Samsung’s memory performance is also underwhelming when compared to the Snapdragon and Kirin SoCs.

PCMark Work 2.0 - Data ManipulationPCMark Work 2.0 - Writing 2.0

The Data Manipulation and Writing 2.0 tests make heavy use of the Android runtime and APIs and also a very memory latency sensitive. Between the best showings of the Snapdragon 835 variant of the S8 and the Pixel 2 XL in each respective benchmark, the Snapdragon QRD845 showed conservative increases of 8 to 14%. The Exynos SoCs lacklustre performance is again hampered by software and by bad memory performance.

PCMark Work 2.0 - Video Editing

The video editing test is PCMark’s weak-point as it’s bottlenecked by things such as OS API overhead, and why we see tight grouping of performance results across a large range of SoCs. The Snapdragon 845 ends up high, but below the Pixel 2 XL. I would not put much weight on the results of this test as they’re not necessarily representative. Futuremark claims that the test is a lot more sensitive in mid- and low-range devices which can exhibit performance issues.

PCMark Work 2.0 - Photo Editing 2.0

The photo editing test makes heavy use of Renderscript and use GPU acceleration to apply various effects on an image set. The QRD845 here shines as it’s able to showcase a 38% performance improvement over the Pixel 2 XL. Again the test not solely tests the raw performance of the system but also how optimized it is in terms of the software stack. This can be seen in the Kirin vs Exynos devices as Huawei’s phones vastly outperform Samsung’s devices in this test.

PCMark Work 2.0 - Performance

Overall PCMark’s performance score for the QRD845 increases by 17% over the Pixel 2 XL. Disregarding the video test, we see a similar scenario as in the synthetic tests as the new SoC’s CPU performance increases are lower than we had expected. Still the Snapdragon 845 is able to top the charts and should adequately power 2018’s flagship devices.

For 2018 we are reviewing our mobile benchmarking suite and altering some of the benchmarks we use. One of the changes in the way we benchmark devices is that we’re moving away from standalone browser and rather are benchmarking the OS’s WebView implementations. In general this seems to be a better choice for testing device experience as there is a lot of content that is being consumed via WebView windows. We also avoid the argument about different browser performance and since Google has now made WebView an updatable Play Store component we should also have valid comparisons older devices and systems. On the iOS side we do the same as we now benchmark browser tests within a WkWebView shell.

WebXPRT 2015 - OS WebView

Starting off with WebXPRT 2015 for a last time before we’ll retire it in favour of WebXPRT 3, we see the QRD845 performing fantastically. Here the 44% performance increase over the Pixel 2 XL is a lot more in line with what we had expected of the new SoC. The QRD845 is even able to catch up a lot with Apple’s newest A11 and Monsoon cores in this test.

To keep up with the ever changing landscape of the developing web, we’re also retiring past JavaScript benchmarks in favour of a brand new and more representative benchmark developed by the WebKit team and welcomed by Google; Speedometer 2.0.

Speedometer 2.0 - OS WebView

Here the Snapdragon 845 showcased another healthy performance increase of 37% over the Snapdragon 835 devices. Apple’s superior JavaScript performance can be attributed to a much faster and more optimized Nitro engine while Google’s V8 has only seen meagre improvements over the years. Notable is the Apple A11’s massive performance jump over the A10 – vastly increasing the distance to Android devices.

CPU & Memory Subsystem GPU Performance & Power Estimates


View All Comments

  • BenSkywalker - Monday, February 12, 2018 - link

    Legal studies I am lacking, you are correct, alas you can't keep out of the press how much trouble QC is in for their practices-

    China fined them for a billion already, Korea for $850 Million, Taiwan for another $750 Million- US suit is in progress.
  • Andrei Frumusanu - Monday, February 12, 2018 - link

    No I'm comparing the same methodology on the devices, active system power. The Shield TV on the X1 does 12W at 61fps. The QRD845 did 82fps at 4.4W.

    Nowhere in this article nor from Qualcomm is there the pure GPU power figure published, but always the system power. Your Nvidia figure is running at half performance meaning up to a 3x higher efficiency point. The GPU at full power is at 5-6W and that's why the Shield and Switch need an active fan to cool them.

    The gap is not closed by process normalisation.
  • BenSkywalker - Monday, February 12, 2018 - link

    Samsung says the gap is entirely closed by process normalisation alone(20nm<35%14nm<,40%10nmLPE<15%10nm LPE) , their claims- but hey, they just actually make the chips. What do the people who run the 10nm fab know compared to you, right?

    Three years later, they have a competitive part, you want to consider that a QC win, well, you are clearly their target customer. Spend more on lawyers, less on engineers :)
  • mfaisalkemal - Monday, February 12, 2018 - link

    after calculate with samsung process normalisation, nvidia still have a gap around 20% worse than qualcomm on gfxbench 3.0 manhattan.
    Nvidia tegra x1 : 12W * 0.3315 = 3.978 W (Normalize from 20nm to 10nm)
    61FPS @ 3.978W -->15.33FPS/W

    Adreno 640
    82FPS @ 4.4W --> 18.63FPS/W
  • mfaisalkemal - Monday, February 12, 2018 - link

    i mean adreno 630 lol Reply
  • BenSkywalker - Monday, February 12, 2018 - link

    Could you provide some links? Seems odd that the Adreno 640 uses the exact same wattage on two different benches.

    Also- your wattage consumption assumption negates the earlier link I provided showing a massive efficiency gain in terms of performance/watt once you moved away from nearing thermal limits. Either you would be able to clock the part higher at the same power level(reduced leakage, better matching of optimal power usage for the die etc) or you would use less power- not to mention you would no longer be using active cooling for something that low power(again reducing power draw).

    That would assume, of course, that no other improvement was possible in the last *THREE YEARS* since we saw this level of performance.

    BTW- We could also ask things like why is tessellation performance still *half* of a three year old SoC, but that would imply that Qualcomm actually cared about moving forward with technology.
  • mfaisalkemal - Monday, February 12, 2018 - link

    that data was from Andrei Frumusanu comment, and i think he test it but not published. yup your're right nvidia better on tesselation offscreen test, but adreno 630 better on texturing offscreen(15424mtexel/s vs 13427mtexel/s) and i guess gfxbenchmark car chase test(tesselation test) adreno 630 better than tegra x1 although the tesselation offscreen worse.

    your link provided why so low wattage(1.51W) i think because nvidia only estimate gpu power without ram power, in this article andrei and ryan estimate system power(device substract idle power like display etc.
  • Kvaern1 - Tuesday, February 13, 2018 - link

    I'd be much more disappointed with NVidia if Qualcomm could make GPUs as well as them.

    Now, if only NVidia could make a competitive CPU.
  • Eximorph - Tuesday, February 13, 2018 - link

    I have the shield tv, shield k1 and an lg g5 ( tegra x1, tegra k1 and adreno 530) let me tell you that the tegra x1 is powerful but let's be honest. First The tegra x1 is connected to a power sources, second the tegra x1 is on max performance mode at all time, 3rd it have fans and 4th lets go to the specs, 256 cores at 1000 mhz vs 256 cores at 624 mhz on the adreno 530 we are talking about 376 mhz more per core for the tegra x1 over the adreno 530, screen resolution 1080 vs 2k. Now lets under clock the x1 to a 624 mhz and let put a 2k resolution screen and let see whats happen ? The result with just a 2k screen is the next one, Manhattan 3.0 offscreen, google pixel c 46 fps adreno 530 46 fps. So the true here is that Qualcomm is a beast on the gpu side and nvidia, apple and amr have alote to learn. Qualcomm is not behind, Qualcomm is far ahead. a really small chip with a really low power consumption and a great performance. The tegra x1 do not have nothing to look against the adreno 630. Reply
  • Eximorph - Tuesday, February 13, 2018 - link Reply

Log in

Don't have an account? Sign up now