System Performance

To see how the new CPUs and memory subsystem translate into more real system performance, we move onto more representative tests such as PCMark. PCMark’s performance is affected by several factors: not only does raw performance of the hardware count but also we need to consider the individual system’s software stack. We’ve seen large differences between Android OS major versions where the improvements of the Android Runtime can be directly visible in subtests such as the Writing test. Also a SoC’s DVFS schemes and schedulers can have huge impacts on “performance-latency”, meaning how fast the CPUs can ramp up a workload. This directly translates in a lot more performance in several of PCMark’s subtests as in the default settings none of the tests actually represent the pure performance of the CPU if it were locked at maximum frequency on the performance cores. The results of the tests are also overall a good representation of “snappiness” of a device.

PCMark Work 2.0 - Web Browsing 2.0

In the web browsing test the Snapdragon 845 QRD manages to outpace the Pixel 2 XL by 20%. Here we’re also looking at performance across devices with different OS versions. The Google devices are running Android 8.1 while the Samsung devices were tested with Android 7.0. The Mate 10 Pro runs Android 8.0 while the Mate 9 still had 7.0. The Qualcomm QRD we tested ran Android 8.0.

Again the performance increase over Snapdragon 835 devices isn’t all that great. DynamiQ allows for far more efficient thread transitions between the CPU cores and subsequently I expected Qualcomm to take advantage of this through more aggressive scheduling resulting in more than just a 20% increase. The difference between the Mate 9 and Mate 10 here is a good example of what a software configuration change can bring in terms of performance (both devices employ same performance CPU configurations). Samsung’s Exynos’ SoCs still use GTS scheduling and have non-optimal performance-latency resulting in bad scores, amplified by the fact that Samsung’s memory performance is also underwhelming when compared to the Snapdragon and Kirin SoCs.

PCMark Work 2.0 - Data ManipulationPCMark Work 2.0 - Writing 2.0

The Data Manipulation and Writing 2.0 tests make heavy use of the Android runtime and APIs and also a very memory latency sensitive. Between the best showings of the Snapdragon 835 variant of the S8 and the Pixel 2 XL in each respective benchmark, the Snapdragon QRD845 showed conservative increases of 8 to 14%. The Exynos SoCs lacklustre performance is again hampered by software and by bad memory performance.

PCMark Work 2.0 - Video Editing

The video editing test is PCMark’s weak-point as it’s bottlenecked by things such as OS API overhead, and why we see tight grouping of performance results across a large range of SoCs. The Snapdragon 845 ends up high, but below the Pixel 2 XL. I would not put much weight on the results of this test as they’re not necessarily representative. Futuremark claims that the test is a lot more sensitive in mid- and low-range devices which can exhibit performance issues.

PCMark Work 2.0 - Photo Editing 2.0

The photo editing test makes heavy use of Renderscript and use GPU acceleration to apply various effects on an image set. The QRD845 here shines as it’s able to showcase a 38% performance improvement over the Pixel 2 XL. Again the test not solely tests the raw performance of the system but also how optimized it is in terms of the software stack. This can be seen in the Kirin vs Exynos devices as Huawei’s phones vastly outperform Samsung’s devices in this test.

PCMark Work 2.0 - Performance

Overall PCMark’s performance score for the QRD845 increases by 17% over the Pixel 2 XL. Disregarding the video test, we see a similar scenario as in the synthetic tests as the new SoC’s CPU performance increases are lower than we had expected. Still the Snapdragon 845 is able to top the charts and should adequately power 2018’s flagship devices.

For 2018 we are reviewing our mobile benchmarking suite and altering some of the benchmarks we use. One of the changes in the way we benchmark devices is that we’re moving away from standalone browser and rather are benchmarking the OS’s WebView implementations. In general this seems to be a better choice for testing device experience as there is a lot of content that is being consumed via WebView windows. We also avoid the argument about different browser performance and since Google has now made WebView an updatable Play Store component we should also have valid comparisons older devices and systems. On the iOS side we do the same as we now benchmark browser tests within a WkWebView shell.

WebXPRT 2015 - OS WebView

Starting off with WebXPRT 2015 for a last time before we’ll retire it in favour of WebXPRT 3, we see the QRD845 performing fantastically. Here the 44% performance increase over the Pixel 2 XL is a lot more in line with what we had expected of the new SoC. The QRD845 is even able to catch up a lot with Apple’s newest A11 and Monsoon cores in this test.

To keep up with the ever changing landscape of the developing web, we’re also retiring past JavaScript benchmarks in favour of a brand new and more representative benchmark developed by the WebKit team and welcomed by Google; Speedometer 2.0.

Speedometer 2.0 - OS WebView

Here the Snapdragon 845 showcased another healthy performance increase of 37% over the Snapdragon 835 devices. Apple’s superior JavaScript performance can be attributed to a much faster and more optimized Nitro engine while Google’s V8 has only seen meagre improvements over the years. Notable is the Apple A11’s massive performance jump over the A10 – vastly increasing the distance to Android devices.

CPU & Memory Subsystem GPU Performance & Power Estimates
POST A COMMENT

140 Comments

View All Comments

  • jjj - Monday, February 12, 2018 - link

    In high end phones perf doesn't quite matter anymore but A75 looks interesting.
    You seems to not like it and it's very suspicious that you have GPU power numbers but not CPU - that's the most important info we needed here, the only risk to see a big negative change.The one thing you had to look at and you don't..
    Anyway, since you avoid looking at what matters (and that really stinks), excluding power numbers, why wouldn't you like A75? It's a tiny core that gets quite awesome perf numbers, can't wait for server SoCs at 4GHz or more. 1 or almost 1 per MHz in GB integer for such a small core is awesome. If power is not terrible, sounds like a fantastic core. FP is a bit behind but maybe that's ok nowadays.
    Reply
  • Andrei Frumusanu - Monday, February 12, 2018 - link

    I'm not sure you read the article before writing your criticism, we explicitly talked about CPU power at the end:

    > And while we weren’t able to test for system power efficiency improvements for this preview, we weren’t left empty-handed and were able to quickly do a CPU power virus on the QRD845. The results there have turned out promising, with 1W per-core and slightly under 4W for four-core power usage, which are very much in line with the Snapdragon 835.

    I also explicitly state that it's disappointing that it didn't reach many of ARM's performance targets, in some tests it merely falls back to a clock frequency advantage.
    Reply
  • jjj - Monday, February 12, 2018 - link

    My bad, only looked at the benchmarks, for the most part. Thanks for pointing out the power numbers. and if I may suggest, that info, even if limited, should be in the CPU section and visible.

    In GB, I am quite happy with it. To quote GB's knowledge base "Geekbench 4 uses a Microsoft Surface Book with an Intel Core i7-6600U processor as the baseline with a score of 4,000 points.".
    The clocks there peak at 3.4GHz so that means 1.1764 per MHz. Granted, not sure how much of that stands with the latest versions of the benchmark , might have changed since v.4.0. Anyway, you got Intel at 18% higher in integer and 59% in FP, clock for clock but with a many times larger core- we'll see how power compares at 4GHz or above.
    I'm impressed by perf density, not necessarily absolute perf and i also remind myself that very high ST perf is only needed in some niches. They do need some further gains with future gens but they are on the right track and it's starting to be exciting.

    Any chance you guys could do a A75 vs Coffee Lake, Ryzen and Atom and not focus on only SPEC?
    Reply
  • ZolaIII - Monday, February 12, 2018 - link

    Geek bench is highly unoptimized for X86 (probably not good optimized for ARM either) so for any real comparation you would have to compile it from the source with optimal flags... Reply
  • name99 - Monday, February 12, 2018 - link

    Of course it is ("highly unoptimized for X86")...
    And you learned this how, exactly?
    Hell, why don't you throw in the obligatory "John Poole is on the Apple payroll" while you're about it?
    Reply
  • ZolaIII - Monday, February 12, 2018 - link

    As it is! & it's Open Source but people like you don't even know what that is. So you fatch the code, add optimisation flags and compile it for a given target instead using generic one built for degeneric. Reply
  • Wilco1 - Monday, February 12, 2018 - link

    Geekbench isn't open source. Neither is it unoptimized for x86, the same compiler and the same options are used across multiple OSes and ISAs. Reply
  • Pdimri - Monday, February 12, 2018 - link

    What are the chances Google coming up its own SOC in next Pixel device to compete with A11 chip. Snapdragon chipset is lagging behind on year to year basis and this gap is going to widen even more. Reply
  • Andrei Frumusanu - Monday, February 12, 2018 - link

    Zero in my opinion. Reply
  • Speedfriend - Monday, February 12, 2018 - link

    Andrei - is that zero that they do they own chip or zero that it is competitive with Apple's? Reply

Log in

Don't have an account? Sign up now