The Snapdragon 865 Performance Preview: Setting the Stage for Flagship Android 2020
by Andrei Frumusanu on December 16, 2019 7:30 AM EST- Posted in
- Mobile
- Qualcomm
- Smartphones
- 5G
- Cortex A77
- Snapdragon 865
Machine Learning Inference Performance
AIMark 3
AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.
In AIMark 3, the benchmark uses each vendor’s proprietary SDK in order to accelerate the NN workloads most optimally. For Qualcomm’s devices, this means that seemingly the benchmark is also able to take advantage of the new Tensor cores. Here, the performance improvements of the new Snapdragon 865 chip is outstanding, posting in 2-3x performance compared to its predecessor.
AIBenchmark 3
AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.
We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.
AIBenchmark 3 - NNAPI CPU
We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.
Starting off with the CPU accelerated benchmarks, we’re seeing some large improvements of the Snapdragon 865. It’s particularly the FP workloads that are seeing some big performance increases, and it seems these improvements are likely linked to the microarchitectural improvements of the A77.
AIBenchmark 3 - NNAPI INT8
INT8 workload acceleration in AI Benchmark happens on the HVX cores of the DSP rather than the Tensor cores, for which the benchmark currently doesn’t have support for. The performance increases here are relatively in line with what we expect in terms of iterative clock frequency increases of the IP block.
AIBenchmark 3 - NNAPI FP16
FP16 acceleration on the Snapdragon 865 through NNAPI is likely facilitated through the GPU, and we’re seeing iterative improvements in the scores. Huawei’s Mate 30 Pro is in the lead in the vast majority of the tests as it’s able to make use of its NPU which support FP16 acceleration, and its performance here is quite significantly ahead of the Qualcomm chipsets.
AIBenchmark 3 - NNAPI FP32
Finally, the FP32 test should be accelerated by the GPU. Oddly enough here the QRD865 doesn’t fare as well as some of the best S855 devices. It’s to be noted that the results here today were based on an early software stack for the S865 – it’s possible and even very likely that things will improve over the coming months, and the results will be different on commercial devices.
Overall, there’s again a conundrum for us in regards to AI benchmarks today, the tests need to be continuously developed in order to properly support the hardware. The test currently doesn’t make use of the Tensor cores of the Snapdragon 865, so it’s not able to showcase one of the biggest areas of improvement for the chipset. In that sense, benchmarks don’t really mean very much, and the true power of the chipset will only be exhibited by first-party applications such as the camera apps, of the upcoming Snapdragon 865 devices.
Facebook
Twitter
RSS
178 Comments
View All Comments
yankeeDDL - Tuesday, December 17, 2019 - link
No, I was looking at the Web benchmarks. All of them are miserable compared even to the iPhone X.And Web browsing is certainly a key part of mobile experience.
joms_us - Tuesday, December 17, 2019 - link
And yet in real world comparison, iPhone is faster only in handful of sites (excluding apple.com LOL) by millisec. Does that look miserable to you? =DNicon0s - Tuesday, December 17, 2019 - link
Yes and have you actually seen an Android flagship that performs noticeably worse than an iphone(any iphone) in web browsing? Because I sure haven't.What I have seen is Android phones with better connection speed and better reception in general, especial in crowded places like concerts, stadiums, subways etc. In those places the performance from the iphone x was actually 0 in many instances because there was no signal. If we are talking about the mobile experience let's not ignore the Modem.
s.yu - Tuesday, December 17, 2019 - link
"If we are talking about the mobile experience let's not ignore the Modem."And there are connectivity tests for that, although controlling variables in a connectivity test is almost impossible outside of lab conditions.
Ultimately these are separate tests and test entirely different aspects. You're free to test both, but if you try to test them simultaneously outputting one single result, you will obtain one worthless data point.
Wilco1 - Tuesday, December 17, 2019 - link
Synthetic browsing benchmarks depend highly on software implementation and tuning, so they are not useful to compare CPU performance.yeeeeman - Tuesday, December 17, 2019 - link
A bit MEH...Alistair - Tuesday, December 17, 2019 - link
The Android fanatics are out tonight. I only buy Android phones, period. Clearly Apple's CPUs are miles ahead in performance. Anyone use the latest iPad Pro? It's faster than any Windows laptop I've bought or used personally. Boss gave us the latest and greatest dual core HP ultra slim laptop, and I immediately replaced it with a Ryzen 8 core computer and said "I don't use laptops for real work". We don't need benchmarks to tell us how fast Apple's CPUs are (though Andrei's benchmarks are perfectly valid), it is immediately apparent when comparing vs. similarly clocked Intel products. The reasons Ryzen and Intel seem great right now are high clocks and many cores. Run your 9900k in 2 core mode, at 2.6 Ghz and squirm at its low speed. That's what the iPhone uses, a low clocked dual core. Put 8 together, and run it at 4 Ghz, and you have a monster.id4andrei - Tuesday, December 17, 2019 - link
You're forgetting a stripped down mobile OS and relatively stripped down mobile apps that are part of the speed equation.Alistair - Tuesday, December 17, 2019 - link
that has nothing to do with it, that's what I'm saying, that's what Andrei is sayingAlistair - Tuesday, December 17, 2019 - link
the other Andrei lol