The Snapdragon 855 Phone Roundup: Searching for the Best Implementations
by Andrei Frumusanu on September 5, 2019 8:30 AM ESTMachine Learning Inference Performance
AIMark 3
AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.
Summarizing the four sub-tests of AIMark 3, we see a few clear outliers: OnePlus 7 Pro and Xiaomi’s Black Shark 2 lack Qualcomms driver libraries on which the benchmark relies on (or they’re broken), and the application crashes. Another outlier is the G8, here the phone also lacked the libraries for hardware acceleration, but the app at least fell back onto CPU execution of the workloads, albeit at a massive performance penalty.
We notably see that AIMark has been able to implement Samsung’s NPU SDK as we’re seeing evident hardware acceleration. Huawei’s P30 Pro also makes use of its NPU here via its proprietary SDK. Naturally, Apple’s iPhone XS uses the CoreML framework to accelerate the AI workloads.
Overall, the Snapdragon 855 devices with the latest SDKs and frameworks here seem to compete extremely well, offering extensive performance that is leading the pack across all the different sub-tests.
AIBenchmark 3
AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.
We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.
AIBenchmark 3 - NNAPI CPU
We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.
We’re seeing largely regular results here, although some observations pop up again, such as seeing that the Black Shark 2 having a very conservative result in some subtests. The other big outlier is the OPPO Reno 10x, here we’re seeing that the phone consistently performs better than the rest of the pack. This is very interesting, particularly because the phone actually isn’t able to actually use the NNAPI acceleration in the subsequent tests we’re covering next. This means that the “plain” TensorFlow libraries the OPPO is making use of are performing better than what’s employed for the rest of the devices.
AIBenchmark 3 - NNAPI INT8
INT8 performance is dominated by the Snapdragon 855 devices, and this is thanks to the vector processing units of the Hexagon DSP. Test 3-6 stands out for some devices, and it’s likely that this is due to discrepancies in the NNAPI drivers which don’t fully accelerate things on the hardware for the S10, Mi9 and G8.
AIBenchmark 3 - NNAPI FP16
FP16 performance is again quite even across the board, with the difference coming down to DVFS and scheduler responsiveness. Here the Snapdragon 855 competes neck-in-neck with the Kirin 980, winning some tests while losing others. It’s to be remembered that the Kirin 980’s NPU doesn’t support INT8 acceleration at this time, and that’s why it shows up better in the FP16 benchmarks.
AIBenchmark 3 - NNAPI FP32
Finally, on the FP32 benchmark, Qualcomm accelerates these workloads on the GPU and is far ahead of the competition, which either have lacking GPU drivers or have to fall back to CPU acceleration.
Facebook
Twitter
RSS
47 Comments
View All Comments
cha0z_ - Monday, September 9, 2019 - link
Actually the exynos is more EXPENSIVE! Note 9 when released costed 1100 euros in Europe vs 1000 dollars in US, you can check the conversion yourself as 1000 euros are over 1100 dollars. The fact you got a samsung device for cheaper is nothing exclusive to you nor China - there are super big discounts in US too for the qualcomm variants.s.yu - Monday, September 9, 2019 - link
Actually I look before I buy, the SK version is the cheapest anywhere, I had the option of buying the HK version(on SD) of the same tier for ~$100 more but I decided against it thinking it's not worth the premium.A friend in the US also bought a Note 8 at the time and went for an HK version because getting it from HK is still cheaper than in the US and he preferred SD. I don't know how there are significant discounts in the US but suspect they're bound by contracts.
As for the high price in Europe that should come as a surprise to no one. Most electronics are more expensive in Europe, it's not specific to Samsung.
1nterceptor - Tuesday, September 10, 2019 - link
I agree, would be really great if we could see exy9825 vs exy 9820 vs sd855+ vs sd855. Why exy9820 and sd855 you may ask, well because of the software updates, i wonder if and how much difference does it make now after 6 months on the market and couple of firmware updates...1nterceptor - Tuesday, September 10, 2019 - link
...although, i believe exy9825 is not by any means much faster/better than the "old" 9820, it is probbably more efficient on the other hand...jrocket - Thursday, September 5, 2019 - link
Since many of these phones vary in price significantly, it would be interesting to see a "performance per dollar" value comparison.IUU - Friday, September 6, 2019 - link
Would be if it was not so trivial. I mean , if one phone costs 400 dollars and another one 800, there you have straight away your performance per dollar. All are sd 855s , so there is nothing more to consider. One could argue about camera, screen, battery life, but these are irrelevant to performance.flyingpants265 - Tuesday, September 17, 2019 - link
It says right in the article that the performance differs depending on software implementation.Wardrive86 - Thursday, September 5, 2019 - link
Excellent article as always! I would like to see which version of Android each phone has in the system performance tests to see if some of the older device performance data is up to date and also see which Opengl and Vulkan drivers each device has while testing was done...however even without that data still the best tech site there is. Thanks!Wardrive86 - Sunday, September 8, 2019 - link
For example:LG G7 Android 9 opengl driver : 331.0
Slingshot Extreme Unlimited OpenGL ES 3.1
Physics peak : 3486
Physics sustained : 3392
Graphics peak : 5467
Graphics sustained: 5326
(5 runs, 20 minutes, Room temp: 78F/25.6C)
Immediately followed by Work 2.0
Performance : 8146
Web : 6588
Video : 5701
Writing. : 9554
Photo : 15830
Data : 6314
yacoub35 - Thursday, September 5, 2019 - link
Aside from battery life, the things I would care most about in a phone comparison are:Does it have stereo speakers?
Does it have a headphone jack?
Does it use vanilla Android (or how close to vanilla is what it uses)?
Does it get all of the Android updates and security patches in a timely fashion?
How is the camera performance (speed to load the app and take a photo, image stabilization, low light performance)?
How much RAM does it have?
How much internal storage (excluding the OS) and how much expandable, if any?