The ASUS ROG Phone II Review: Mobile Gaming First, Phone Secondby Dr. Ian Cutress on September 30, 2019 11:00 AM EST
Machine Learning Inference Performance
AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.
Unfortunately for the ROG Phone II, it’s another device that’s lacking the proper Qualcomm proprietary libraries that AI Mark makes use of, ending up not being in the charts anywhere as the application couldn’t start.
AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.
We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.
AIBenchmark 3 - NNAPI CPU
We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.
We performed the AI Benchmark tests in the default mode of the phone, and here the ROG Phone II lands relatively in average to better than average amongst S855 devices.
AIBenchmark 3 - NNAPI INT8
AIBenchmark 3 - NNAPI FP16
AIBenchmark 3 - NNAPI FP32
The same conclusion can be had of the INT8, FP16 and FP32 scores. It’s odd that the FP32 scores aren’t better than that of the competition as the workload is accelerated by the GPU, and we had expected the S855+’s higher GPU frequency to have a larger impact on this score.
We also have to note that if it weren’t for a very recent firmware update from ASUS we wouldn’t be able to publish any AI benchmarks at all, as it previously lacked the proper NNAPI drivers. As such, it seems that this aspect was pretty much just an afterthought for the phone rather than something that was on a critical to-do list.