Machine Learning Inference Performance

AIMark 3

AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.

鲁大师 / Master Lu - AIMark 3 - InceptionV3 鲁大师 / Master Lu - AIMark 3 - ResNet34 鲁大师 / Master Lu - AIMark 3 - MobileNet-SSD 鲁大师 / Master Lu - AIMark 3 - DeepLabV3

Unfortunately for the ROG Phone II, it’s another device that’s lacking the proper Qualcomm proprietary libraries that AI Mark makes use of, ending up not being in the charts anywhere as the application couldn’t start.

AIBenchmark 3

AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.

We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.

AIBenchmark 3 - NNAPI CPU

We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.

AIBenchmark 3 - 1 - The Life - CPU/FP AIBenchmark 3 - 2 - Zoo - CPU/FP AIBenchmark 3 - 3 - Pioneers - CPU/INT AIBenchmark 3 - 4 - Let's Play - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/INT AIBenchmark 3 - 8 - Blur iT! - CPU/FP

We performed the AI Benchmark tests in the default mode of the phone, and here the ROG Phone II lands relatively in average to better than average amongst S855 devices.

AIBenchmark 3 - NNAPI INT8

AIBenchmark 3 - 1 - The Life - INT8 AIBenchmark 3 - 2 - Zoo - Int8 AIBenchmark 3 - 3 - Pioneers - INT8 AIBenchmark 3 - 5 - Masterpiece - INT8 AIBenchmark 3 - 6 - Cartoons - INT8

AIBenchmark 3 - NNAPI FP16

AIBenchmark 3 - 1 - The Life - FP16 AIBenchmark 3 - 2 - Zoo - FP16 AIBenchmark 3 - 3 - Pioneers - FP16 AIBenchmark 3 - 5 - Masterpiece - FP16 AIBenchmark 3 - 6 - Cartoons - FP16 AIBenchmark 3 - 9 - Berlin Driving - FP16 AIBenchmark 3 - 10 - WESPE-dn - FP16

AIBenchmark 3 - NNAPI FP32

AIBenchmark 3 - 10 - WESPE-dn - FP32

The same conclusion can be had of the INT8, FP16 and FP32 scores. It’s odd that the FP32 scores aren’t better than that of the competition as the workload is accelerated by the GPU, and we had expected the S855+’s higher GPU frequency to have a larger impact on this score.

We also have to note that if it weren’t for a very recent firmware update from ASUS we wouldn’t be able to publish any AI benchmarks at all, as it previously lacked the proper NNAPI drivers. As such, it seems that this aspect was pretty much just an afterthought for the phone rather than something that was on a critical to-do list.

System Performance GPU Performance - Top Performance
Comments Locked

75 Comments

View All Comments

  • PeachNCream - Tuesday, October 1, 2019 - link

    Ian doesn't mention a weakness to kryptonite and I didn't see it wearing a cape so at this point the answer is unclear.
  • Notmyusualid - Sunday, October 6, 2019 - link

    Ha!
  • Chad - Tuesday, October 1, 2019 - link

    Pretty amazing device. Glad to see someone pushing boundaries in some ways.
  • sonicmerlin - Tuesday, October 1, 2019 - link

    If they want to make a "gaming phone" why not add some physical controls like the Nintendo Switch?
  • hemedans - Thursday, October 3, 2019 - link

    This Asus phone has physical control like switch, but you have to buy them separate.
  • sonicmerlin - Tuesday, October 1, 2019 - link

    I just realized I can't edit my comment so... at least include a controller accessory that attaches to it directly and acts as a cover as well
  • DianaDSmith - Wednesday, October 2, 2019 - link

    sheraz
  • Notmyusualid - Wednesday, October 2, 2019 - link

    As a ROG2 owner, happy to see you reviewed this 'phone'.

    I didn't buy this for gaming.

    For me, the purchasing decision lay with the battery size, the front facing speakers (great for GPS in the rental car to be heard), the dual sim & dual 4G (it was 4G+3G on S8+), headphone jack, and finally, lack-of curved screen - which I shall never part money for again.

    Mine is the 128GB/8G version, it was too cheap to ignore.

    Oddly, my Slingshot Extreme Physics score is 4,837 vs your 4,541 (ran Sep 27, 2019 22:08, check it online), which would easily top your chart. Glitch in the matrix? Dunno.

    I have a Chinese ROM, (and it was updated this morning). Whilst its rare to see the fact that it is a Chinese ROM, you do see it from time to time, in some screens, despite being set to English.

    I had it on launch-day. There was no 'sim card manager' in the s/w!?! I was unable to change data sim, choose a sim with which to respond with sms etc without popping a sim out to force a pop-up. This has now been corrected.

    Additionally, I initally never saw my dual sims report 4G & 4G. This has also been somewhat corrected, but as I sit in the UAE now, I am 3G & 4G. There is no way to force 4G & 4G from the menus. It will either do it (not often) or not, you have no choice. You can however set 3G & 3G, or 2G & 3G etc (but why). On the plus side, whichever sim is chosen for data, with take over the 4G role. So something here still very much requires attention.

    I miss the wireless charging, but the battery-size somewhat negates that. It was nice with the Samsung to drop it on the pad, everytime I sat down at the desk, and know it would be further topped-up when I picked it up to walk away again. And it was nice having the water protection as well, (considering we have a pool), as I have drowned a new handset before...Hmm, maybe I should back this thing up.
  • s.yu - Wednesday, October 2, 2019 - link

    What did you get to cover the back?
  • Notmyusualid - Wednesday, October 2, 2019 - link

    It shipped with a clear plastic hard case. But I changed it for a clear rubberised one.

Log in

Don't have an account? Sign up now