Machine Learning Inference Performance

AIMark 3

AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.

鲁大师 / Master Lu - AIMark 3 - InceptionV3 鲁大师 / Master Lu - AIMark 3 - ResNet34 鲁大师 / Master Lu - AIMark 3 - MobileNet-SSD 鲁大师 / Master Lu - AIMark 3 - DeepLabV3

Unfortunately for the ROG Phone II, it’s another device that’s lacking the proper Qualcomm proprietary libraries that AI Mark makes use of, ending up not being in the charts anywhere as the application couldn’t start.

AIBenchmark 3

AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.

We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.

AIBenchmark 3 - NNAPI CPU

We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.

AIBenchmark 3 - 1 - The Life - CPU/FP AIBenchmark 3 - 2 - Zoo - CPU/FP AIBenchmark 3 - 3 - Pioneers - CPU/INT AIBenchmark 3 - 4 - Let's Play - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/INT AIBenchmark 3 - 8 - Blur iT! - CPU/FP

We performed the AI Benchmark tests in the default mode of the phone, and here the ROG Phone II lands relatively in average to better than average amongst S855 devices.

AIBenchmark 3 - NNAPI INT8

AIBenchmark 3 - 1 - The Life - INT8 AIBenchmark 3 - 2 - Zoo - Int8 AIBenchmark 3 - 3 - Pioneers - INT8 AIBenchmark 3 - 5 - Masterpiece - INT8 AIBenchmark 3 - 6 - Cartoons - INT8

AIBenchmark 3 - NNAPI FP16

AIBenchmark 3 - 1 - The Life - FP16 AIBenchmark 3 - 2 - Zoo - FP16 AIBenchmark 3 - 3 - Pioneers - FP16 AIBenchmark 3 - 5 - Masterpiece - FP16 AIBenchmark 3 - 6 - Cartoons - FP16 AIBenchmark 3 - 9 - Berlin Driving - FP16 AIBenchmark 3 - 10 - WESPE-dn - FP16

AIBenchmark 3 - NNAPI FP32

AIBenchmark 3 - 10 - WESPE-dn - FP32

The same conclusion can be had of the INT8, FP16 and FP32 scores. It’s odd that the FP32 scores aren’t better than that of the competition as the workload is accelerated by the GPU, and we had expected the S855+’s higher GPU frequency to have a larger impact on this score.

We also have to note that if it weren’t for a very recent firmware update from ASUS we wouldn’t be able to publish any AI benchmarks at all, as it previously lacked the proper NNAPI drivers. As such, it seems that this aspect was pretty much just an afterthought for the phone rather than something that was on a critical to-do list.

System Performance GPU Performance - Top Performance
Comments Locked

75 Comments

View All Comments

  • mrochester - Tuesday, October 1, 2019 - link

    *buy.
  • PeachNCream - Tuesday, October 1, 2019 - link

    Anything Google touches basically turns into a platform for data collection that exploits the user and yest the phone is ugly. What sorts of alternatives did you have in mind?
  • AdhesiveTeflon - Wednesday, October 2, 2019 - link

    You want us to get a Blackberry instead?
  • Azurael - Tuesday, October 1, 2019 - link

    A phone with a large battery, no notch and a screen which isn't wrapped around the edges of the phone. Perfect but for the fact it looks like it was designed for a 12 year old (with rich parents?) and the camera sucks
  • s.yu - Tuesday, October 1, 2019 - link

    1. 3rd party cases look serious enough. Also consider removing the back panel and the whole paint job inside like how some people made Samsungs transparent.
    2. Port Gcam.
  • abufrejoval - Tuesday, October 1, 2019 - link

    These devices are simply the new [very] personal computers. The fact, that they evolved from phones is about as meaningful as to say that humans are a special type of single-cell organism, that stopped separating completely after cell division.

    And when you look at the typical daily usage pattern of these VPCs, you'll find that many, especially younger users, will go days without using the phone functionality at all: That's Mom & Pop stuff, ancient history and typically only used to remind them of chores left undone and thus silenced!

    The phone moniker only serves to justify why these devices you're supposedly buying to own, are kept tethered to vendors and telcos, who have no business whatsoever on your very personal computers, after they habe become your property and digital brain extension or Internet of Bodies limb.

    Please help ending this abuse if only by calling them by what they are instead of whence they originated eons ago.
  • PeachNCream - Tuesday, October 1, 2019 - link

    It's not just a kid thing. I could easily get by without any sort of more conventional PC by relying exclusively on my phone. In a roundabout way, I've already done that by neglecting to boot up a PC for several days and more often than not, I'm only turning them on to fetch updates or do something that benefits from a keyboard (less often now since I have a bluetooth keyboard paired up with my phone). The reverse situation where I would have to resort to only a PC and omit the phone would be more troublesome because of a lack of effective communications mechanisms and a lack of portability.
  • abufrejoval - Wednesday, October 2, 2019 - link

    Absolutely, and with this dock, I am seriosly considering to buy this as a mobile workstation: I do have some usable notebooks, quite a bit slower than this device (e.g. ChuWi 12.3).

    Of course, there is yet some other elements missing: The clamshell dock, which allows me to use this device as a notebook and the ability to run Linux desktop apps with the proper GPU acceleration: The current hacks running a Linux userland in a chroot() and an X-Server on the Android end eat too much 'snappyness' to.

    I'd keep a dock in every major office location I work in, with either a nice 4k screen or dual monitor setup, keyboard and mouse and then use the clamshell on planes and trains if the ride is long enough to make it worthwhile and otherwise just use it handheld or with WiDi for presentations. A serious conference room could have a WGig dock.

    It's not a hardware issue any longer, just "opposing software empires".
  • 29a - Tuesday, October 1, 2019 - link

    Is there anyway to get uBlock to block these advertisement articles?
  • zeeBomb - Tuesday, October 1, 2019 - link

    Is this the first real superphone???

Log in

Don't have an account? Sign up now