System Performance

While synthetic test performance is one thing, and hopefully we’ve covered that well with SPEC, interactive performance in real use-cases behaves differently, and here software can play a major role in terms of the perceived performance.

I will openly admit that our iOS system performance suite looks extremely meager: we are only really left with our web browser tests, as iOS is quite lacking in meaningful alternatives such as to PCMark on the Android side.

Speedometer 2.0 - OS WebView

Speedometer 2.0 is the most up-to-date industry standard JavaScript benchmark which tests the most common and modern JS framework performance.

The A12 sports a massive jump of 31% over the A11, again pointing out that Apple’s advertised performance figures are quite underselling the new chipset.

We’re also seeing a small boost from iOS 12 on the previous generation devices. Here the boost comes not only thanks to an a change in how iOS’s scheduler handles load, but also thanks to further improvements in the ever evolving JS engine that Apple uses.

WebXPRT 3 - OS WebView

WebXPRT 3 is also a browser test, however its workloads are more wide-spread and varied, containing also a lot of processing tests. Here the iPhone XS showcases a smaller 11% advantage over the iPhone X.

Former devices here also see a healthy boost in performance, with the iPhone X ticking up from 134 to 147 points, or 10%. The iPhone 7’s A10 sees a larger boost of 33%, something we’ll get into more detail in a little bit.

iOS12 Scheduler Load Ramp Analyzed

Apple promised a significant performance improvement in iOS12, thanks to the way their new scheduler is accounting for the loads from individual tasks. The operating system’s kernel scheduler tracks execution time of threads, and aggregates this into an utilisation metric which is then used by for example the DVFS mechanism. The algorithm which decides on how this load is accounted over time is generally simple a software decision – and it can be tweaked and engineered to whatever a vendor sees fit.

Because iOS’s kernel is closed source, we’re can’t really see what the changes are, however we can measure their effects. A relatively simple way to do this is to track frequency over time in a workload from idle, to full performance. I did this on a set of iPhones ranging from the 6 to the X (and XS), before and after the iOS12 system update.

Starting off with the iPhone 6 with the A8 chipset, I had some odd results on iOS11 as the scaling behaviour from idle to full performance was quite unusual. I repeated this a few times yet it still came up with the same results. The A8’s CPU’s idled at 400MHz, and remained here for 110ms until it jumped to 600MHz and then again 10ms later went on to the full 1400MHz of the cores.

iOS12 showcased a more step-wise behaviour, scaling up earlier and also reaching full performance after 90ms.

The iPhone 6S had a significantly different scaling behaviour on iOS11, and the A9 chip’s DVFS was insanely slow. Here it took a total of 435ms for the CPU to reach its maximum frequency. With the iOS12 update, this time has been massively slashed down to 80ms, giving a great boost to performance in shorter interactive workloads.

I was quite astonished to see just how slow the scheduler was before – this is currently the very same issue that is handicapping Samsung’s Exynos chipsets and maybe other Android SoCs who don’t optimise their schedulers. While the hardware performance might be there, it just doesn’t manifest itself in short interactive workloads because the scheduler load tracking algorithm is just too slow.

The A10 had similar bad characteristics as the A9, with time to full performance well exceeding 400ms. In iOS12, the iPhone 7 slashes this roughly in half, to around 210ms. It’s odd to see the A10 being more conservative in this regard compared to the A9 – but this might have something to do with the little cores.

In this graph, it’s also notable to see the frequency of the small cores Zephyr cores – they start at 400MHz and peak at 1100MHz. The frequency in the graph goes down back to 758MHz because at this point there was a core switch over to the big cores, which continue their frequency ramp up until maximum performance.

On the Apple A11 – I didn’t see any major changes, and indeed any differences could just be random noise between measuring on the different firmwares. Both in iOS11 and iOS12, the A11 scales to full frequency in about 105ms. Please note the x-axis in this graph is a lot shorter than previous graphs.

Finally on the iPhone XS’s A12 chipset, we can’t measure any pre- and post- update as the phone comes with iOS12 out of the box. Here again we see that it reaches full performance after 108ms, and we see the transition of the tread from the Tempest cores over to the Vortex cores.

Overall, I hope this is the best and clear visual representation of the performance differences that iOS12 brings to older devices.

In terms of the iPhone XS – I haven’t had any issues at all with performance of the phone and it was fast. I have to admit I’m still a daily Android user, and I use my phones with animations completely turned off as I find they get in the way of the speed of a device. There’s no way to completely turn animation off in iOS, and while this is just my subjective personal opinion, I found they are quite hampering the true performance of the phone. In workloads that are not interactive, the iPhone XS just blazed through them without any issue or concern.

The A12 Tempest CPU & NN Performance Tests GPU Performance & Power
POST A COMMENT

245 Comments

View All Comments

  • Andrei Frumusanu - Friday, October 05, 2018 - link

    Hello all,

    This article is still fresh off the presses - I'm sure there's still typos/grammar to be fixed in the coming hours.

    Just a general note; this is my first iPhone review, and hopefully it makes up for AnandTech not having a review of the iPhone 8's/X last year. Unfortunately that happened in a time when there was no mobile editor with the site anymore, and I only rejoined after a few months after that.

    As always, feel free to reach out in the comments or per emails.
    Reply
  • OMGitsShan - Friday, October 05, 2018 - link

    As always, this is the iPhone review to wait for! Thank you Andrei Reply
  • versesuvius - Friday, October 05, 2018 - link

    Potato. Reply
  • Jetcat3 - Friday, October 05, 2018 - link

    Wonderful review Andrei! Just a few minor quibbles for the capacities of each li-ion battery.

    Xs is 2658 mAh
    Xs Max is 3174 mAh

    Keep up the good work!
    Reply
  • wicasapa - Friday, October 05, 2018 - link

    one thing was not clear in the display section ... the Xs display (OLED) uses more power on a black screen compared to 8's LCD at the same brightness?!! this doesn't sound right! or is this power draw attributable to something else (face detection sensors, etc.)? Reply
  • Andrei Frumusanu - Saturday, October 06, 2018 - link

    No, that's accurate. It's part of why the new phones regress in battery over the LCD iPhones. Reply
  • DERSS - Saturday, October 06, 2018 - link

    You asked about the display's matrix controlling logic. It was found out through patent research that Apple has developed LTPO TFT Technology for that, however it is so far is confirmed to only used in the displays LG Display makes for Apple Watch. For smartphone OLEDs Apple has older simpler technology designed for the transistor layer as these big OLEDs are already super pricy anyway. And additional manufacturers like LG Display where Apple has started to implement manufacturing of iPhone XS/Max displays only has very few Tokki Canon equipment sets so they can not really low the manufacturing pricing down. (And Apple can not move to a different equipment as it has designed iPhone X/XS/Max displays only for Tokki Canon equipment). Reply
  • wicasapa - Saturday, October 06, 2018 - link

    I understand the regression on the battery test you run, it is basically browsing the web, which is by and large a white background and generally above the 60-65% threshold for the crossover of efficiency between OLEDs and LCDs. it actually makes sense for OLEDs to be less efficient in web browsing-based battery tests. however, I was surprised that you measured less efficiency on a pure black screen! Reply
  • Andrei Frumusanu - Sunday, October 07, 2018 - link

    Actually the web test is varied, I have a few lower APL sites and even some black ones. Reply
  • tipoo - Sunday, October 14, 2018 - link

    Common misconception - OLEDS don't just turn off fully for pure blacks, it takes active power to create a black. When they're off they're a murky grey. Reply

Log in

Don't have an account? Sign up now