System & ML Performance

Having investigated the new A13’s CPU performance, it’s time to look at how it performs in some system-level tests. Unfortunately there’s still a frustrating lack of proper system tests for iOS, particularly when it comes to tests like PCMark that would more accurately represent application use-cases. In lieu of that, we have to fall back to browser-based benchmarks. Browser performance is still an important aspect of device performance, as it remains one of the main workloads that put large amounts of stress on the CPU while exhibiting performance characteristics such as performance latency (essentially, responsiveness).

As always, the following benchmarks aren’t just a representation of the hardware capabilities, but also the software optimizations of a phone. iOS13 has again increased browser-based benchmarks performance by roughly 10% in our testing. We’ve gone ahead and updated the performance figures of previous generation iPhones with new scores on iOS13 to have proper Apple-to-Apple comparisons for the new iPhone 11’s.

Speedometer 2.0 - OS WebView

In Speedometer 2.0 we see the new A13 based phones exhibit a 19-20% performance increase compared to the previous generation iPhone XS and the A12. The increase is in-line with Apple’s performance claims. The increase this year is a bit smaller than what we saw last year with the A12, as it seems the main boost to the scores last year was the upgrade to a 128KB L1I cache.

JetStream 2 - OS Webview

JetStream 2 is a newer browser benchmark that was released earlier this year. The test is longer and possibly more complex than Speedometer 2.0 – although we still have to do proper profiling of the workload. The A13’s increases here are about 13%. Apple’s chipsets, CPUs, and custom Javascript engine continue to dominate the mobile benchmarks, posting double the performance we see from the next-best competition.

WebXPRT 3 - OS WebView

Finally WebXPRT represents more of a “scaling” workload that isn’t as steady-state as the previous benchmarks. Still, even here the new iPhones showcase a 18-19% performance increase.

Last year Apple made big changes to the kernel scheduler in iOS12, and vastly shortened the ramp-up time of the CPU DVFS algorithm, decreasing the time the system takes to transition from lower idle frequencies and small cores idle to full performance of the large cores. This resulted in significantly improved device responsiveness across a wide range of past iPhone generations.

Compared to the A12, the A13 doesn’t change all that much in terms of the time it takes to reach the maximum clock-speed of the large Lightning cores, with the CPU core reaching its peak in a little over 100ms.

What does change a lot is the time the workload resides on the smaller Thunder efficiency cores. On the A13 the small cores are ramping up significantly faster than on the A12. There’s also a major change in the scheduler behavior and when the workload migrates from the small cores to the large cores. On the A13 this now happens after around 30ms, while on the A12 this would take up to 54ms. Due to the small cores no longer being able to request higher memory controller performance states on their own, it likely makes sense to migrate to the large cores sooner now in the case of a more demanding workload.

The A13’s Lightning cores are start off at a base frequency of around 910MHz, which is a bit lower than the A12 and its base frequency of 1180MHz. What this means is that Apple has extended the dynamic range of the large cores in the A13 both towards higher performance as well as towards the lower, more efficient frequencies.

Machine Learning Inference Performance

Apple has also claimed to have increased the performance of their neural processor IP block in the A13. To use this unit, you have to make use of the CoreML framework. Unfortunately we don’t have a custom tool for testing this as of yet, so we have to fall back to one of the rare external applications out there which does provide a benchmark for this, and that’s Master Lu’s AIMark.

Like the web-browser workloads, iOS13 has brought performance improvements for past devices, so we’ve rerun the iPhone X and XS scores for proper comparisons to the new iPhone 11.

鲁大师 / Master Lu - AIMark 3 - InceptionV3 鲁大师 / Master Lu - AIMark 3 - ResNet34 鲁大师 / Master Lu - AIMark 3 - MobileNet-SSD 鲁大师 / Master Lu - AIMark 3 - DeepLabV3

The improvements for the iPhone 11 and the new A13 vary depending on the model and workload. For the classical models such as InceptionV3 and ResNet34, we’re seeing 23-29% improvements in the inference rate. MobileNet-SSD sees are more limited 17% increase, while DeepLabV3 sees a major increase of 48%.

Generally, the issue of running machine learning benchmarks is that it’s running through an abstraction layer, in this case which is CoreML. We don’t have guarantees on how much of the model is actually being run on the NPU versus the CPU and GPU, as things can differ a lot depending on the ML drivers of the device.

Nevertheless, the A13 and iPhone 11 here are very competitive and provide good iterative performance boosts for this generation.

Performance Conclusion

Overall, performance on the iPhone 11s is excellent, as we've come to expect time and time again from Apple. With that said, however, I can’t really say that I notice too much of a difference to the iPhone XS in daily usage. So while the A13 delivers class leading performance, it's probably not going to be very compelling for users coming from last year's A12 devices; the bigger impact will be felt coming from older devices. Otherwise, with this much horsepower I feel like the user experience would benefit significantly more from an option to accelerate application and system animations, or rather even just turn them off completely, in order to really feel the proper snappiness of the hardware.

SPEC2006 Perf: Desktop Levels, New Mobile Power Heights GPU Performance & Power
POST A COMMENT

239 Comments

View All Comments

  • HammerStrike - Wednesday, October 16, 2019 - link

    FYI, while al other new models are rated IP68, the pro’s are listed as having a maximum submerged depth of 4 meters for 30 minutes, vs 2 for the non-pro model. Your spec comparison on the first page lists them all at 2 meters. Reply
  • colonelclaw - Wednesday, October 16, 2019 - link

    I think my perfect phone right now would be all of the 11 Pro's hardware shoved into the body the size of the iPhone 5, but obviously with the full-front screen. Unfortunately there doesn't appear to be much will amongst any of the premium phone manufacturers in wanting stuff to be smaller these days :(
    iPhone 11 Pro Nano anyone?
    Reply
  • p1esk - Wednesday, October 16, 2019 - link

    Yes, I’d love a smaller iPhone. Reply
  • nirolf - Thursday, October 17, 2019 - link

    Smaller and lighter! I'm holding on to my 7 until they release something similar in size. Reply
  • Eliadbu - Wednesday, October 16, 2019 - link

    no iphone 11 pro but *rumors* say there is new Iphone SE2 for 399$ with a13 and 3GB of ram
    but I doubt it will be full front screen or AMOLED screen. but imo it will sell like hot buns from the oven. there is huge market for budget phone and a new iphone SE with same SOC as flagship, plenty of ram and IOS for 399$ is really compelling I know many people just lurk for used\old gen Iphones just so they can get Iphone for low price, this phone will draw the masses.
    Reply
  • MamiyaOtaru - Friday, October 18, 2019 - link

    unfortunately the rumors seem to paint Apple as being more interested in the price aspect than the size aspect. Like it will be more modern innards in an iphone 8 body. Which misses a great deal about what I liked in the original SE. Reply
  • Eliadbu - Saturday, October 19, 2019 - link

    I see your point, but you need to understand the current market. Iphone sales and other premium flagship are declining while budget phones are on the rise especially in certain markets where most people can't afford expensive phone so they look for the best phone they can get at the budget lineup. Now there are cheaper options than 399$ but arguably no better options. Apple knows it loses huge market share by not creating option for that price point. Also if they put a13 in that phone it will be overkill. They should either use older chips that cost less now like the a12 or a11 which both are more than enough for budget phones, or they can use smaller more power efficient a13 that would cost less and put the money on newer display design or more cameras which will be better selling point than using a13, imo. Reply
  • Oyster - Wednesday, October 16, 2019 - link

    I believe it's time for AT to really upgrade its website to be able to make use of the full width of modern displays. It's excruciating to go through these reviews and click through each camera sample without being able to compare things side-by-side. Even the performance graphs are so cramped. I hope an upgrade is on the way... Reply
  • Raqia - Wednesday, October 16, 2019 - link

    "Nevertheless, I do question why Apple decided to be so aggressive in terms of power this generation."

    I imagine it has something to do with the much bigger battery they put into the phone; current draw wearing down the battery isn't as much of a concern when it is larger. The little cores are far more impressive this year with roughly A73-75 levels of performance at much better efficiency. They seemed to have listened to users by making the phone a bit chunkier in exchange for a bigger battery.

    The inefficient modem is still a sore spot this year which should hopefully be addressed next year with the inclusion of Qualcomm parts and IP. More importantly, the impact of a closed ecosystem with management who care more about profit than users is being felt by real customers in places like Hong Kong or vendors like EPIC who have avenues other than the official OS vendor sponsored app store on Android. Regulators should force Apple to offer an open store API for third parties to set up store fronts so that customers could have more choice based on pricing, quality, and reputation.
    Reply
  • UglyFrank - Wednesday, October 16, 2019 - link

    I really did not need to see those T Rex scores, it hurt me spirit as a lifetime Android Phone user.
    My S4 would get 17fps in T-Rex and the iPhone is getting ~16-19x that after only 6 years later.
    Reply

Log in

Don't have an account? Sign up now