System & ML Performance

Having investigated the new A13’s CPU performance, it’s time to look at how it performs in some system-level tests. Unfortunately there’s still a frustrating lack of proper system tests for iOS, particularly when it comes to tests like PCMark that would more accurately represent application use-cases. In lieu of that, we have to fall back to browser-based benchmarks. Browser performance is still an important aspect of device performance, as it remains one of the main workloads that put large amounts of stress on the CPU while exhibiting performance characteristics such as performance latency (essentially, responsiveness).

As always, the following benchmarks aren’t just a representation of the hardware capabilities, but also the software optimizations of a phone. iOS13 has again increased browser-based benchmarks performance by roughly 10% in our testing. We’ve gone ahead and updated the performance figures of previous generation iPhones with new scores on iOS13 to have proper Apple-to-Apple comparisons for the new iPhone 11’s.

Speedometer 2.0 - OS WebView

In Speedometer 2.0 we see the new A13 based phones exhibit a 19-20% performance increase compared to the previous generation iPhone XS and the A12. The increase is in-line with Apple’s performance claims. The increase this year is a bit smaller than what we saw last year with the A12, as it seems the main boost to the scores last year was the upgrade to a 128KB L1I cache.

JetStream 2 - OS Webview

JetStream 2 is a newer browser benchmark that was released earlier this year. The test is longer and possibly more complex than Speedometer 2.0 – although we still have to do proper profiling of the workload. The A13’s increases here are about 13%. Apple’s chipsets, CPUs, and custom Javascript engine continue to dominate the mobile benchmarks, posting double the performance we see from the next-best competition.

WebXPRT 3 - OS WebView

Finally WebXPRT represents more of a “scaling” workload that isn’t as steady-state as the previous benchmarks. Still, even here the new iPhones showcase a 18-19% performance increase.

Last year Apple made big changes to the kernel scheduler in iOS12, and vastly shortened the ramp-up time of the CPU DVFS algorithm, decreasing the time the system takes to transition from lower idle frequencies and small cores idle to full performance of the large cores. This resulted in significantly improved device responsiveness across a wide range of past iPhone generations.

Compared to the A12, the A13 doesn’t change all that much in terms of the time it takes to reach the maximum clock-speed of the large Lightning cores, with the CPU core reaching its peak in a little over 100ms.

What does change a lot is the time the workload resides on the smaller Thunder efficiency cores. On the A13 the small cores are ramping up significantly faster than on the A12. There’s also a major change in the scheduler behavior and when the workload migrates from the small cores to the large cores. On the A13 this now happens after around 30ms, while on the A12 this would take up to 54ms. Due to the small cores no longer being able to request higher memory controller performance states on their own, it likely makes sense to migrate to the large cores sooner now in the case of a more demanding workload.

The A13’s Lightning cores are start off at a base frequency of around 910MHz, which is a bit lower than the A12 and its base frequency of 1180MHz. What this means is that Apple has extended the dynamic range of the large cores in the A13 both towards higher performance as well as towards the lower, more efficient frequencies.

Machine Learning Inference Performance

Apple has also claimed to have increased the performance of their neural processor IP block in the A13. To use this unit, you have to make use of the CoreML framework. Unfortunately we don’t have a custom tool for testing this as of yet, so we have to fall back to one of the rare external applications out there which does provide a benchmark for this, and that’s Master Lu’s AIMark.

Like the web-browser workloads, iOS13 has brought performance improvements for past devices, so we’ve rerun the iPhone X and XS scores for proper comparisons to the new iPhone 11.

鲁大师 / Master Lu - AIMark 3 - InceptionV3 鲁大师 / Master Lu - AIMark 3 - ResNet34 鲁大师 / Master Lu - AIMark 3 - MobileNet-SSD 鲁大师 / Master Lu - AIMark 3 - DeepLabV3

The improvements for the iPhone 11 and the new A13 vary depending on the model and workload. For the classical models such as InceptionV3 and ResNet34, we’re seeing 23-29% improvements in the inference rate. MobileNet-SSD sees are more limited 17% increase, while DeepLabV3 sees a major increase of 48%.

Generally, the issue of running machine learning benchmarks is that it’s running through an abstraction layer, in this case which is CoreML. We don’t have guarantees on how much of the model is actually being run on the NPU versus the CPU and GPU, as things can differ a lot depending on the ML drivers of the device.

Nevertheless, the A13 and iPhone 11 here are very competitive and provide good iterative performance boosts for this generation.

Performance Conclusion

Overall, performance on the iPhone 11s is excellent, as we've come to expect time and time again from Apple. With that said, however, I can’t really say that I notice too much of a difference to the iPhone XS in daily usage. So while the A13 delivers class leading performance, it's probably not going to be very compelling for users coming from last year's A12 devices; the bigger impact will be felt coming from older devices. Otherwise, with this much horsepower I feel like the user experience would benefit significantly more from an option to accelerate application and system animations, or rather even just turn them off completely, in order to really feel the proper snappiness of the hardware.

SPEC2006 Perf: Desktop Levels, New Mobile Power Heights GPU Performance & Power
POST A COMMENT

239 Comments

View All Comments

  • michael2k - Wednesday, October 16, 2019 - link

    You're asking that the phone under report it's battery reserve and shut the phone down at 40% battery to preserve battery longevity?

    Because that would be the effect. So instead of a battery that lasts 14 hours for the first year and then 10 hours the second, it would 'shut down' after 11 hours the first year, and 'shut down' after 11 hours the second year, and 'shut down' after 11 hours the third year, before the degradation actually causes the battery life to actually be 10 hours in year four.
    Reply
  • melgross - Wednesday, October 16, 2019 - link

    Wow! That makes no sense. All phones slow down over time, and all batteries hold less charge. Apple’s are t worse, if anything, they’re better. My Max, from last year still reads 100% on battery health after more than 11 months of fairly heavy daily use. I’d like to see other phones that do better. Reply
  • shompa - Thursday, October 17, 2019 - link

    Look at intel /AMD / Qualcomm. They list a "turbo speed" that is not guaranteed. But customers believe it is. That's why they don't need to downclock stuff because they never need to hit their speeds. Apple is the last vendor having a real CPU speed and holds it. I have had a multitude of Intel CPUs that under-deliver in speed and as a customer you can't do anything. The service centers simply don't understand the problem since they only do a CPID check and says "it works". Take any intel laptop and fire up an H265 encode and watch the CPU speed go down. A CPU labeled 2.9ghz /3.9ghz turbo suddenly is a 2ghz part and you can't do anything about it. At least with Apple: get a good battery and it works. Reply
  • Total Meltdowner - Wednesday, October 16, 2019 - link

    All for the low low price of $1300. Pass. Reply
  • Zerrohero - Wednesday, October 16, 2019 - link

    $999 actually.

    This is a device that you can use for five years, or more, always with the latest software. Just get the battery replaced once.

    Amazing value, as iPhones always are.
    Reply
  • Total Meltdowner - Thursday, October 17, 2019 - link

    Nice troll brother. But a full loaded iPhone 11 Pro is $1299.

    iPhones are trash.
    Reply
  • Total Meltdowner - Thursday, October 17, 2019 - link

    Sorry, it's $1450! LOL!

    Almost $1800 with applecare! https://www.apple.com/shop/buy-iphone/iphone-11-pr...

    ahahahahahahha
    Reply
  • Irish910 - Friday, October 18, 2019 - link

    You can get an iPhone 11 with 128 gigs for $749, which pretty much mops the competition with battery life, CPU, GPU, longevity and value.

    I know your troll self will say something like “BR0 it’s only GoT a 720P scr33n, my 2015 GaLaXy HaS h1gher Res0lution!

    Fact is, most people don’t care about that. That’s why the XR was the most popular phone last year and that trend will continue.

    The pro starts at $999. Stop trolling. This site is for adults only. If you only post your lame hate comments, please go to YouTube. There’s plenty of room for your kind there.

    Shoo shoo now.
    Reply
  • Quantumz0d - Wednesday, October 16, 2019 - link

    This analysis is great but the Whiteknighting is insane.

    The design the primary aspect of a device, the display itself is notched no matter what calibration it has it's destroying the proper nature of how we perceive through our eyes and mind. Dead pixel zone for $999

    No expandable storage, No Filesystem access - Must use iTunes. This means for every basic work you must rely on your computer and the iCloud, mega ecosystem lockdown.

    No 3.5mm jack. Its really a shame how this company made billions by buying up Beats (Sub par garbage audio) and AirPods ($179 of Sub 320kbps audio with limited life due to Li Ion) and making a dongle business out of an Analog standard and whole industry reeks of this greed by dongles snd their own BT products which are massively inferior in sound snd usage and also wear down the only port it has.

    Sealed battery, Strong adhesive with new 4m depth rating. $600 for back glass repair if no Apple care and they are forcing you to buy because its $300 only and 70USD per repair. More profit for Apple for $100 battery services. Unfortunate that Apple has brick and mortar rest do not but they want to siphon off. Also did author note how iPhone XS got the new battery throttling with latest iOS update ? Yeah bonus package to wreck all that performance, inherent overdrawing of Voltage and planned obsolescence.

    Too much of this price hike and offering measly 64GB base. Next year another $50-100 due to new design or whatever they want to call.

    Desktop performance. I want to see, can this A series chip run an Adobe CS6 or Blender or do a H264 Conversion faster or on par with a desktop chip ? Or play high refresh rate gaming or can it execute x86 instructions with ease and replace my PC with this BGA pile of junk ? (It cannot, I think it's too much of blowing into this hot balloon of Apple for mega limelight) same for 9900K or 3950X they can't be fit in a pocket.

    Finally the corporation of American back, has no backbone when it comes to China. The $$$$ speaks. Censorship and aiding the Orwellian draconian principles for cash is more than the American culture that spawned the company and its people. A big shame.
    Reply
  • Total Meltdowner - Wednesday, October 16, 2019 - link

    Bingo. I think the writer was just happy that he did notice performance improvements in this model. Still, the iPhone lacks innovation and that price tag is absurd. I'd buy it for $600 but $1000 or more? hah! Reply

Log in

Don't have an account? Sign up now