The Apple iPhone 11, 11 Pro & 11 Pro Max Review: Performance, Battery, & Camera Elevated
by Andrei Frumusanu on October 16, 2019 8:30 AM ESTSystem & ML Performance
Having investigated the new A13’s CPU performance, it’s time to look at how it performs in some system-level tests. Unfortunately there’s still a frustrating lack of proper system tests for iOS, particularly when it comes to tests like PCMark that would more accurately represent application use-cases. In lieu of that, we have to fall back to browser-based benchmarks. Browser performance is still an important aspect of device performance, as it remains one of the main workloads that put large amounts of stress on the CPU while exhibiting performance characteristics such as performance latency (essentially, responsiveness).
As always, the following benchmarks aren’t just a representation of the hardware capabilities, but also the software optimizations of a phone. iOS13 has again increased browser-based benchmarks performance by roughly 10% in our testing. We’ve gone ahead and updated the performance figures of previous generation iPhones with new scores on iOS13 to have proper Apple-to-Apple comparisons for the new iPhone 11’s.
In Speedometer 2.0 we see the new A13 based phones exhibit a 19-20% performance increase compared to the previous generation iPhone XS and the A12. The increase is in-line with Apple’s performance claims. The increase this year is a bit smaller than what we saw last year with the A12, as it seems the main boost to the scores last year was the upgrade to a 128KB L1I cache.
JetStream 2 is a newer browser benchmark that was released earlier this year. The test is longer and possibly more complex than Speedometer 2.0 – although we still have to do proper profiling of the workload. The A13’s increases here are about 13%. Apple’s chipsets, CPUs, and custom Javascript engine continue to dominate the mobile benchmarks, posting double the performance we see from the next-best competition.
Finally WebXPRT represents more of a “scaling” workload that isn’t as steady-state as the previous benchmarks. Still, even here the new iPhones showcase a 18-19% performance increase.
Last year Apple made big changes to the kernel scheduler in iOS12, and vastly shortened the ramp-up time of the CPU DVFS algorithm, decreasing the time the system takes to transition from lower idle frequencies and small cores idle to full performance of the large cores. This resulted in significantly improved device responsiveness across a wide range of past iPhone generations.
Compared to the A12, the A13 doesn’t change all that much in terms of the time it takes to reach the maximum clock-speed of the large Lightning cores, with the CPU core reaching its peak in a little over 100ms.
What does change a lot is the time the workload resides on the smaller Thunder efficiency cores. On the A13 the small cores are ramping up significantly faster than on the A12. There’s also a major change in the scheduler behavior and when the workload migrates from the small cores to the large cores. On the A13 this now happens after around 30ms, while on the A12 this would take up to 54ms. Due to the small cores no longer being able to request higher memory controller performance states on their own, it likely makes sense to migrate to the large cores sooner now in the case of a more demanding workload.
The A13’s Lightning cores are start off at a base frequency of around 910MHz, which is a bit lower than the A12 and its base frequency of 1180MHz. What this means is that Apple has extended the dynamic range of the large cores in the A13 both towards higher performance as well as towards the lower, more efficient frequencies.
Machine Learning Inference Performance
Apple has also claimed to have increased the performance of their neural processor IP block in the A13. To use this unit, you have to make use of the CoreML framework. Unfortunately we don’t have a custom tool for testing this as of yet, so we have to fall back to one of the rare external applications out there which does provide a benchmark for this, and that’s Master Lu’s AIMark.
Like the web-browser workloads, iOS13 has brought performance improvements for past devices, so we’ve rerun the iPhone X and XS scores for proper comparisons to the new iPhone 11.
The improvements for the iPhone 11 and the new A13 vary depending on the model and workload. For the classical models such as InceptionV3 and ResNet34, we’re seeing 23-29% improvements in the inference rate. MobileNet-SSD sees are more limited 17% increase, while DeepLabV3 sees a major increase of 48%.
Generally, the issue of running machine learning benchmarks is that it’s running through an abstraction layer, in this case which is CoreML. We don’t have guarantees on how much of the model is actually being run on the NPU versus the CPU and GPU, as things can differ a lot depending on the ML drivers of the device.
Nevertheless, the A13 and iPhone 11 here are very competitive and provide good iterative performance boosts for this generation.
Performance Conclusion
Overall, performance on the iPhone 11s is excellent, as we've come to expect time and time again from Apple. With that said, however, I can’t really say that I notice too much of a difference to the iPhone XS in daily usage. So while the A13 delivers class leading performance, it's probably not going to be very compelling for users coming from last year's A12 devices; the bigger impact will be felt coming from older devices. Otherwise, with this much horsepower I feel like the user experience would benefit significantly more from an option to accelerate application and system animations, or rather even just turn them off completely, in order to really feel the proper snappiness of the hardware.
Facebook
Twitter
RSS
242 Comments
View All Comments
BradleyTwo - Monday, October 21, 2019 - link
My apologies if this has been disclosed already, but would it be possible to ask if Apple supplied these phones for testing?The reason I ask is that there is quite a long thread over at a popular Mac rumors forum where a number of us are concerned at the variable screen quality on the iPhone 11 Pro and Pro Max.
Many of us, myself included, have received a suboptimal screen, in that it was a dim, murky yellow display (the less polite of us have called them p-stained), while others have received screens which are not uniformly lit.
We have generally exchanged them to receive marginally better units, a few of which have been perfect, but a disappointing majority of the exchanges are often still below the apparently impressive characteristics of the displays discussed in the review.
As this is not mentioned in the various iPhone 11 Pro reviews, a number of us have formed suspicions that Apple has cherry picked the best screens to supply to reviewers.
A clarification whether Apple did indeed supply the units, or if they were bought off the shelf, would be much appreciated.
techsorz - Monday, October 21, 2019 - link
Apple calibrates each device, this is what XDR essentially is. However this will create better uniformity across displays than make them as different as you say.Dim, murky yellow is probably caused by you not disabling true-tone and auto-brightness. Otherwhise you have a very faulty unit, as this display should be bright enough to nearly burn out your retina. (Exaggeration)
Not uniformly lit could be an error, just return it in this case. Clearly Apple wouldn't supply faulty hardware to anyone on purpose, not testers or consumers.
Andrei Frumusanu - Monday, October 21, 2019 - link
These are Apple review samples, but in our experience and testing they don't differ from commercial models.BradleyTwo - Monday, October 21, 2019 - link
Thank you for the clarification. It would of course be negligent for Apple PR not to ensure reviewers receive fully tested units.I can assure you, however, that when it comes to the screen, the number of less than optimal units being sold at retail is probably higher than you might think. While these are most likely all within manufacturing tolerances for QC purposes, some of them I highly doubt Apple would send to reviewers.
Oh well, at least the 14 day return period provides the opportunity to exchange. The "screen lottery" we call it.
Andrei Frumusanu - Tuesday, October 22, 2019 - link
Apple would have to be very misleading in providing fully sealed units. It's possible that some retail units perform worse but over the years we've never really encountered such a unit.s.yu - Tuesday, October 22, 2019 - link
"It would of course be negligent for Apple PR not to ensure reviewers receive fully tested units."lol! Samsung Fold.
joms_us - Tuesday, October 22, 2019 - link
It is pity though the so-called fastest SoC is not even close to these Android phones which are typically half the speed of fastest desktops. How do you expect people to believe A13 is faster than i-9900K or Ryzen 3950X? Where GeekBiased and jurassic SP2006? LOLhttps://youtu.be/ay9V5Ec8eiY?t=514
https://youtu.be/DtSgdrKztGk?t=423
https://youtu.be/PkVW5eSXKfw?t=115
I'd say cut the crap and show us real-world results not cherry-picked worthless numbers from benchmarking tools.
Quantumz0d - Tuesday, October 22, 2019 - link
The fanboys man they are so blinded by reality, Apple was able to set a perfect world utopian dream for them. Can't fix stupid.I used to run Sultanxda kernel on my OP3 with SD820 processor the SD821 had higher clock speed over 820 but guess what OP screwed it up or Qualcomm didn't provide fix there was Clockspeed crashing at high freq so he disabled it entirely on both big and small. Guess what ? Benches took a massive hit. But UX ? Nope. Infact it improved a lot how is that possible ? I guess Spec and GB only matters right.
Pixel 3 lagged badly due to the RAM issue no one mentions all say its beautiful wonderful amazing. Guess what ? 1080P 60FPS doesn't exist as an option and its auto as Google deems. 4XL no 4K60FPS because less storage. No press mentions.
Coming to this garbage phone. iOS 13 whatever. Same icons, same springboard since 1.1.4 (I used it and JBed it) no desktop no customization to OS. All iPhones on the planet look same just like the brainwash here of ridiculous comparision of a GB (bullshit bench) and Spec score. Masterpiece of corporate koolaid.
Why don't they mention how the Audio format which records is not in Lossless but in AAC crap unlike my V30 does with the 192KHz 24Bit option in FLAC with Limiter and Gain switch or the Video mode which had full manual Pro controls or even the camera having any Manual options. All ASUS, Samsung, LG, Sony, OnePlus, Huawei offer Pro camera forget Pro Video which only LG and Sony do. But No one cares, simpletons only care about A series marketing BS.
The worst of all no Filesystem. $1000 device which doesn't even have a Filsystem usable by end user or has an option to install the apps off the AppStore. Nor any SD expansion slot to be prepared for emergency. But people are riling up and getting worked over the ARM masterrace LMAO with BGA MacBook Pro with 1 USB C port. Bonus is, to develop iOS app you must pay $99 yearly fee AND own that BGA Soldered KB/Touchpad/Battery/SSD macbook because XCode !!
The abomination design. Display mutilation for 3 years while heralding best colors best display LOL. Very funny.
And no 3.5mm jack. Because Apple wanted to make $5bn off revenue from AirPods (Higher than AMDs entire profit) guess what ? Less than 320kbps data rate LMAO. My LG V30 absolutely destroys this phone to oblivion with its ESS9218P DAC processor only found in Top motherboards from ASUS/GB/MSI. Even Vivo Nex decimates this garbage audio iPhone.
Very very funny how even Qualcomm who spent billions of dollars in R&D for their Centriq ARM server processor by even relegating the teams which worked on post 810, full custom Kryo 820 series and dumping all it out because of the Broadcom M&A (Major beneficiary was Apple due to the Hock Tan connection with Apple, he would sell out all LTE patents) impact and no profit in the ARM server market, forget logistics, capex, ROI, x86 emulation AND 64Bit x86 Emulation legacy code with a massive scale of Linux community around.
But we want ARM A series BGA processor which has world class Spec and GB score and beats Mainstream and HEDT LGA processors.
Claps !
Anand2019 - Tuesday, October 22, 2019 - link
Why are you so angry?Quantumz0d - Tuesday, October 22, 2019 - link
Fed up of the unending talk of x86 vs ARM is one hell even AT forums cpu and oc subforum. Whole thread dedicated to worship this talk.Two Apple ruined smartphones by this policy of removing jack and features while raising the price to moon, Other companies also want greed by forcing people to buy BT earphones which sound garbage, horrid longevity (Need to charge everyday) pushing people to buy trash (Beats) thus making whole market saturated with Apple agenda. Look at Google Pixel 4 they also removed offering Dongle, Samsung, OnePlus. Same thing like Apple very greedy.
Three destroyed the laptops with thin and light obsession. And soldered junk with less and less I/O.
Finally 4th - this corporation is built on American values but is a stooge to cash from China thus enabling more totalitarianship while claiming Liberty on US land. Spineless positiin.