The Apple iPhone 11, 11 Pro & 11 Pro Max Review: Performance, Battery, & Camera Elevated
by Andrei Frumusanu on October 16, 2019 8:30 AM ESTSystem & ML Performance
Having investigated the new A13’s CPU performance, it’s time to look at how it performs in some system-level tests. Unfortunately there’s still a frustrating lack of proper system tests for iOS, particularly when it comes to tests like PCMark that would more accurately represent application use-cases. In lieu of that, we have to fall back to browser-based benchmarks. Browser performance is still an important aspect of device performance, as it remains one of the main workloads that put large amounts of stress on the CPU while exhibiting performance characteristics such as performance latency (essentially, responsiveness).
As always, the following benchmarks aren’t just a representation of the hardware capabilities, but also the software optimizations of a phone. iOS13 has again increased browser-based benchmarks performance by roughly 10% in our testing. We’ve gone ahead and updated the performance figures of previous generation iPhones with new scores on iOS13 to have proper Apple-to-Apple comparisons for the new iPhone 11’s.
In Speedometer 2.0 we see the new A13 based phones exhibit a 19-20% performance increase compared to the previous generation iPhone XS and the A12. The increase is in-line with Apple’s performance claims. The increase this year is a bit smaller than what we saw last year with the A12, as it seems the main boost to the scores last year was the upgrade to a 128KB L1I cache.
JetStream 2 is a newer browser benchmark that was released earlier this year. The test is longer and possibly more complex than Speedometer 2.0 – although we still have to do proper profiling of the workload. The A13’s increases here are about 13%. Apple’s chipsets, CPUs, and custom Javascript engine continue to dominate the mobile benchmarks, posting double the performance we see from the next-best competition.
Finally WebXPRT represents more of a “scaling” workload that isn’t as steady-state as the previous benchmarks. Still, even here the new iPhones showcase a 18-19% performance increase.
Last year Apple made big changes to the kernel scheduler in iOS12, and vastly shortened the ramp-up time of the CPU DVFS algorithm, decreasing the time the system takes to transition from lower idle frequencies and small cores idle to full performance of the large cores. This resulted in significantly improved device responsiveness across a wide range of past iPhone generations.
Compared to the A12, the A13 doesn’t change all that much in terms of the time it takes to reach the maximum clock-speed of the large Lightning cores, with the CPU core reaching its peak in a little over 100ms.
What does change a lot is the time the workload resides on the smaller Thunder efficiency cores. On the A13 the small cores are ramping up significantly faster than on the A12. There’s also a major change in the scheduler behavior and when the workload migrates from the small cores to the large cores. On the A13 this now happens after around 30ms, while on the A12 this would take up to 54ms. Due to the small cores no longer being able to request higher memory controller performance states on their own, it likely makes sense to migrate to the large cores sooner now in the case of a more demanding workload.
The A13’s Lightning cores are start off at a base frequency of around 910MHz, which is a bit lower than the A12 and its base frequency of 1180MHz. What this means is that Apple has extended the dynamic range of the large cores in the A13 both towards higher performance as well as towards the lower, more efficient frequencies.
Machine Learning Inference Performance
Apple has also claimed to have increased the performance of their neural processor IP block in the A13. To use this unit, you have to make use of the CoreML framework. Unfortunately we don’t have a custom tool for testing this as of yet, so we have to fall back to one of the rare external applications out there which does provide a benchmark for this, and that’s Master Lu’s AIMark.
Like the web-browser workloads, iOS13 has brought performance improvements for past devices, so we’ve rerun the iPhone X and XS scores for proper comparisons to the new iPhone 11.
The improvements for the iPhone 11 and the new A13 vary depending on the model and workload. For the classical models such as InceptionV3 and ResNet34, we’re seeing 23-29% improvements in the inference rate. MobileNet-SSD sees are more limited 17% increase, while DeepLabV3 sees a major increase of 48%.
Generally, the issue of running machine learning benchmarks is that it’s running through an abstraction layer, in this case which is CoreML. We don’t have guarantees on how much of the model is actually being run on the NPU versus the CPU and GPU, as things can differ a lot depending on the ML drivers of the device.
Nevertheless, the A13 and iPhone 11 here are very competitive and provide good iterative performance boosts for this generation.
Performance Conclusion
Overall, performance on the iPhone 11s is excellent, as we've come to expect time and time again from Apple. With that said, however, I can’t really say that I notice too much of a difference to the iPhone XS in daily usage. So while the A13 delivers class leading performance, it's probably not going to be very compelling for users coming from last year's A12 devices; the bigger impact will be felt coming from older devices. Otherwise, with this much horsepower I feel like the user experience would benefit significantly more from an option to accelerate application and system animations, or rather even just turn them off completely, in order to really feel the proper snappiness of the hardware.
Facebook
Twitter
RSS
242 Comments
View All Comments
Irish910 - Friday, October 18, 2019 - link
Why so salty? If you hate Apple so much why are you here reading this article? Sounds like you’re insecure with your android phone which basically gets mopped up with by the new iPhones in every area where it counts. Shoo shoo now.shompa - Thursday, October 17, 2019 - link
Desktop performance. Do you understand the difference between CPU performance and App performance? X86 has never had the fastest CPUs. They had windows and was good enough / cheaper than RISC stuff. The reason why for example Adobe is "faster" in X86 is that Intel adds more and more specific instructions AVX/AVX512 to halt competition. Adobe/MSFT are lazy companies and don't recompile stuff for other architectures.For example when DVD encoding was invented in 2001 by Pioneer/Apple DVD-R. I bought a 10K PC with the fastest CPU there was. Graphics, SCSI disks and so on. Doing a MPEG 2 encoding took 15 hours. My first mac was a 667mhz PowerBook. The same encoding took 90 minutes. No. G4 was not 10 times faster, it was ALTIVEC that intel introduced as AVX when Apple switched to Intel. X86 dont even have real 64bit and therefore the 32bit parts in the CPU cant be removed. X86 is the only computer system where 64bit code runs slower than 32bit (about 3%). All other real 64bit systems gained 30-50% in speed. And its not about memory like PC clickers belive. Intel/ARM and others had 38bit memory addressing. That is 64gig / with a 4gig limit per app. Still, today: how many apps use more than 4gig memory? RISC went 64bit in 1990. Sun went 64bit / with 64bit OS in 1997. Apple went 64bit in 2002. Windows went 64bit after Playstation4/XboxOne started to release 64bit games.
By controlling the OS and hardware companies can optimize OS and software. That is why Apple/Google and MSFT are starting to use own SoCs. And its better for customers. There are no reason a better X86 chip cost 400 dollars + motherboard tax 100 dollars. Intel 4 core CPUs 14nm cost less than 6 dollars to produce. The problem is customers: they are prepared to pay more for IntelInside and its based on the wrong notion "its faster". The faster MSFT moves to ARM / RISCV. The better. And if the rumors are right, Samsung is moving to RISCV. That would shake up the mobile market.
Quantumz0d - Thursday, October 17, 2019 - link
Samsung just killed Texas team funding. And you don't want to pay for a socketed board and industry standard but rather have a surfacw which runs on an off the shelf processor and has small workload target in a PC ?Also dude from where you are pulling this $6 of Intel CPUs and I presume you already know how the R&D works right in Lithography ? ROI pays off once the momentum has began. So you are frustrated of 4C8T Intel monopoly amd want some magical unicorn out of thin air which is as fast that and is cheap and is portable a.k.a Soldered. Intel stagnated because of no competition. Now AMD came with better pricing and more bang for buck.
Next from Bigroom Mainframes to pocket PC (unfortunate with iOS its not because of no Filesystem anf Google following same path of Scoped Storage) microsoft put computers in homes and now they recently started moving away into SaaS and DaaS bs. And now with thin client dream of yours Itll be detrimental to the Computer HW owners or who want to own.
We do not want a Propreitary own walled gardens with orwellian drama like iOS. We need more Linux and more powerful and robust OS like Windows which handles customization despite getting sandbagged by M$ on removing control panel slowly and migrating away from Win32. Nobody wants that.
https://www.computerworld.com/article/3444606/with...
jv007 - Wednesday, October 16, 2019 - link
The lighting big cores are not very impressive this time.From 4 Watt to 5 Watt a 25% increase in power for 17% more performance.
Good for benchmarks (and the phone was actively cooled here), but not good for throttling.
7nm and no EUV, maybe next year with 5nm and EUV will improve seriously.
I wonder if we will see a A13X.
name99 - Wednesday, October 16, 2019 - link
"The lighting big cores are not very impressive this time"A PHONE core that matches the best Intel has to offer is "not impressive"?
OK then...
Total Meltdowner - Thursday, October 17, 2019 - link
Comparing this CPU to intel is silly. They run completely different instructions.Quantumz0d - Sunday, October 20, 2019 - link
It has been overblown. The Spec score is all the A series chips have. They can't replace x86 chips even Apple uses x86 cores with Linux RHEL or Free OS Linux distribution to run their services. Whole world runs on the same ISA. These people just whiteknight it like a breakthrough while the whole iOS lacks basic Filesystem access and the latest Catalina cannot run non notarized apps.Also to note the Apple First party premium optimization, that Apple pays for companies like Adobe. If you run MacOS / Trashbook Pro BGA / iOS on any non optimized SW it will be held back both on power consumption and all. It's just a glorified Nix OS and with the first party support it keeps floating. They missed out on the mass scale deployment like Windows or Linux and that's going to be their Achilles heel along with the more transformation of MacOS into iOS rather than opposite.
It's really funny when you look how 60% of the performance is max that one can get from MacOS based HW/Intel machines due to severe thinning on chassis for that sweet BGA appeal and non user serviceable HW while claiming all recycled parts and all. I'm glad that Apple can't escape Physics. VRM throttling, low quality BGA engineering with cTDP garbage etc. Also people just blatantly forget how the DRAM of those x86 processors scales with more than 4000MHz of DDR4 and the PCIe lanes it pushes out with massive I/O while the anemic trash on Apple Macs is a USB C with Dongle world, ARM replicating the same esp the Wide A series with all the Uncore and PCIe I/O support ? Nope. It's not going to happen. Apple needs to invest Billions again and they are very conservative when it comes to this massive scale.
Finally to note, ARM cannot replace x86. Period. The HPC/ DC market of the Chipzilla Intel and AMD, they won't allow for this BS, Also the ISA of x86 is so mature plus how the LGA and other sockets happen along. While ARM is stuck with BGA BS and thus they can never replace these in the Consumer market.
Let the fanboys live in their dream utopia.
tipoo - Thursday, October 17, 2019 - link
Being that the little cores are more efficient, and the battery is significantly larger, maybe they allowed a one time regresion in peak performance per watt to gain that extra performance, without a node shrink this year.zeeBomb - Wednesday, October 16, 2019 - link
the time has come.joms_us - Wednesday, October 16, 2019 - link
Show us that A13 can beat even the first gen Ryzen or Intel Skylake , run PCMark, Cinebench or any modern games otherwise this nonsense desktop level claim should go to the bin. You are using a primitive Spec app to demonstrate the IPC?I can't wait for Apple to ditch the Intel processor inside their MBP and replace with this SoC. Oh wait no, it won't happen in a decade because this cannot run a full pledge OS with real multi-tasking. =D