The Apple iPhone 11, 11 Pro & 11 Pro Max Review: Performance, Battery, & Camera Elevatedby Andrei Frumusanu on October 16, 2019 8:30 AM EST
SPEC2006 Perf: Desktop Levels, New Mobile Power Heights
Given that the we didn’t see too many major changes in the microarchitecture of the large Lighting CPU cores, we wouldn’t expect a particularly large performance increase over the A12. However, the 6% clock increase alongside with a few percent improvement in IPC – thanks to improvements in the memory subsystems and core front-end – could, should, and does end up delivering around a 20% performance boost, which is consistent with what Apple is advertising.
I’m still falling back to SPEC2006 for the time being as I hadn’t had time to port and test 2017 for mobile devices yet – it’s something that’s in the pipeline for the near future.
In SPECint2006, the improvements in performance are relatively evenly distributed. On average we’re seeing a 17% increase in performance. The biggest gains were had in 471.omnetpp which is latency bound, and 403.gcc which puts more pressure onto the caches; these tests saw respective increases of 25 and 24%, which is quite significant.
The 456.hmmer score increases are the lowest at 9%. That workload is highly execution backend-bound, and, given that the Lightning cores didn’t see much changes in that regard, we’re mostly seeing minor IPC increases here along with the 6% increase in clock.
While the performance figures are quite straightforward and not revealing anything surprising, the power and efficiency figures on the other hand are extremely unexpected. In virtually all of the SPECint2006 tests, Apple has gone and increased the peak power draw of the A13 SoC; and so in many cases we’re almost 1W above the A12. Here at peak performance it seems the power increase was greater than the performance increase, and that’s why in almost all workloads the A13 ends up as less efficient than the A12.
In the SPECfp2006 workloads, we’re seeing a similar story. The performance increases by the A13 are respectable and average at 19% for the suite, with individual increases between 14 and 25%.
The total power use is quite alarming here, as we’re exceeding 5W for many workloads. In 470.lbm the chip went even higher, averaging 6.27W. If I had not been actively cooling the phone and purposefully attempting it not to throttle, it would be impossible for the chip to maintain this performance for prolonged periods.
Here we saw a few workloads that were more kind in terms of efficiency, so while power consumption is still notably increased, it’s more linear with performance. However in others, we’re still seeing an efficiency regression.
Above is a more detailed historical overview of performance across the SPEC workloads and our past tested SoCs. We’ve now included the latest high-end desktop CPUs as well to give context as to where the mobile is at in terms of absolute performance.
Overall, in terms of performance, the A13 and the Lightning cores are extremely fast. In the mobile space, there’s really no competition as the A13 posts almost double the performance of the next best non-Apple SoC. The difference is a little bit less in the floating-point suite, but again we’re not expecting any proper competition for at least another 2-3 years, and Apple isn’t standing still either.
Last year I’ve noted that the A12 was margins off the best desktop CPU cores. This year, the A13 has essentially matched best that AMD and Intel have to offer – in SPECint2006 at least. In SPECfp2006 the A13 is still roughly 15% behind.
In terms of power and efficiency, the A13 seemingly wasn’t a very successful iteration for Apple, at least when it comes to the efficiency at the chip’s peak performance state. The higher power draw should mean that the SoC and phone will be more prone to throttling and sensitive to temperatures.
One possible explanation for the quite shocking power figures is that for the A13, Apple is riding the far end of the frequency/voltage curve at the peak frequencies of the new Lightning cores. In the above graph we have an estimated power curve for last year’s A12 – here we can see that Apple is very conservative with voltage up until to the last few hundred MHz. It’s possible that for the A13 Apple was even more aggressive in the later frequency states.
The good news about such a hypothesis is that the A13, on average and in daily workloads, should be operating at significantly more efficient operating points. Apple’s marketing materials describe the A13 as being 20% faster along with also stating that it uses 30% less power than the A12, which unfortunately is phrased in a deceiving (or at least unclear) manner. While we suspect that a lot of people will interpret it to mean that A13 is 20% faster while simultaneously using 30% less power, it’s actually either one or the other. In effect what this means is that at the performance point equivalent to the peak performance of the A12, the A13 would use 30% less power. Given the steepness of Apple’s power curves, I can easily imagine this to be accurate.
Nevertheless, I do question why Apple decided to be so aggressive in terms of power this generation. The N7P process node used in this generation didn’t bring any major improvements, so it’s possible they were in a tough spot of deciding between increasing power or making due with more meager performance increases. Whatever the reason, in the end it doesn’t cause any practical issues for the iPhone 11’s as the chip’s thermal management is top notch.
Post Your CommentPlease log in or sign up to comment.
View All Comments
UglyFrank - Wednesday, October 16, 2019 - linkI really did not need to see those T Rex scores, it hurt me spirit as a lifetime Android Phone user.
My S4 would get 17fps in T-Rex and the iPhone is getting ~16-19x that after only 6 years later.
Death666Angel - Wednesday, October 16, 2019 - link"only 6 years later" Heh.
Pro-competition - Wednesday, October 16, 2019 - linkWhy was there no storage performance test? This affects app loading times (esp since Apple doesn't have much RAM), and app installation times, which affect real-world performance.
Andrei Frumusanu - Wednesday, October 16, 2019 - linkWe're lacking a better test that works properly. I prefer to skip it rather than have misleading figures.
Pro-competition - Wednesday, October 16, 2019 - linkUnderstood. Instead of synthetic benchmarks, perhaps just have a table of app loading times of some games? What I have in mind is the following:
1. Compare current iPhones with the previous generations of iPhones (maybe just go two generations back). And use iPhones with different storage capacities.
2. Ensure all iPhones have the same iOS version.
3. Select ~5 games for your test suite, ensuring that the same version is installed on all iPhones.
4. Before loading the game, close all apps and then restart the phone, so that when the game is launched, we can be assured that are no apps already running.
5. Do step 4 around five times.
6. Use a video recorder to measure the time.
What do you think of my proposal?
masimilianzo - Wednesday, October 16, 2019 - linkGreat review, thanks!
Any idea on LD/ST bandwidth? Have they increase the number of AGUs?
Andrei Frumusanu - Wednesday, October 16, 2019 - linkBandwidth is included in one of the charts. It looks unchanged, and I didn't see anything different on the instruction side either.
masimilianzo - Wednesday, October 16, 2019 - linkAre you measuring number of ports in the L1 RAM or number of address generation units?
I guess it depends on having or not conflict to read ports in the RAM.
The large uplift in h264 test in SpecInt2k6 could come from an additional LD pipe..
willis936 - Wednesday, October 16, 2019 - link"To be sure, this isn’t the very time we’ve seen this, as OnePlus, LG and Google have already introduced it in their phones over last year."
eastcoast_pete - Wednesday, October 16, 2019 - linkAndrei, thanks for your review! One question/request, both for this iPhone review, but also future phone reviews:. Please test the call quality and reception (especially in challenging situations with a single bar/ low signal strength)! I found a couple of otherwise very attractive smartphones fall flat on their phone function. Since my mobile is my main phone, which I also need for work, call quality is a non-negotiable. Unfortunately, even your otherwise excellent reviews skip that aspect. So, any words on call quality of the new iPhones? Thanks!