System Performance

Section by Brandon Chester

At the heart of a smartphone lies the SoC. While there's now an increasingly common belief that specifications don't matter, the truth of the matter is that almost all of the software features that users now take for granted in a smartphone have only been made possible by the continued improvements in hardware performance. Modern smartphones with high resolution displays and complex interfaces would not exist if the available CPU and GPU processing power hadn't advanced as much as it has. On top of that, performance is something of a gating factor for software development, as the innovation that happens in software has to happen within the boundaries of what can be done with the hardware. Specifications in a smartphone actually matter quite a lot, even if the user isn't actively aware of all the individual components that make up their smartphone's SoC.

With the iPhone 7 Apple doesn't have the benefit of a new major process node to help improve performance. Any improvements will be the result of architectural improvements, as well as improvements that have been made to TSMC's 16nm process since the release of the Apple A9, which was Apple's first FinFET SoC. It's important to note that one of the goals of A10 Fusion is improving dynamic range, so the focus isn't solely on improving performance. However, device performance seemingly must improve with each generation, and Apple is advertising a 40% improvement in CPU performance and a 50% improvement in GPU performance with A10 Fusion compared to A9.

A10 Fusion's Floorplan (Special thanks to Chipworks)

While an in-depth look at A10 Fusion will have to wait until our seperate technology deep dive, we can still take a look at how performance has changed at a higher level. A10 Fusion's peak frequency is 2.3GHz, up from 1.8GHz on A9. This gives a theoretical improvement of 28% on its own, and the remainder will have to come from improvements to the architecture in Apple's (big) Hurricane cores. Based on our testing so far, Hurricane is not radically different from Twister (A9), but Apple has been making some optimizations. Meanwhile it should be noted that while A10 technically has four CPU cores – the two Hurricane cores and the two smaller cores – this is not a heterogeneous design, and only two cores are active at once. So for the purposes of high performance benchmarking, this means we're benchmarking the big cores nearly exclusively.

Anyhow, we've run our standard suite of benchmarks on the iPhone 7 and 7 Plus to see if A10 Fusion stands up Apple's performance claims.

Basemark OS II 2.0 - System

Basemark OS II 2.0 - Memory

Basemark OS II 2.0 - Graphics

Basemark OS II 2.0 - Web

Basemark OS II 2.0 - Overall

I've brought back BaseMark OS II for this review, although unfortunately the reason has less to do with wanting it back in the benchmark suite and more to do with there being very few cross-platform general system benchmarks nowadays. You can focus on the GPU, or the CPU, but the major system performance benchmark in mobile is PCMark and it remains exclusive to Android. BaseMark OS II is a pretty large optimization target now, and while it does test many aspects of the system it doesn't do so by directly simulating the tasks that a user would perform. Despite that, it helps in getting a holistic view of a device's performance by going beyond tests of a single component, so it's not without its uses.

The iPhone 6s and 6s Plus have remained the fastest overall devices in this test, so it's really just a matter of seeing how much Apple has improved. The latest generation of Android devices outpaced the GPU performance Apple's A9 SoC some time ago, so that's one area where Apple could stand to improve against the competition. As it turns out, they have. The iPhone 7 and 7 Plus show significant gains over their predecessors across the board. Performance in the system sub-test is up by 25-30%, and the graphics score is right in line with Apple's claimed 50% increase in performance. Web shows similar gains to system, which makes sense given that both are relatively CPU-bound.

The memory/storage test also shows a large improvement, which is interesting at first glance given that Apple didn't highlight any improvements in this area. However, this is just the result of the tested units being the 256GB model. On my retail iPhone 7 which is 128GB the memory score is essentially the same as the iPhone 6s which was also tested in a 128GB configuration. This comes down to the fact that these devices use a hybrid SLC/TLC storage solution, where the SLC storage can be used for writes and data can be moved later as needed. It's likely that on the 128GB iPhones the BaseMark OS II memory test is exceeding the size of the SLC cache, leading to lower scores based on the performance of the TLC NAND. On the 256GB iPhone the SLC cache is large enough to fit the entire data set used in the test, leading to a much higher score.

Kraken 1.1 (Chrome/Safari/IE)

WebXPRT 2015 (Chrome/Safari/IE)

JetStream 1.1 (Chrome/Safari)

Web performance is an interesting case. While Apple's continued increases in single core CPU performance have benefited web performance with each generation, the truth is that improvements Mobile Safari's Nitro JavaScript engine have contributed a great deal as well. When looking at the charts you can see that there's a significant gap between the iPhone 7 and the next device, with the exception of WebXPRT where it's tied. What's more important than this gap is what the second device actually is. It's always another Apple device, and the same is true for the third device, and the fourth device, and so on. In Kraken and Jetstream the top seven devices on the chart are Apple's mobile devices, and the charts only have seven Apple devices in total. The Huawei Mate 8 is able to move ahead of the iPhone 6 in WebXPRT, but the gap between it and the iPhone 6s is still enormous. While Apple has continued to advance their web performance, it hasn't been with the immediate goal of beating the Android competition, as the best Android devices are still just trading blows with the iPhone 6.

WebXPRT is a small exception. As far as mobile benchmarks go, it's quite long and has short bursty workloads with pauses between each to simulate how a user would pause between different actions while using the browser. I suspect that in this case we're seeing the effects of A10 switching between its low power and high performance cores, causing parts of the test to be run on the little cores, which are not as performant. In fact, In the case of very quick actions that take only tenths or hundreths of a second to complete, it may not be possible to switch to the high performance cores before the operation has ended, causing the score to reflect the performance of the low power cores instead.

While it might seem reasonable to attribute the superior web performance of iOS devices to Apple's focus on improving single threaded CPU performance, the fact that Android devices with Cortex A72 CPUs are only matching Apple's A8 SoC shows that the gap is not only due to the CPU power available. Chrome's generally poor performance on Android is a significant limiting factor, and you can see in the chart how the improvement in Android device CPU performance over time has not translated into anything close to the sorts of gains that Apple has seen in the same period. While there are customized versions of Chromium like Snapdragon Browser that provide optimizations for a class of SoCs, it doesn't look like the gap between Android devices running Chrome and iOS devices running Mobile Safari is going to close any time soon, and if anything, it's only widening with each year.

Display System Performance Cont'd and NAND Performance


View All Comments

  • TEAMSWITCHER - Monday, October 10, 2016 - link

    I kept my iPhone 5s for three years...and just got a 128GB iPhone 7. It was a HUGE upgrade for me.. I can't wait for the new Air Pods .. cables are so last century. Reply
  • Lolimaster - Monday, October 10, 2016 - link

    So in the name of "fashion" you listen to music/video with sh*tty quality? And willing to pay the overprice for the luxury of listening with again sh*tty quality. Why not buy a 2nd hand 4S, will be the same. Reply
  • dakishimesan - Monday, October 10, 2016 - link

    When did he/she say anything about fashion? Cables are a pain in the ass to keep untangled, often cause you to accidentally pull the headphones off, are the primary source of failure and short life-spans for headphones, etc. In the context of a home, where Ethernet is so much faster and more reliable than WiFi, I'm fine with cables (that don't tend to need to be moved). On the go, by all means, get rid of them. Reply
  • sprockkets - Monday, October 10, 2016 - link

    Yes, but a headphone jack and bt wireless stuff isn't mutually exclusive. It's still a bull shit decision, one that only benefits apple, their OEMs who now have to sell and pay apple for accessories. Reply
  • steven75 - Wednesday, October 12, 2016 - link

    It will be highly amusing when Android phones drop the 3.5 mm just as they one proclaimed Flash support as the best thing ever until it wasn't. So predicable! Reply
  • ex2bot - Wednesday, October 12, 2016 - link

    The <$50 wired earbuds develop shorts after a few months, too. I'm using the wireless Anker Soundbuds Sport. They're okay, though the tips are too flimsy. Sound q is good. Reply
  • lolipopman - Tuesday, October 11, 2016 - link

    You sure feel superior for yourself. I'm guessing you're exactly the kind of person thinks your music preferences are 'better' than others' and even jerk off to classical music.

    I'd rather not listen to the ramblings who still dwells in their mother's basement.
  • ex2bot - Wednesday, October 12, 2016 - link

    Seriously? The adaptor in the box allows for standard wired headphones, and the 7 and 7 Plus are light years better, faster and more capable than the 4S (though I do love the 4/4s design). Reply
  • Mumrik - Monday, October 10, 2016 - link

    " I can't wait for the new Air Pods .. cables are so last century."

    Wonder how you'll be charging your new toys then.
  • steven75 - Wednesday, October 12, 2016 - link

    I agree that wireless charging should be next. We don't need no steenkin cables! Reply

Log in

Don't have an account? Sign up now