Since we first learned that the A9 SoC in Apple’s iPhone 6s lineup is dual sourced - that is that it's being made by two different vendors with two distinct manufacturing processes - one major question has remained in the process of reviewing these two phones. The main issue under question here is whether the TSMC A9 or Samsung A9 have any difference in performance and power consumption. If there is a difference, the question then becomes whether the difference is significant.

In an atypical move for the normally tight lipped manufacturer, Apple issued a statement this afternoon in response to these questions and some rudamentary end-user benchmarking showing that there may be a difference:

With the Apple-designed A9 chip in your iPhone 6s or iPhone 6s Plus, you are getting the most advanced smartphone chip in the world. Every chip we ship meets Apple's highest standards for providing incredible performance and deliver great battery life, regardless of iPhone 6s capacity, color, or model.

Certain manufactured lab tests which run the processors with a continuous heavy workload until the battery depletes are not representative of real-world usage, since they spend an unrealistic amount of time at the highest CPU performance state. It's a misleading way to measure real-world battery life. Our testing and customer data show the actual battery life of the iPhone 6s and iPhone 6s Plus, even taking into account variable component differences, vary within just 2-3% of each other.

It interesting to see this response as Apple normally doesn’t comment on anything like this, which in turn is likely a good indicator of how seriously Apple is taking any concerns. However, this statement is also of interest because it's revealing in terms of what internal data Apple has collected on the issue. Apple has in recent years been one of the better companies in accurately promoting the battery life of their products, and that kind of accuracy comes not only from taking a conservative (safe) stance in marketing, but also collecting massive amounts of data to understand their products and their capabilities.

The Test

To get to the meat of matters then, let's talk about battery life, tests, chips, and statistics. In terms of the testing that has seemingly spurred on this Apple response, it’s likely that the “manufactured lab tests” in the statement refer directly to Primate Labs’ GeekBench battery life benchmark. The GeekBench test runs parts of the GeekBench CPU benchmark in a loop, making sure to do a fixed amount of work per time interval while idling the rest of the time, and using the score result as a modifier for the runtime score. This makes the GeekBench battery life benchmark primarily a SoC/CPU/Memory benchmark, and that in turn has repercussions for interpreting the data.

In the case of our own web browsing battery life test, for example, this is a test that attempts to simulate light reading of web pages, meaning that the SoC is only working hard a fraction of the time. The vast majority of the time the SoC is idling, making the display the biggest power consumer; and this is especially the case on these latest generations of high performance flagship smartphones. A heavy test on the other hand would be a test that keeps a sustained and significant load on the CPU and GPU, which shifts the power consumption of a test from the display to the SoC.

Battery Life: Light vs. Heavy
An Example of the Wide Gulf Between Light and Heavy Battery Life Testing

Due to the nature of its use of fixed size workloads, the GeekBench battery life benchmark lies somewhere in between a heavy load and a light load (Primate Labs states it's around 30% on the 6s). And this is notable because if this is the case, it means that GeekBench is in fact highlighting the difference in power consumption between the TSMC and Samsung A9s. However as Apple points out in their statement, a sustained workload is not necessarily representative of what real world usage is like, with the real world having a burst of of different types of workloads. This doesn't mean GeekBench doesn't return valuable data, however it means we're looking at a slice of a bigger picture. Ultimately if there is a difference between the TSMC and Samsung A9s, then it means that GeekBench is likely to be exacerbating the difference versus what a real world mixed use case test would see.

The Chips

As for the data itself, due to the fact that GeekBench is a heavier workload, it means that there are a number of factors that could explain why battery life in this test shows such a large difference, and not all of these factors are easy to account for. With the chips themselves, it could be that the Samsung A9 variant is simply reaching higher average temperatures due to its smaller die size (same heat over a smaller area), which then accordingly affects power draw due to the nature of semiconductor physics (increased leakage). It doesn’t have to be at the point where the workload is causing thermal throttling, but even a sustained load for significant periods of time could be enough to cause this effect, as the CPU will reach higher temperatures even if the phone is cool to the touch.


An example of the temperature versus power consumption principle on an Intel Core i7-2600K. Image Credit: AT Forums User "Idontcare"

Given the nature of chip manufacturing, it’s also hard to say one way or another whether an individual chip and phone pair will have better or worse battery life than another chip and phone pair. This is a pretty complicated subject, but it basically boils down to the difficulty of injecting exactly a certain number of ions into a small part of the silicon wafer or depositing a layer of insulator that meets an exact thickness. This results in chip manufacturing quality being distribution based - a wafer will come out of production with individual chips of varying quality, with some chips operating at lower voltages or lower leakages than others, and other chips being altogether defective. This is colloquially known as the "silicon lottery."

It's then from these chips that a customer (e.g. Apple) needs to made tradeoffs between how "poor" of a chip they are willing to accept and how many chips per wafer they'd like to be able to use from each wafer (the yield). In doing so, a customer will set minimum tolerances, certain parameters that a chip needs to meet to be qualified. However as these are minimums, it means that a customer will also receive chips that exceed these minimums, as we can see in our completely fictitious chart below.


A completely fictitious processor quality distribution example. Chips in the white area are used, outlying chips in red areas are rejected

In the case of someone like Intel, they will bin these passing chips as different products (Core i3/i5/i7) and different clockspeeds to charge the most for the best chips. However since Apple currently only uses at most two bins of chips (those suitable for 6s and those suitable for 6s Plus), this means there is a wider variation in the chips used in each phone. As a result, even if there isn't a true and consistent difference between TSMC and Samsung for A9 SoCs, you could easily have a pair of phones where due to the silicon lottery there is a notable - though not extreme - difference in power consumption.

This variation is an expected part of chip manufacturing, and while Apple could disqualify more chips and thereby reduce the yield, they will always have a certain degree of variation. The key here is to set rigorous minimums, advertise a phone based on those minimums (e.g. battery life), and should a customer end up with a phone, then they have won the silicon lottery in this case.

The Statistics

So how does one compare phones when there's a natural variance in chip quality? Ideally it would be done just like Apple does, testing a large number of phones (chips) for power consumption and battery life to determine the distribution. Otherwise if we only test one TSMC A9 and one Samsung A9, we don't know where in the distribution each A9 lies, and consequently whether each phone is a representative "average" sample or not.

Of course this is easier said than done, as the greater the accuracy desired the greater the number of iPhones required, a bill that at $650/$750 a unit adds up quickly. This makes it incredibly impractical for any one group short of a large corporate competitive analysis team to get enough samples, especially since at the press level Apple only distributes one phone of each type (for a total of 2) to each press organization. In lieu of that, typically the best one can do is look at a small number of samples, which offers some data to account for variance, but not much.

This brings us back to where we started with GeekBench. As part of the GeekBench benchmark, results are uploaded to Primate Labs' servers, where they are available for browsing. I've had a couple of people ask whether these results are a collective whole - in essence crowdsourced benchmarking - can answer anything, and the answer to that is a "yes, but" kind of scenario.

The big problem right now is that GeekBench doesn't know what model processor is being tested, so there's no way to sort out TSMC versus Samsung. Update: And as if right on cue, the same day we publish this GeekBench 3.4 makes it through the iOS app store approval process, adding the ability for GeekBench to tell which processor is in a phone.

Even if there was, we'd get to matters of testing rigor. How bright is the screen on each phone? Are there any background tasks running? Is it in airplane mode or spending power talking on WiFi/cellular? With Geekbench running at on average just a 30% duty cycle, these are all potential power consumers that can significantly impact the resulting battery life. In turn, these are all things that are accounted for in formal testing, but they cannot reliably be accounted for in benchmarks run by the wider public. This as a result adds even more variance to the equation, which makes individual or even small groups of results potentially very inaccurate.

The Conclusion

Wrapping things up then, where do we stand? The short answer is that all we know is that we don't know. What we know is that there isn't enough information currently out there to accurately determine whether the TSMC or Samsung A9 SoC has better power consumption, and more importantly just how large any difference might be. 1-on-1 comparisons under controlled conditions can provide us with some insight in to how the TSMC and Samsung A9s compare, but due to the natural variation in chip quality, it's possible to end up testing two atypical phones and never know it.

To that end I suspect that Apple's statement is not all that far off. They are of course one of the few parties able to actually analyze a large number of phones, and perhaps more to the point, having a wide variation in battery life on phones - even if every phone meets the minimum specifications - is not a great thing for Apple. It can cause buyers to start hunting down phones with "golden" A9s, and make other buyers feel like they've been swindled by not receiving an A9 with as low the power consumption as someone else. To be clear there will always be some variance and this is normal and expected, but if Apple has done their homework they should have it well understood and reasonably narrow. The big risk to Apple is that dual sourcing A9s in this fashion makes that task all the harder, which is one of the reasons why SoCs are rarely dual sourced.

As for AnandTech, we'll continue digging into the matter. Unfortunately all of the iPhones we've received and purchased so far have used TSMC A9s - it's a silicon lottery, after all - but whether there is a real and consistent difference between the TSMC and Samsung A9s is a very interesting question and one we're still looking to ultimately be able to address.

POST A COMMENT

147 Comments

View All Comments

  • ddriver - Saturday, October 10, 2015 - link

    Well, even thou there have been cases of people selling or trying to sell organs to get apple toys, I'd say even below average people in the developed world can afford to buy an iphone every other year or so.

    I am from eastern Europe, where wages are like 10 times lower, and yet there are many dum-dums who buy apple products. Many even go for credit, ending up paying even more. And hey, believe it or not, but hardware prices here are marginally higher here...

    And hey, just because some people are into say graphics design or video editing doesn't mean they are smart enough to know what's best for them. I know many people in content creation who buy apple products, because they genuinely believe the marketing hype that apple products are better for suck workloads. They could easily get a 100% better performance for the same price, which really does matter in most aspects of content creation, but somehow having a custom built system is just not "artsy fartsy" enough, it is better to get a pretty looking imac. I mean, even if you are a graphics designer or a video editor, which may seem like not-so-dum-dum thing to be working with, paying premium price for an inferior system because of your vanity is quite dum-dum indeed.
    Reply
  • jospoortvliet - Saturday, October 10, 2015 - link

    I think many of you underestimate the trick Apple plays: deliver a consistent level of quality and only cater to the high end. You can make products expensive as much as you want, just don't sell bad ones. Every other brand will happily sell customers cheap crap if they can make a proffit on it. remember the 'laptops' with a Brazos CPU? Over 75% of those must have been a disappointment, getting replaced within the year if money allowed... Apple just doesn't play that game. Yes, there is advertising and yes, I will always refuse to buy their products because of the walled garden (Linux user and advocate here) but be honest: they just do NOT sell mediocre products. They make great products and ask an insane amount of money for them. Sure not great value for your $ but that is indeed the one thing they will never claim to give you...

    It might not reach every niche, have exactly what YOU want (i will not buy a phone without front facing speakers, for example) but fact is that you will rarely if ever have to feel bad about your Apple product choice.

    Add in some cognitive dissonance about the insane price you pay for their stuff (it better be good and if it isn't you tell yourself it is) and we have a great and extremely profitable business model.

    It amazes me nobody has really tried to copy it - it really isn't that complicated.
    Reply
  • ddriver - Saturday, October 10, 2015 - link

    Apple has plenty of products that are NOT high end. Reply
  • ex2bot - Saturday, October 10, 2015 - link

    Jospoortvliet, I think you won this round of the comments because you've hit the nail on the head.

    Plenty of companies *have* attempted to follow the Apple model or whatever you want to call it, with varying levels of success. That expensive Chromebook (Pixel, IIRC); that was a head scratcher for me. Microsoft's Surface line. HP and other PC manufacturers have had premium notebooks (and I'm sure they still do.

    I use Apple stuff. I prefer my MacBook Pro. Best trackpad, best construction, best support. I paid extra for an iPhone. Being an experienced tech user, I know the competition. I've owned a few Dells, Palm, Moto. I have the Google. Still prefer Apple.

    I suppose I buy them to prove I'm better than the rest of you, though I'm a tech novice and don't know there are better products out there. Besides, it makes shopping easier; all I have to do is look for the big Apple, right? ;) /s

    Oh, no! Walled garden. Where did I put my ladder?
    Reply
  • ddriver - Sunday, October 11, 2015 - link

    Samsung have ruined the note line in ill conceived attempts to lure in the dum dums. And it actually worked, but ruined a great product.

    I am a power user. I don't use my phone to take selfies, I rarely use it to make calls too. I bought a galaxy note 3, because it was vastly superior to iphones at the time in every productive centered way. I run my own custom software, I run an actual C++ compiler on the phone, I deploy my own custom binaries. I made my own "smart cover", hosting a microcontroller I can program with the phone on the go, and a number of analog and digital IO, allowing such great things as data acquisition and analysis, rapid prototyping and interfacing with different devices. You cannot even hope to get that amount of productivity on an iphone. Also

    - 3 gigs of ram - to most people it doesn't matter, to me it does, I actually use it
    - quad core - to most people it is just a number, but I use them, getting better overall performance despite of the lower single core compared to a8
    - large screen, which is a huge plus for the software I use, which none of the iphones had at the time, because it was "stupid" according to apple zealots who now line up to buy large screen iphones
    - pressure sensitive pen - great for taking notes quickly, annotating stuff, drawing sketches, charts and diagrams
    - a OS which is not a walled garden and allowed me to run my own software, not just limited to installing what apple deems fit
    - removable battery - I bought 3 right away, can use the phone for days without charging, can use the phone for years after the original battery gives out
    - SD card slot - had several in the phone back cover, giving me access to tons of data and the ability to easily change those data sets, without having to connect to a computer, delete the old data and upload new data

    I'd say when it comes to phone use (and not just that), I am as "poweruser" as it gets, I make more use of a phone than most people make of desktop systems. And yet, the "impeccable" apple phone products are vastly inferior in practically every aspect and any way you look at it. And sure, you could use a ladder in apple's walled garden, but then you will simply void your warranty. And even then, it won't be on par.

    And LOL, gotta love the distorted view of apple fanboys. Microsoft copied apple with the surface? LOL, it is actually apple which is currently attempting to copy the surface, and the result will still be a useless toy compared to it. It sure did take M$ a long time to come with a tablet that was actually useful to power users, but apple sure is still struggling to do that, they suck even following in others' footsteps.
    Reply
  • HammerStrike - Tuesday, October 13, 2015 - link

    I don't think their products are ridiculously overpriced - most comparable hardware in the Android space (or Windows for laptops/AIO) price out in the same neighborhood. Also, 2/3 of their profit comes from iPhone, and given the insane amount of time that people spend on their phone daily, the extra 20 or 30 cents a day, over the lifetime of the phone, that the iPhone will cost vs a lower cost android is a worthwhile trade off for most people.

    Anecedotally, I've owned two Android high end phones over the last 4 years - a Samsung Galaxy 2 and a Samsung Note 3. I liked both of them a lot in the start, but both have had their camera's fail on me out of warranty and, after a few updates, both have cropped up glitches, crashes, random reboots and periods of super high battery usage that fluctuate in and out for no apparent reason. In a nut shell, they start out great, but by the end of year two (where I currently am with my Note 3) they both have aged very poorly.

    In contrast, I've had an iPad 3 since it came out, and my wife as had a couple of iPhones (4, 5c). All of those devices have run flawlessly, and while progressive iOS releases may not run as quickly on older hardware as newer, we have never experienced any "slow degradation" issues surrounding the performance of the devices. I'll happily pay an extra couple hundred dollars every few years for a device I use multiple times an hour, every day, for that type of reliability.
    Reply
  • beggerking@yahoo.com - Thursday, October 15, 2015 - link

    Apple is not "high-end". It's trying and some people believes it, but in actuality it is never high-end.It's always 1 step behind in everything. Reply
  • artificialintel - Sunday, October 11, 2015 - link

    I'm talking about developers. I work as a developer in San Francisco and Silicon Valley, and basically every developer's laptop is a Mac, though most are also running Linux boxes via VirtualBox or in the cloud. Well over half these people carry iPhones. The people I see with Windows laptops are almost never engineers; mostly they're sales or marketing. Reply
  • ddriver - Sunday, October 11, 2015 - link

    Really, engineers almost never use windows hardware?

    Last time I checked, the only bit of engineering software that was running on mac was plain old AutoCad. And in engineering, AutoCad is like MS Paint in graphics design...

    None of the serious engineering software even runs on a Mac. Not Catia, not SolidWorks, not Inventor, not Creo. I guess engineers simply love not being able to run any engineering software on their machines.

    Programmers prefer apple? This doesn't seem like this is true either, at least according to programmers:
    http://programmers.stackexchange.com/a/65381/19289...
    Plenty of good reasons to NOT use a Mac for programming. The only exception is if you target apple devices. Then you don't have a choice, thanks to good old apple's politics.

    You obviously live in a place and keep acquaintances with people who care little about productivity and efficiency and prefer to waste money on product fads instead. With apple you simply pay more for less, unless your N1 priority is the apple fad.

    Oddly enough, the best thing you can do in order to boost your productivity on a mac is... to install windows on it LOL. Get rid of the apple OS and it gets a tad better, but is still really poor value for the money.
    Reply
  • artificialintel - Monday, October 12, 2015 - link

    Dude, I was talking about developers, AKA software engineers. My department is called engineering and the developers are generally called engineers. I was not suddenly switching topics from software development to mechanical engineering.

    Anyway, I'm not sure what "Windows hardware" would be. Plenty of people use Linux boxes, of course, though the big barrier I've seen to getting non-Apple hardware in most shops is that some devs really don't want to deal with the management overhead of Linux (drivers, filesystem mounting, AD support, etc). I mostly use my work Mac as a BSD box with Homebrew package management, though I'm gradually warming up to the Mac app Store. When I was doing docker development support for boot2docker was pretty far ahead of where it was on Windows, but with Docker Toolbox things seem to have reached a level playing field. In any case, I have never even seen a Silicon Valley software developer who develops primarily on Windows, though I know they exist especially amongst the companies founded in the mid-90s. Mac and Linux is essentially all I see. I definitely don't see people with Windows phones very often, and iPhones are at least as common as Macs. Does that mean that Mac/iPhone is better? Of course not. It does make assertions that Macs and iPhones are just for non-technical dum-dums sound uninformed and silly.
    Reply

Log in

Don't have an account? Sign up now