The Reality of Silicon And Market Pressure

Section By Andrei Frumusanu

In a sense, the Kirin 960 and Kirin 970 have been a welcome addition to our mobile testing suite. As a result of having devices powered by the two chipsets, we have switched over to a new testing methodology where we now always publish peak and sustained performance figures alongside each other. Without the behavior of these devices, we might never have changed our methods to catch these shenanigans.

But if we’re to go back to a paragraph in the Kirin 970 SoC piece:

Indeed, the Kirin 960 and 970’s vast discrepancies between peak performance and their inability to sustain those performance was one of the key reasons why for this year I opted change our mobile GPU performance testing methodology. All reviews this year were published with peak and sustained performance figures alongside each other, trying to unveil some of the more negative aspects of sustained performance among some of today’s smartphones.

The behaviour of this year’s Kirin 970 devices is, in a sense, not surprising. Huawei & Honor's power throttling adjustments are a great positive for the actual user-experience as they solve one of the key issues I had brought up about the chips in the review: they limit phone power consumption to reasonable levels, rather than burning through power and battery capacity like crazy. This new behavior on power throttling is essentially an aftershock to the Kirin 960’s awful GPU power characteristics. Somebody smart at Huawei decided that the high power draw was indeed not good, and they introduced a new strict throttling mechanism to keep temperatures in check.

This means that when we look at the efficiency table, it makes a lot of sense. Both chips showcase instantaneous power draws way above the sustainable levels for their form-factors, which the throttling mechanism keeps in check.

Competing Against Cheaters: Two Options

While I fully support Huawei in introducing the new throttling mechanisms, the big faux-pas here was in terms of them excluding benchmark applications via a whitelist. During the Kirin 950 days when we talked to HiSilicon’s managers, we discussed GPU power as an important topic even back then. Those generation chipsets had substantially lower GPU performance compared to the competition, however the GPU power was always within the sustainable thermal envelope of the phones – around 3.5W.

Now, when we look at total system power, we see that Huawei has made improvements:

GFXBench Manhattan 3.1 Offscreen Power Efficiency
(System Active Power)
AnandTech Mfc. Process FPS Avg. Power
(W)
Perf/W
Efficiency
Galaxy S9+ (Snapdragon 845) 10LPP 61.16 5.01 11.99 fps/W
Galaxy S9 (Exynos 9810) 10LPP 46.04 4.08 11.28 fps/W
Galaxy S8 (Snapdragon 835) 10LPE 38.90 3.79 10.26 fps/W
LeEco Le Pro3 (Snapdragon 821) 14LPP 33.04 4.18 7.90 fps/W
Galaxy S7 (Snapdragon 820) 14LPP 30.98 3.98 7.78 fps/W
Huawei Mate 10 (Kirin 970) 10FF 37.66 6.33 5.94 fps/W
Galaxy S8 (Exynos 8895) 10LPE 42.49 7.35 5.78 fps/W
Galaxy S7 (Exynos 8890) 14LPP 29.41 5.95 4.94 fps/W
Meizu PRO 5 (Exynos 7420) 14LPE 14.45 3.47 4.16 fps/W
Nexus 6P (Snapdragon 810 v2.1) 20Soc 21.94 5.44 4.03 fps/W
Huawei Mate 8 (Kirin 950) 16FF+ 10.37 2.75 3.77 fps/W
Huawei Mate 9 (Kirin 960) 16FFC 32.49 8.63 3.77 fps/W
Huawei P9 (Kirin 955) 16FF+ 10.59 2.98 3.55 fps/W

The Kirin 960’s GPU power and inefficiency was a direct response to market pressure, as well as negative user feedback regarding GPU performance. I don’t really blame Huawei; I highly praised the Mate 8 with its Kirin 950, irrespective of the lower GPU performance, it was an excellent device because the thermals and sustained performance were outstanding. Despite this, the very first comment of that review was a 'despite the GPU …'. Here the average user will just look at the benchmarks and see it’s ranked lower, and not think any better. It also shows that companies do care what users want, and do listen to requests, but might react in a way users were not expecting.

Unfortunately the only way we can avoid this situation of a perceived performance deficit as a whole is if we as journalists, and companies like Huawei, educate users better. It also helps if device vendors have a more steadfast philosophy about remaining within reasonable power budgets.

Huawei and Its Future

Last Friday Huawei’s CEO announced the new Kirin 980, which is set to be the centerpiece in the Mate 20 lineup coming soon. The big messaging for this new chip is that it is on a new 7nm manufacturing node, and the biggest improvements have been on the GPU side. Huawei has promised power efficiency increases of a staggering 178%. If the math checks out and Kirin 980 devices indeed deliver these figures, then it would mean the company would finally get back to sustainable ~3.5W for GPU workloads, and simultaneously be competitive to some degree.

I’ve already seen a lot of users dismiss the GPU performance of the new SoC. It seemingly, as admitted by Huawei, doesn’t beat the peak performance of the Snapdragon 845, the Qualcomm flagship announced last year. Yet this doesn’t matter, because the efficiency should be better for the new SoC. Because of this, real world sustained performance would be better as well, even if the peak figures don’t quite compete.

Here the only thing I can do is reiterate the balance between performance and efficiency as much as I can, in the hope to shift more people away from the narrative of only looking at peak performance. I’m quite happy with our new GPU testing methodology, because frankly it works – our sustained performance numbers were mostly unaffected by the cheating behaviour. Here I see the sustained scores as a good showcase of performance and efficiency across all devices.

The Honor Play: A Gaming Phone, or Just More Marketing?

Returning to square one, one of the reasons we’ve been analysing Huawei & Honor's phones in this level of detail again is because we've been trying to determine what exactly GPU Turbo is. We've addressed that technology in a separate article, and find that it does have technical merit. Here Huawei tried to compensate for its hardware disadvantages by innovating through software. However, software can only do so much, and Huawei tries to exaggerate the benefits of the new technology on devices like the Honor Play.

Unfortunately I see the reasons for the overzealous marketing of GPU Turbo, and the cheating behaviour of this article, as one and the same: the current SoCs are far behind in graphics performance and efficiency. The reality of things is that currently Qualcomm’s GPU architecture has a major advantage in terms of efficiency, which allows it to reach far higher performance figures.

So Honor is trying to promote the Honor Play as a gaming-centric phone, making bold marketing claims about its performance and experience. This is a quite courageous marketing strategy given the fact that the SoC powering the phone is currently the worst of its generation when it comes to gaming. Here the competition just has a major power efficiency advantage, and there is no way around that.

We actively discourage such marketing strategies as it just tries to pull the wool over user’s eyes. While the Honor Play is a quite good phone in itself, a gaming phone it is not. Here we just hope that in the future we’ll see more responsible and honest marketing, as this summer’s materials were rather, incredible, in the worst sense of the word.

Getting the Real Data: Kirin 970 GPU Performance Overview
POST A COMMENT

84 Comments

View All Comments

  • Amandtec - Wednesday, September 5, 2018 - link

    Actually, social science style studies reveal that (ballpark) 10% of people always cheat, 10% never cheat no matter what, and the 80% in the middle are a continum who are mostly swayed by perceived risks and incentives. If you are always mistrusting everyone it prevents one from building profitable relations. Reply
  • skoondi - Sunday, September 9, 2018 - link

    Not sure about Chinese culture but you'll find more than a few people care about phone GPU performance, I can only wonder why you are on a site basically dedicated to people who are interested in such things Phones are the new computer, with a decent docking set up many people probably don't need a desktop or laptop. I've been gaming since the days of the first civilisation game and currently do a lot of gaming on my phone. Reply
  • techconc - Monday, September 10, 2018 - link

    @shogun18 - You said:
    "But I also have to wonder, are there really more than a 100 basement-dwelling morons who give a flying F about how fast a GPU is on a stupid phone? And furthermore play games on such pointless platforms where frame rates or triangles/sec matters?"

    It seems your comments are very much out of touch with what is happening today, even in the world of gaming.
    Reply
  • PeachNCream - Tuesday, September 4, 2018 - link

    I don't get the point in cheating on phone benchmarks or even the point of phone benchmarks in general. I also don't get the point of sticking an auto playing video about buying the right CPU on EVERY DAMNED PAGE of this article. The same video. Every time! If seeing it on the first page pissed me off and I didn't watch it there (while also being enraged at the auto starting nature of it) what on Earth would make you think it's a good idea to do it to me three more times in the same article? Did the Purch buyout somehow involve a lobotomy that removed logic and reason or was that SOP already in place beforehand? Reply
  • vgray35@hotmail.com - Tuesday, September 4, 2018 - link

    I would use stronger words than that to really convey the egregious nature of repetitive onslaughts of their bloody (not a swear word) videos against us. Their blatant "up yours, screw you" attitude turns rage to a frothing mouth frenzy ready to spill blood (especially on Toms Hardware). What the flying F is wrong with allowing users to decide whether to hit the play button or not? Or am I really only talking to a lobotomized miscreant that should be restrained from interacting with the public. Cease and desist auto play videos! A "up yours, screw you" attitude is not a smart way to deal with the public. Toms are you listening, or are sending us all another finger image? Reply
  • shogun18 - Tuesday, September 4, 2018 - link

    er it's 2018. How the hell have you been using the internet for the last decade without all browser plugins disabled or at the very least noScript (or similar) with or without AdBlock (or similar)? Reply
  • vgray35@hotmail.com - Tuesday, September 4, 2018 - link

    You missed the point. Most of us are clueless. Reply
  • shogun18 - Tuesday, September 4, 2018 - link

    No, I totally get your point and I agree (with) you have every right to be incensed with auto-play videos. However a random comment here is unlikely to get the author or site-admin's attention. And said person is doubtless unable to oppose corporate policy. When Anand's site readership takes a major nose dive (further?) and the morons in marketing (but I repeat myself) can be beaten over the head that one of the major reasons they are losing revenue and clicks is because of auto-play, THEN and ONLY then will the practice stop.

    There haven't been more than a minuscule handful (again, basically zero given the side of the ecosystem) that could be described as "responsible" or "good" website operators for well over 10 years. Therefore if you haven't been black-listing every feature in your browser and toggling the choices conveniently given to you, then your ire is misplaced IMO. I am an equal opportunity banner of site content. Join the club and be amazed at far more enjoyable the experience is.
    Reply
  • vgray35@hotmail.com - Tuesday, September 4, 2018 - link

    Put that way you may be right. Reply
  • 1_rick - Tuesday, September 4, 2018 - link

    Well, Purch has been pushing those annoying ads on Tom's Hardware for quite some time now, so I guess it's only to be expected they'd be forcing them on this site as well.

    TBH I hadn't even noticed the ads because I used my hosts file to block assets.purch.com, because my adblocker wasn't taking care of it. You could try that too.
    Reply

Log in

Don't have an account? Sign up now