The Qualcomm Snapdragon 835 Performance Preview
by Matt Humrick & Ryan Smith on March 22, 2017 4:30 AM EST- Posted in
- Smartphones
- Snapdragon
- Qualcomm
- Mobile
- SoCs
- Snapdragon 835
- Kryo
Qualcomm's Thoughts on Benchmarks versus End-User Experiences
While the primary purpose of our visit out to Qualcomm’s offices was for benchmarking, for this year’s event Qualcomm went one step further and invited us to take a tour of some of their labs. The tour itself was plenty interesting (more on this in a bit), however arguably the more important aspect of the tour isn’t what we saw, but why Qualcomm is giving the press lab tours after so long. A tour may sound trite, but for Qualcomm it’s a sign of bigger things.
Perhaps the most important thing to understand about Qualcomm is that while the bulk of the company’s revenue comes from chips, the majority of their actual profits come from various technology and patent licensing programs. The actual breakdown between the two halves of Qualcomm gets a bit weird when trying to best account for which half foots the bill for R&D, but at the end of the day it’s Qualcomm’s licensing business that is making them the most money.
| Qualcomm Financials (FY'2016) | ||||
| Chip-Making | Licensing | |||
| Revenue | $15.4B | $7.6B | ||
| Profit (Pretax) | $1.8B | $6.5B | ||
As a result, if boardroom-level rumors are to be believed, the company is never quite sure how to balance the two businesses. Some groups want licensing and chip design separated, and others want them to stay combined. Qualcomm seems content to stick with the status quo and their $5.7B in profits for 2016, but as a consequence of this unusual split it means that the chip side of Qualcomm is always under a bit of pressure to prove itself.
The good news for the chip business is that it’s doing fairly well, especially in North America. Most high-end Android smartphones sold in the US are based on Qualcomm’s chips, Qualcomm’s modems are in some (but no longer all) of Apple’s iPhone SKUs, and even in the midrange market Qualcomm’s Snapdragon 600 series chips are in a number of popular phones. The flip side of this is that outside of the US and especially outside of the high-end, Qualcomm’s chip business is under constant pressure from both their partners and their competitors. Samsung is happy to stick with their own home-grown Exynos chips outside of North America, and other chip vendors like MediaTek nip at Qualcomm’s heels with competent midrange SoCs that are priced very aggressively. So while Qualcomm’s chip business is doing well right now, that could very well change in the future.
But what does this have to do with a press tour of their labs? One of the big initiatives for Qualcomm, as exemplified in last week’s Snapdragon branding tweak, is that the chip arm of the company wants to get away from being so frequently judged on the basis of their CPU and only their CPU. This is why Snapdragon is now a platform versus a processor, and similarly, it’s why we’re going to see Qualcomm increasingly talking up the other aspects of their SoCs in the future. Their SoCs and associated front-end hardware are more than just the Kryo CPU, and they want the world to know that.
For Qualcomm this move makes a lot of sense. CPU performance is still growing year-after-year, but as we’ve seen, those gains are plateauing some as a consequence of diminishing IPC gains and diminishing clockspeed gains. Meanwhile, although the Kryo is a customized CPU design, as we’ve seen in our benchmark data it doesn’t behave radically different from ARM’s Cortex-A72 and Cortex-A73 designs that other vendors can license. Short of a “Cyclone” moment for Qualcomm, their CPU performance is close enough to the rest of the Android pack that they can’t clearly stand out from other vendors in the way Apple can. And that means they need to promote and sell their SoCs as the sum of their parts, and not just a CPU with a bunch of extra stuff bolted on.
This also means that even though Qualcomm is one of the first vendors to ship a 10nm high-end SoC – and enjoying the performance benefits thereof – we’re also going to see the company downplay benchmarks a bit for not capturing the complete “end-user experience.”
And to the company’s credit, they aren’t wrong. CPU, GPU, and memory performance are all easy enough to test, vary a great deal among SoC designs, and are similarly easy to communicate. DSPs, ISPs, and other aspects are not nearly as easy to test, never mind communicating those advantages to a wider audience.
Still, once we get outside of CPU/GPU/memory and into other functional blocks of a SoC, things get murkier for everyone. A video decode block may as well be a static piece of hardware – as long as it does its job well, no one will notice – and a great Wi-Fi radio can be kneecapped by a bad device design. So as much as Qualcomm wants to push the broader end-user experience, they will never completely escape the fact that the core features of their SoCs will be the first to be put to the test, and that wider testing is going to be less meaningful to customers, or more practically speaking outside the budgets of technology journalists.
Facebook
Twitter
RSS
128 Comments
View All Comments
zeeBomb - Wednesday, March 22, 2017 - link
Its that time of year again!name99 - Wednesday, March 22, 2017 - link
"The [3DMark] Physics test runs on the CPU and is heavily influenced by how well an SoC’s memory controllers handle random access patterns. "No it isn't, at least not to an extent that matters in any modern CPU. Why do you keep posting this rubbish in review after review?
The source code is available for examination. It basically tests (frequency)*(number of cores) and is useless for learning anything beyond that. That's why it's always the only test in which Apple looks bad --- because Apple's running two cores as opposed to 4/6/8/10 on Android, and, at least in the past, those cores were under-clocked relative to the Android cores.
If people want to post the 3DMark Physics numbers, whatever, I don't care. But I do think doing so is a waste of reviewers' and readers' time --- there is simply no useful additional information provided by that benchmark.
The fact that 3DMark continues to push it (as opposed to the way GeekBench every year or two tries to respond to complaints and concerns about its benchmarks) tell you something about the relative professionalism of the two companies.
Matt Humrick - Wednesday, March 22, 2017 - link
"It basically tests (frequency)*(number of cores)"Both of these are factors, but it's not the whole story according to the developer I spoke with at Futuremark. If you have additional information to prove your claim, please share it with me via email. My mind is always open :)
name99 - Thursday, March 23, 2017 - link
I looked into this in detail years ago when there was a big kerfuffle about the iPhone 5S score.I'm not interested in spending another day doing the exact same thing. I'll just point out that what I am saying matches the data.
Sure, I'm not saying that THE ONLY THING is (frequency)*(number of cores), there's some small 5 to 10% variation around that; but that variation is unimportant --- the big picture is embedded in what I said.
Now, does this mean it's a good benchmark? Well, how much code that people care about is multi-threaded (on Android and otherwise)?
I'm not interested in relitigating that (given what I consider to be the astonishing incompetence and ignorance we saw on Anandtech the last time this was discussed, with a VAST proportion of readers apparently unaware of such concepts as timesharing, or how to accurately calculate the thread level parallelism of an executing piece of code).
I will say that the most recent academic papers I've read, dated 2016, referring to work in around 2014, show that it's higher than you might expect, not as high as you might hope. Across a very wide range of Android apps the thread level parallelism is slightly larger than 2, showing, basically (in my interpretation)
- an Android controlled thread doing misc stuff that's pretty busy
- a main app thread
- various small completion routines, async routines, and interrupts
So basically two cores get you almost all the value in real world core, a third core occasionally picks up a small amount of extra available work.
Now read what I am saying before getting upset. I'm NOT saying that ARM is stupid to ship 4 (performance) cores. ARM cores are tiny, they can be of (very occasional) value to a few talented developers today, and the only way we'll EVER get the mass market to code more parallel is to have the hardware out there as the default. So I'm happy that ARM is flooding the world with hexacore, octacore, decacore chips. (And I think Apple is being penny-wise and pound-foolish by not making every SoC they ship a triple core ala A8X --- the extra area would be small, and it would likewise provide an incentive for developers to get off their asses.)
But that's a different argument from whether core-count provides "visible performance" today.
I think the answer to that is clearly no. The first thing that matters to most users is snappiness (which depends, primarily, on flash performance, GPU [and the quality of the graphics code], performance governor (so does the CPU "start off" fast or "start off" slow and only get fast after .2 seconds of UI interaction? Then there are a few places where overall "endurance" performance matters (like much browser stuff, or viewing complicated PDFs --- both of which are very poorly threaded even as of 2017). Finally the cases where all cores all the time matters, and almost nothing else (the sort of thing 3DMark Physics is testing) are REALLY few and far between.
Or to put it another way. Most CPU microarchitecture improvement since 2000 has been about discovering and exploiting the stochastic structure of REAL-WORLD computation. There are re-uses and patterns in branching, in memory access, in instruction execution that are exploited ever more aggressively in branch prediction, in cache insertion and liveness tracking, in prefetching, in loop buffers, etc. A benchmark that prides itself on randomness and in providing no way for all those smarts to add value is saying SOMETHING about the worst case performance of a CPU, but it's not clear that that something is of any value to almost everyone.
Frenetic Pony - Wednesday, March 22, 2017 - link
How disappointing that yet another of the very few custom CPU designers is now gone. Looking at the general performance of the CPU now, I see no reason whatsoever to choose Qualcomm over some other, generic ARM hawker that's probably cheaper. They could at least stop pretending and just become a module seller, selling their GPU/modems/etc. separately as there doesn't seem to be any reason to choose a Qualcomm SOC as a whole.Other than ditching their stock (if you haven't already) none of this looks good for Qualcomm. Or for ARM for that matter, the A73 doesn't offer any performance boost over the A72 and is still trounced by Apple. Maybe the rumors of Google building its own CPU will come true and we'll see it in the Pixel 2.
serendip - Wednesday, March 22, 2017 - link
No reason? They're one of the few developer and open source friendly chip manufacturers around, even if that relationship ventures into frenemy territory once in a while. Qualcomm modems and imaging blocks are pretty good too.Intel are developer friendly but their GPUs can be an abomination to work with. They've also effectively abandoned the mobile space. Mediatek, Huawei and Samsung either give a cold shoulder or a middle finger to devs.
StrangerGuy - Thursday, March 23, 2017 - link
They dropped their own custom cores because why even bother when vanilla ARM does a better job while being cheaper...Plus the economics for a non-Apple, non-Samsung bleeding edge SoC no longer makes much sense, and 99.99% of the population buying these phones doesn't and won't give a shit to the SoC or the benchmarks.Meteor2 - Thursday, March 23, 2017 - link
It wouldn't surprise me if Qualcomm had multiple core design teams competing with each other. We've seen ARM cores come before; maybe full-custom will come back next year.SyukriLajin - Thursday, March 30, 2017 - link
I think they are just shifting their focus. The fact that they "rebranded" the snapdragon as a platform instead of just processor is one indicator. SOC is more than just cpu cores and they want people to know that. My guess is, they think that investing more money to develop a more unique platform is more important than spending valuable time and money on redoing the cpu core, ARM already invested a tons of money to develop it, no reason to reinvent the wheel when you can use the resources to provide a platform that will help them be more different then the others. I think it's wise for them. The resources would better off be spent to create a better DSP, modem, GPU etc, which will give them more return than a custom cpu core.MrPhilo - Wednesday, March 22, 2017 - link
So the Exynos 8895 GPU should in theory be faster than the 540 by quite a bit? Since the Huawei 9 uses 8, whereas Exynos uses 20, but of course with a lower clockspeed. I can see it being at least 20-30% faster than the 540.