The Qualcomm Snapdragon 835 Performance Preview

Qualcomm's Thoughts on Benchmarks versus End-User Experiences

While the primary purpose of our visit out to Qualcomm’s offices was for benchmarking, for this year’s event Qualcomm went one step further and invited us to take a tour of some of their labs. The tour itself was plenty interesting (more on this in a bit), however arguably the more important aspect of the tour isn’t what we saw, but why Qualcomm is giving the press lab tours after so long. A tour may sound trite, but for Qualcomm it’s a sign of bigger things.

Perhaps the most important thing to understand about Qualcomm is that while the bulk of the company’s revenue comes from chips, the majority of their actual profits come from various technology and patent licensing programs. The actual breakdown between the two halves of Qualcomm gets a bit weird when trying to best account for which half foots the bill for R&D, but at the end of the day it’s Qualcomm’s licensing business that is making them the most money.

As a result, if boardroom-level rumors are to be believed, the company is never quite sure how to balance the two businesses. Some groups want licensing and chip design separated, and others want them to stay combined. Qualcomm seems content to stick with the status quo and their $5.7B in profits for 2016, but as a consequence of this unusual split it means that the chip side of Qualcomm is always under a bit of pressure to prove itself.

The good news for the chip business is that it’s doing fairly well, especially in North America. Most high-end Android smartphones sold in the US are based on Qualcomm’s chips, Qualcomm’s modems are in some (but no longer all) of Apple’s iPhone SKUs, and even in the midrange market Qualcomm’s Snapdragon 600 series chips are in a number of popular phones. The flip side of this is that outside of the US and especially outside of the high-end, Qualcomm’s chip business is under constant pressure from both their partners and their competitors. Samsung is happy to stick with their own home-grown Exynos chips outside of North America, and other chip vendors like MediaTek nip at Qualcomm’s heels with competent midrange SoCs that are priced very aggressively. So while Qualcomm’s chip business is doing well right now, that could very well change in the future.

But what does this have to do with a press tour of their labs? One of the big initiatives for Qualcomm, as exemplified in last week’s Snapdragon branding tweak, is that the chip arm of the company wants to get away from being so frequently judged on the basis of their CPU and only their CPU. This is why Snapdragon is now a platform versus a processor, and similarly, it’s why we’re going to see Qualcomm increasingly talking up the other aspects of their SoCs in the future. Their SoCs and associated front-end hardware are more than just the Kryo CPU, and they want the world to know that.

For Qualcomm this move makes a lot of sense. CPU performance is still growing year-after-year, but as we’ve seen, those gains are plateauing some as a consequence of diminishing IPC gains and diminishing clockspeed gains. Meanwhile, although the Kryo is a customized CPU design, as we’ve seen in our benchmark data it doesn’t behave radically different from ARM’s Cortex-A72 and Cortex-A73 designs that other vendors can license. Short of a “Cyclone” moment for Qualcomm, their CPU performance is close enough to the rest of the Android pack that they can’t clearly stand out from other vendors in the way Apple can. And that means they need to promote and sell their SoCs as the sum of their parts, and not just a CPU with a bunch of extra stuff bolted on.

This also means that even though Qualcomm is one of the first vendors to ship a 10nm high-end SoC – and enjoying the performance benefits thereof – we’re also going to see the company downplay benchmarks a bit for not capturing the complete “end-user experience.”

And to the company’s credit, they aren’t wrong. CPU, GPU, and memory performance are all easy enough to test, vary a great deal among SoC designs, and are similarly easy to communicate. DSPs, ISPs, and other aspects are not nearly as easy to test, never mind communicating those advantages to a wider audience.

Still, once we get outside of CPU/GPU/memory and into other functional blocks of a SoC, things get murkier for everyone. A video decode block may as well be a static piece of hardware – as long as it does its job well, no one will notice – and a great Wi-Fi radio can be kneecapped by a bad device design. So as much as Qualcomm wants to push the broader end-user experience, they will never completely escape the fact that the core features of their SoCs will be the first to be put to the test, and that wider testing is going to be less meaningful to customers, or more practically speaking outside the budgets of technology journalists.