Qualcomm on Tour: Power, Camera Testing, & More

In any case, let’s talk about the tour itself. A first for Qualcomm, the company has given us a bit of access to show off some of the aspects of their SoCs we can’t easily measure ourselves, or to show off other parts of the Snapdragon platform (such as the software stack) that can’t be empirically measured. Given that Qualcomm has historically kept to themselves and been hesitant to engage with tech journalists, even a limited tour is a notable shift for the company. Not to mention a promising sign that, if nothing else, they better understand that the work their engineers and other staff put into products like the Snapdragon 835 deserves to be in the spotlight as well. The idea that engineering is cool isn’t just a STEM educational platform, but something we at AnandTech experience week in and week out.

Power Lab

Given that Qualcomm’s meeting room for press testing was only setup to test performance and not power consumption, it was only fitting that the company’s tour started at their power lab. Here, director of product management Johnny John had setup a demo comparing the power consumption of Snapdragon 820 versus 835. While the usual caveats apply – mainly, that this was a prearranged demo that we didn’t control – it none the less suitably highlights both the power consumption improvements of 835, and Qualcomm’s direction with balancing power consumption with performance for the new SoC.

For this demo, Qualcomm set up otherwise identical development phones running the SD820 and SD835 respectively. Both were running the same fixed VR workload as an example of a high power consumption task. Since this was a fixed workload, the faster SD835 phone in turn gets to bank the entirety of its advantage in power savings. Meanwhile to measure power consumption, Qualcomm’s power measurement gear tapped into the phones at the battery level, so these are phone-level measurements.

Qualcomm Power Testing - Device Level w/Fixed Workload
  Power Consumption
SD820 Reference Phone 4.60W
SD835 Reference Phone 3.56W

The end result had the SD820 phone drawing an average of 4.6W, while the SD835 phone was drawing 3.56W, a power reduction of 23%. Real world use cases won’t be fixed workloads, so the power gains won’t be quite as great, but it shows where Qualcomm’s customers can go in configuring their devices. And indeed, Qualcomm’s own reference devices seem to be tuned a bit more towards power savings than performance, going hand-in-hand with the SoC size reduction that Qualcomm has also gone for with their new SoC. Customers make the final call, but Qualcomm seems to be nudging customers towards using their 10nm gains to curb power consumption more than improve performance.

Graphics & VR

The second stop on Qualcomm’s tour was what they call their Snapdragon Advanced Content Lab. This lab’s focus was on graphics and AR/VR development, though as the polar opposite of a Spartan lab or meeting room, “den of geeks” may be the better description.

To be honest, coming off of CES and GDC, Qualcomm’s advanced content group didn’t have much new to show off. The company is continuing to focus on getting Snapdragon SoCs into VR/AR headsets, and has been producing demos, hardware prototypes, and software toolsets to that end, all of which they have been showing off at the aforementioned trade shows. This is essentially the backend heavy-lifting that Qualcomm is doing to enable devices like the Pico Neo CV that we saw at GDC this year.

Along those lines, the company is also keen on showing off the software side of the equation with their performance profiling tools. The nuances are admittedly more something a developer is going to appreciate than an end-user, but it is a prime example of why the company is eager to brand Snapdragon as a platform as opposed to a processor. In the long run, they expect that software will become a much greater part in defining the overall platform.

Camera Lab

Our third stop was the company’s camera testing lab, which although was primarily demonstrating well-known methods for camera testing, was impressive in scope and price tag (ed: especially to tech journalists who would kill for similar equipment for phone reviews). The takeaway, at a high level, is that Qualcomm wants to show off the rigors of their testing methodology, and that every decision they make with their ISPs and associated software are based on significant empirical testing.

On the photo side of matters, the company has a few interesting tools at their disposal, the most useful likely being their variable lighting system, a pair of massive light cabinets that can generate light at a range of intensities and color temperatures. And though it may sound trivial, as our own Joshua Ho can attest to first-hand, this kind of consistent, systematic testing is not easy to do.

Meanwhile for testing the EIS capabilities of their ISP, Qualcomm has a specialized rig just for shaking phones. The particular ability that makes this rig noteworthy is that it can shake a phone using a pre-determined, tightly timed sequence, so that engineers can go back and see how well their EIS system handled specific motions. The ultimate goal here is to tweak their algorithms to produce good EIS results across a variety of scenarios, so that in average use cases the phone isn’t struggling to stabilize video.

Snapdragon Demo Room

The final stop on Qualcomm’s lab tour was what the company refers to as the Snapdragon Demo Room – which is to say that the company had rolled out a number of experience-based demos to show off various non-benchmark related aspects of their SoCs. This included audio, computer vision, and of course, LTE.

In recent months Qualcomm has been pushing the advantages of higher performance LTE modes, which in turn are the basis of what Qualcomm is branding as Gigabit LTE. The most recent LTE categories are leveraging both carrier aggregation and higher-order QAM modes, namely, 256-QAM. These higher-order modes require greater signal-to-noise ratios to be properly received, but in return allow a signal to carry more data, improving the total throughput of the network. The key point of Qualcomm’s simulations being that even with the tighter requirements of Category 16, it’s useful enough of the time to have a meaningful impact on improving spectral efficiency/reducing network (airtime) loads. Though, as I’m sure Qualcomm is painfully aware, putting theory into practice means getting carriers to upgrade their networks to support higher LTE categories.

One particularly interesting demo, even if things didn’t actually go quite according to plan, was iris scanning/recognition on a SD835 reference phone. Manufacturers have been toying with iris scanning as an alternative for fingerprint unlocking for a bit now, both as a means to remove the relatively large fingerprint sensor from their bezels and to offer a means for unlocking a phone that doesn’t require one’s hands. With the latest rendition of the technology, Qualcomm was eager to show off the improvements in the technology, as well as reiterate its security. The result was something of a wash; the demo worked very well with the product manager, but the phone couldn’t see/recognize my irises consistently enough to unlock the phone (ed: or perhaps Ryan is just soulless). Which this being a prototype, problems are not unexpected, but it’s a reminder that the tech hasn’t had the same number of development cycles as more proven fingerprint scanning technology.

On the flip side of the coin, how well the phone can see the rest of the world is also a subject of interest to Qualcomm. Computer vision/object detection demos aren’t new, but like other players in the industry, Qualcomm is lining up behind the recent advances in machine learning. By being able to efficiently executer (infer) highly trained neural networks, they hope to be able to do things faster and other new things entirely than what traditional computer vision has been capable of doing.

Finally, the company was also showing off their audio efforts, both on the playback and recording sides. On the former, they had an A/B setup between a phone and a dedicated receiver to show off the audio quality of the Snapdragon’s audio codecs and DAC, reiterating that at this point a properly designed phone should be able to keep up with dedicated audio gear for non-audiophiles, even with CD (or better) audio quality. Meanwhile on the audio input side, the company was showing off their improved voice activation capabilities for Snapdragon 835. While speed was hit & miss – both the SD835 and SD820 phones often responded in around the same time – over the day the company had recorded the newer phone as more frequently recognizing the activation phrase than the older phone.

Overall, while Qualcomm can’t easily quantify most of these experiences, it’s exactly these kinds of experiences that the company is wanting to bring to the forefront of the public’s mind, in order to show how Snapdragon is a platform, and to differentiate it from other SoCs. Just how much success they will have at this remains to be seen, but in the long run how successful they are here stands to have a significant impact in how the company’s chip-design arm presents itself to the world at large, and how it advertises its wares.

Qualcomm on Benchmarks versus End-User Experiences First Thoughts
Comments Locked

128 Comments

View All Comments

  • zeeBomb - Wednesday, March 22, 2017 - link

    Its that time of year again!
  • name99 - Wednesday, March 22, 2017 - link

    "The [3DMark] Physics test runs on the CPU and is heavily influenced by how well an SoC’s memory controllers handle random access patterns. "

    No it isn't, at least not to an extent that matters in any modern CPU. Why do you keep posting this rubbish in review after review?

    The source code is available for examination. It basically tests (frequency)*(number of cores) and is useless for learning anything beyond that. That's why it's always the only test in which Apple looks bad --- because Apple's running two cores as opposed to 4/6/8/10 on Android, and, at least in the past, those cores were under-clocked relative to the Android cores.

    If people want to post the 3DMark Physics numbers, whatever, I don't care. But I do think doing so is a waste of reviewers' and readers' time --- there is simply no useful additional information provided by that benchmark.
    The fact that 3DMark continues to push it (as opposed to the way GeekBench every year or two tries to respond to complaints and concerns about its benchmarks) tell you something about the relative professionalism of the two companies.
  • Matt Humrick - Wednesday, March 22, 2017 - link

    "It basically tests (frequency)*(number of cores)"

    Both of these are factors, but it's not the whole story according to the developer I spoke with at Futuremark. If you have additional information to prove your claim, please share it with me via email. My mind is always open :)
  • name99 - Thursday, March 23, 2017 - link

    I looked into this in detail years ago when there was a big kerfuffle about the iPhone 5S score.
    I'm not interested in spending another day doing the exact same thing. I'll just point out that what I am saying matches the data.
    Sure, I'm not saying that THE ONLY THING is (frequency)*(number of cores), there's some small 5 to 10% variation around that; but that variation is unimportant --- the big picture is embedded in what I said.

    Now, does this mean it's a good benchmark? Well, how much code that people care about is multi-threaded (on Android and otherwise)?
    I'm not interested in relitigating that (given what I consider to be the astonishing incompetence and ignorance we saw on Anandtech the last time this was discussed, with a VAST proportion of readers apparently unaware of such concepts as timesharing, or how to accurately calculate the thread level parallelism of an executing piece of code).
    I will say that the most recent academic papers I've read, dated 2016, referring to work in around 2014, show that it's higher than you might expect, not as high as you might hope. Across a very wide range of Android apps the thread level parallelism is slightly larger than 2, showing, basically (in my interpretation)
    - an Android controlled thread doing misc stuff that's pretty busy
    - a main app thread
    - various small completion routines, async routines, and interrupts
    So basically two cores get you almost all the value in real world core, a third core occasionally picks up a small amount of extra available work.

    Now read what I am saying before getting upset. I'm NOT saying that ARM is stupid to ship 4 (performance) cores. ARM cores are tiny, they can be of (very occasional) value to a few talented developers today, and the only way we'll EVER get the mass market to code more parallel is to have the hardware out there as the default. So I'm happy that ARM is flooding the world with hexacore, octacore, decacore chips. (And I think Apple is being penny-wise and pound-foolish by not making every SoC they ship a triple core ala A8X --- the extra area would be small, and it would likewise provide an incentive for developers to get off their asses.)

    But that's a different argument from whether core-count provides "visible performance" today.
    I think the answer to that is clearly no. The first thing that matters to most users is snappiness (which depends, primarily, on flash performance, GPU [and the quality of the graphics code], performance governor (so does the CPU "start off" fast or "start off" slow and only get fast after .2 seconds of UI interaction? Then there are a few places where overall "endurance" performance matters (like much browser stuff, or viewing complicated PDFs --- both of which are very poorly threaded even as of 2017). Finally the cases where all cores all the time matters, and almost nothing else (the sort of thing 3DMark Physics is testing) are REALLY few and far between.

    Or to put it another way. Most CPU microarchitecture improvement since 2000 has been about discovering and exploiting the stochastic structure of REAL-WORLD computation. There are re-uses and patterns in branching, in memory access, in instruction execution that are exploited ever more aggressively in branch prediction, in cache insertion and liveness tracking, in prefetching, in loop buffers, etc. A benchmark that prides itself on randomness and in providing no way for all those smarts to add value is saying SOMETHING about the worst case performance of a CPU, but it's not clear that that something is of any value to almost everyone.
  • Frenetic Pony - Wednesday, March 22, 2017 - link

    How disappointing that yet another of the very few custom CPU designers is now gone. Looking at the general performance of the CPU now, I see no reason whatsoever to choose Qualcomm over some other, generic ARM hawker that's probably cheaper. They could at least stop pretending and just become a module seller, selling their GPU/modems/etc. separately as there doesn't seem to be any reason to choose a Qualcomm SOC as a whole.

    Other than ditching their stock (if you haven't already) none of this looks good for Qualcomm. Or for ARM for that matter, the A73 doesn't offer any performance boost over the A72 and is still trounced by Apple. Maybe the rumors of Google building its own CPU will come true and we'll see it in the Pixel 2.
  • serendip - Wednesday, March 22, 2017 - link

    No reason? They're one of the few developer and open source friendly chip manufacturers around, even if that relationship ventures into frenemy territory once in a while. Qualcomm modems and imaging blocks are pretty good too.

    Intel are developer friendly but their GPUs can be an abomination to work with. They've also effectively abandoned the mobile space. Mediatek, Huawei and Samsung either give a cold shoulder or a middle finger to devs.
  • StrangerGuy - Thursday, March 23, 2017 - link

    They dropped their own custom cores because why even bother when vanilla ARM does a better job while being cheaper...Plus the economics for a non-Apple, non-Samsung bleeding edge SoC no longer makes much sense, and 99.99% of the population buying these phones doesn't and won't give a shit to the SoC or the benchmarks.
  • Meteor2 - Thursday, March 23, 2017 - link

    It wouldn't surprise me if Qualcomm had multiple core design teams competing with each other. We've seen ARM cores come before; maybe full-custom will come back next year.
  • SyukriLajin - Thursday, March 30, 2017 - link

    I think they are just shifting their focus. The fact that they "rebranded" the snapdragon as a platform instead of just processor is one indicator. SOC is more than just cpu cores and they want people to know that. My guess is, they think that investing more money to develop a more unique platform is more important than spending valuable time and money on redoing the cpu core, ARM already invested a tons of money to develop it, no reason to reinvent the wheel when you can use the resources to provide a platform that will help them be more different then the others. I think it's wise for them. The resources would better off be spent to create a better DSP, modem, GPU etc, which will give them more return than a custom cpu core.
  • MrPhilo - Wednesday, March 22, 2017 - link

    So the Exynos 8895 GPU should in theory be faster than the 540 by quite a bit? Since the Huawei 9 uses 8, whereas Exynos uses 20, but of course with a lower clockspeed. I can see it being at least 20-30% faster than the 540.

Log in

Don't have an account? Sign up now