Qualcomm on Tour: Power, Camera Testing, & More

In any case, let’s talk about the tour itself. A first for Qualcomm, the company has given us a bit of access to show off some of the aspects of their SoCs we can’t easily measure ourselves, or to show off other parts of the Snapdragon platform (such as the software stack) that can’t be empirically measured. Given that Qualcomm has historically kept to themselves and been hesitant to engage with tech journalists, even a limited tour is a notable shift for the company. Not to mention a promising sign that, if nothing else, they better understand that the work their engineers and other staff put into products like the Snapdragon 835 deserves to be in the spotlight as well. The idea that engineering is cool isn’t just a STEM educational platform, but something we at AnandTech experience week in and week out.

Power Lab

Given that Qualcomm’s meeting room for press testing was only setup to test performance and not power consumption, it was only fitting that the company’s tour started at their power lab. Here, director of product management Johnny John had setup a demo comparing the power consumption of Snapdragon 820 versus 835. While the usual caveats apply – mainly, that this was a prearranged demo that we didn’t control – it none the less suitably highlights both the power consumption improvements of 835, and Qualcomm’s direction with balancing power consumption with performance for the new SoC.

For this demo, Qualcomm set up otherwise identical development phones running the SD820 and SD835 respectively. Both were running the same fixed VR workload as an example of a high power consumption task. Since this was a fixed workload, the faster SD835 phone in turn gets to bank the entirety of its advantage in power savings. Meanwhile to measure power consumption, Qualcomm’s power measurement gear tapped into the phones at the battery level, so these are phone-level measurements.

Qualcomm Power Testing - Device Level w/Fixed Workload
  Power Consumption
SD820 Reference Phone 4.60W
SD835 Reference Phone 3.56W

The end result had the SD820 phone drawing an average of 4.6W, while the SD835 phone was drawing 3.56W, a power reduction of 23%. Real world use cases won’t be fixed workloads, so the power gains won’t be quite as great, but it shows where Qualcomm’s customers can go in configuring their devices. And indeed, Qualcomm’s own reference devices seem to be tuned a bit more towards power savings than performance, going hand-in-hand with the SoC size reduction that Qualcomm has also gone for with their new SoC. Customers make the final call, but Qualcomm seems to be nudging customers towards using their 10nm gains to curb power consumption more than improve performance.

Graphics & VR

The second stop on Qualcomm’s tour was what they call their Snapdragon Advanced Content Lab. This lab’s focus was on graphics and AR/VR development, though as the polar opposite of a Spartan lab or meeting room, “den of geeks” may be the better description.

To be honest, coming off of CES and GDC, Qualcomm’s advanced content group didn’t have much new to show off. The company is continuing to focus on getting Snapdragon SoCs into VR/AR headsets, and has been producing demos, hardware prototypes, and software toolsets to that end, all of which they have been showing off at the aforementioned trade shows. This is essentially the backend heavy-lifting that Qualcomm is doing to enable devices like the Pico Neo CV that we saw at GDC this year.

Along those lines, the company is also keen on showing off the software side of the equation with their performance profiling tools. The nuances are admittedly more something a developer is going to appreciate than an end-user, but it is a prime example of why the company is eager to brand Snapdragon as a platform as opposed to a processor. In the long run, they expect that software will become a much greater part in defining the overall platform.

Camera Lab

Our third stop was the company’s camera testing lab, which although was primarily demonstrating well-known methods for camera testing, was impressive in scope and price tag (ed: especially to tech journalists who would kill for similar equipment for phone reviews). The takeaway, at a high level, is that Qualcomm wants to show off the rigors of their testing methodology, and that every decision they make with their ISPs and associated software are based on significant empirical testing.

On the photo side of matters, the company has a few interesting tools at their disposal, the most useful likely being their variable lighting system, a pair of massive light cabinets that can generate light at a range of intensities and color temperatures. And though it may sound trivial, as our own Joshua Ho can attest to first-hand, this kind of consistent, systematic testing is not easy to do.

Meanwhile for testing the EIS capabilities of their ISP, Qualcomm has a specialized rig just for shaking phones. The particular ability that makes this rig noteworthy is that it can shake a phone using a pre-determined, tightly timed sequence, so that engineers can go back and see how well their EIS system handled specific motions. The ultimate goal here is to tweak their algorithms to produce good EIS results across a variety of scenarios, so that in average use cases the phone isn’t struggling to stabilize video.

Snapdragon Demo Room

The final stop on Qualcomm’s lab tour was what the company refers to as the Snapdragon Demo Room – which is to say that the company had rolled out a number of experience-based demos to show off various non-benchmark related aspects of their SoCs. This included audio, computer vision, and of course, LTE.

In recent months Qualcomm has been pushing the advantages of higher performance LTE modes, which in turn are the basis of what Qualcomm is branding as Gigabit LTE. The most recent LTE categories are leveraging both carrier aggregation and higher-order QAM modes, namely, 256-QAM. These higher-order modes require greater signal-to-noise ratios to be properly received, but in return allow a signal to carry more data, improving the total throughput of the network. The key point of Qualcomm’s simulations being that even with the tighter requirements of Category 16, it’s useful enough of the time to have a meaningful impact on improving spectral efficiency/reducing network (airtime) loads. Though, as I’m sure Qualcomm is painfully aware, putting theory into practice means getting carriers to upgrade their networks to support higher LTE categories.

One particularly interesting demo, even if things didn’t actually go quite according to plan, was iris scanning/recognition on a SD835 reference phone. Manufacturers have been toying with iris scanning as an alternative for fingerprint unlocking for a bit now, both as a means to remove the relatively large fingerprint sensor from their bezels and to offer a means for unlocking a phone that doesn’t require one’s hands. With the latest rendition of the technology, Qualcomm was eager to show off the improvements in the technology, as well as reiterate its security. The result was something of a wash; the demo worked very well with the product manager, but the phone couldn’t see/recognize my irises consistently enough to unlock the phone (ed: or perhaps Ryan is just soulless). Which this being a prototype, problems are not unexpected, but it’s a reminder that the tech hasn’t had the same number of development cycles as more proven fingerprint scanning technology.

On the flip side of the coin, how well the phone can see the rest of the world is also a subject of interest to Qualcomm. Computer vision/object detection demos aren’t new, but like other players in the industry, Qualcomm is lining up behind the recent advances in machine learning. By being able to efficiently executer (infer) highly trained neural networks, they hope to be able to do things faster and other new things entirely than what traditional computer vision has been capable of doing.

Finally, the company was also showing off their audio efforts, both on the playback and recording sides. On the former, they had an A/B setup between a phone and a dedicated receiver to show off the audio quality of the Snapdragon’s audio codecs and DAC, reiterating that at this point a properly designed phone should be able to keep up with dedicated audio gear for non-audiophiles, even with CD (or better) audio quality. Meanwhile on the audio input side, the company was showing off their improved voice activation capabilities for Snapdragon 835. While speed was hit & miss – both the SD835 and SD820 phones often responded in around the same time – over the day the company had recorded the newer phone as more frequently recognizing the activation phrase than the older phone.

Overall, while Qualcomm can’t easily quantify most of these experiences, it’s exactly these kinds of experiences that the company is wanting to bring to the forefront of the public’s mind, in order to show how Snapdragon is a platform, and to differentiate it from other SoCs. Just how much success they will have at this remains to be seen, but in the long run how successful they are here stands to have a significant impact in how the company’s chip-design arm presents itself to the world at large, and how it advertises its wares.

Qualcomm on Benchmarks versus End-User Experiences First Thoughts
POST A COMMENT

128 Comments

View All Comments

  • yankeeDDL - Wednesday, March 22, 2017 - link

    In most of the graphs where the iPhone is present, they trounce anything else. It is quite disappointing, being an Android enthusiast, to see that the 835 does not catch up with a 6 month old phone... Reply
  • shing3232 - Wednesday, March 22, 2017 - link

    but the bright side is that huge reduction in Power usage compare to 820 Reply
  • ddriver - Wednesday, March 22, 2017 - link

    This is comparing apples to oranges. Unless the two CPUs run the same code, performance is irrelevant.

    The web is majorly a shitpile of bloat, ridden with inefficiency. Apple simply invested more time into optimizing their web / JS implementation. This is not really indicative of CPU performance, only of web implementation optimizations.

    And while it is true that apple's single threaded performance has been better, that is only a part of the story. You have the budget of n amount of transistors to put into x amount of total performance. If you have more threads, then obviously individual threads will be slower. Optimizing for low count threads is actually a pretty good idea given the typical mobile device usage patterns.

    Why do most ARM chipmakers push for higher core count is a mystery to me. That is a STUPID strategy. It makes it that much harder to squeeze the most of your hardware.

    I've been running proprietary software on phones and tables since 2012, software designed to scale adequately, and as a result, I see better overall performance from flagship android devices despite apple's better ST performance, but only because of the kind of workloads I am running. So while the chips aren't anywhere nearly as slow as AT's lame benchmarking suite would suggest, it is not exactly straightforward to get the max of their performance.

    Lastly, the is this thing called "fast enough". Even if apple chips are traditionally faster in typical mobile applications than those found in android devices, this is not really an issue if the slower devices are still fast enough. I haven't really seen bottlenecks in the few 3rd party android apps I am using, so even with software that has not been designed to make use of many cores, things still run fast enough to not present an issue in regards with user experience.

    All in all, in general I'd say ARM is not really trying to make things good, at least not as far as the user is concerned. The only reason apple invest into tangibly improving on the stock ARM designs is for the hype factor, rather than actually putting that performance into productivity. Mobile platforms are doubly limited just to make sure they don't revolutionize computing, both in terms of hardware, and available software. Pretty much next to useless toys, intended to use you far more than you use them, unless you have the resources to put into developing proprietary software tailored to your productivity needs.
    Reply
  • close - Thursday, March 23, 2017 - link

    Apple has yet another advantage. Since it controls its ecosystem end to end it can optimize the software for their specific hardware. In the Android ecosystem you have configurations ranging from 1 to 10 cores (or more?), so many different generations, so many different custom and semi-custom cores. A little trickier to optimize. So Android OEMs go for the numbers. Core numbers that is. Now with 25% more cores.

    And lets not forget another aspect. Historically Apple focused on optimizing the SoC layout for performance which led to much bigger cores which doesn't seem to be the method of choice in the Android ecosystem. They worked on improving the density, especially with the A8 and the A10.
    Reply
  • ericgl21 - Wednesday, March 22, 2017 - link

    Yep...iPhone's A10 Fusion chip is very capable indeed. And in September they're probably going to announce a better one (maybe called "A11"). Looks like Apple is ahead of everyone else, especially when it comes to web-related speeds. Reply
  • Samip - Wednesday, March 22, 2017 - link

    Did you not read the disclaimer right below the web benchmarks? Reply
  • joms_us - Wednesday, March 22, 2017 - link

    Guess what? I have never seen any comparison between iPhone 7 and flagship Android phone with SD821 where iPhone 7 is faster in app and browsing site. These benchmarks mean nothing if you are using different platforms particularly OS.

    Check out this comparison

    https://youtu.be/mcTAXsFHu5I
    Reply
  • gigathlete - Wednesday, March 22, 2017 - link

    Check out this comparison: https://www.youtube.com/watch?v=vm8zC2VAr8w Reply
  • joms_us - Wednesday, March 22, 2017 - link

    Bwaha PhoneBluff. Who launches 10 apps in a minute and close it right away with the home button? Retards?

    His finger is faster on the home button of iPhone than OP3T =D
    Reply
  • TadzioPazur - Wednesday, March 22, 2017 - link

    This "test" is broken. Instead of measuring each activity individually, the test mashes them all together and measures total time. This makes no sense whatsoever - users are interested in doing one thing at a time, so individual tests somehow reflect perceived speed of the device.

    Measuring all together is not how we use these kind of devices. Especially that a lot of those activities reflect the mass storage sequential read. So all PhoneBuff needed to have done to show his beloved device was the fastest, was to put enough application load activities and leverage faster IO.

    So I do think this "test" is so much worse than the above, posted by @joms_us.
    Reply

Log in

Don't have an account? Sign up now