Triple ISPs: Concurrent Triple-Camera Usage

Interestingly enough, during yesterday’s keynote event, Qualcomm described themselves as a camera company, which is a funny way to see things, but actually somewhat makes sense given the large leaps in smartphone camera capture capabilities over the recent years.

The new Snapdragon 888 pushes the envelope in terms camera abilities by adding a whole new independent third ISP to the SoC, allowing the SoC to now run three independent camera modules concurrently, opening up new use cases for vendors and camera applications.

The new triple-ISP architecture now increases the overall pixel processing throughput by 35% to 2.7Gigapixels/s, allowing for concurrent usage of up to three 28MP sensors with zero shutter lag captures. Alternatively, you can use a combination of 64+25MP sensors with ZSL, or a single 84MP sensor with ZSL. There’s still also support for ultra-high-resolution sensors up to 200MP, but image captures here don’t support ZSL.

Allowing concurrent captures of three sensors now allows for the holy trifecta of ultra-wide-angle, wide-angle and telephoto modules to capture a scene at the same time, allowing for more interesting use-cases such as image stitching and image fusion to happen to seamlessly.

One interesting capability that Qualcomm was advertising is triple-stream 4K HDR video recording. That’s a bit of an odd-ball use-case as I do wonder about the practical benefits, but I do at least hope that the new triple ISP system allows for more seamless switching and zooming in and out between the various camera modules during video recording.

Video recording capabilities this year don’t seem to have changed, compared to the Snapdragon 865. This means 4K120 or 8K30 are still the peak capture modes, supporting also slow-motion of 720p960. Formats are also unchanged, with HEVC encoding in HDR formats such as HDR10+ or Dolby Vision being supported.

AV1 decoding didn’t make the cut this year unfortunately, which means wide-spread adoption in mobile for the codec will be delayed for another year.

While the video encoding formats haven’t changed, the image processing capabilities for HDR capture has. Thanks to the new ISPs and the raw performance throughput, the new Snapdragon 888 will be able to capture 4K HDR footage with the more advanced computational HDR processing being applied on each and every frame of the video.

Qualcomm claims that the new ISP in the Snapdragon 888 is the first to support new next-generation staggered HDR sensors.


Source: OmniVision

These are sensors that can have multiple rolling shutters, meaning sensor line readouts, active at the same time on the sensor. Instead of taking multiple exposures one at a time sequentially by scanning out the sensor matrix from start to finish, the sensor will start another exposure immediately after the completed line read-out, reducing the time in-between exposures greatly. This should allow for significant less motion ghosting between the exposures and a sharper resulting recombined HDR image capture than current generation sensors which only have a single active line readout on the sensor.

While we haven’t actually heard of such sensors from Samsung or Sony yet, Qualcomm is adamant that we’ll be seeing smartphones in 2021 employing this new technology.

A further improvement for still-picture captures is the advancement of the new multi-frame noise reduction engines inside of the ISPs. It’s said that the quality of the noise reduction has been improved this generation, allowing for even better low-light captures with the native capture mode (no computational photography).

Hexagon 780: A Whole new IP for AI & DSP; Adreno 660 GPU Conclusion & First Impressions
POST A COMMENT

122 Comments

View All Comments

  • Alistair - Wednesday, December 2, 2020 - link

    Focus on Camera and AI, with tiny ~10 percent CPU performance improvements, and the exact same 4 little cores for the 4th year. I admit I'm a little disappointed. Last time I paid attention it was all about power and Hercules A88 cores.

    Hopefully we see good $700 flagships from Samsung this year instead of the way overpriced S20 series, maybe the integrated modem will help.
    Reply
  • zeeBomb - Wednesday, December 2, 2020 - link

    Awww yissss. Some of the little things as some people said that could be implemented..but hey I'm all in for the luckiest Snapdragon chipset yet. Reply
  • Raqia - Wednesday, December 2, 2020 - link

    Any thoughts on specific use cases for the hypervisor feature they enabled or any comments from them? It would be nice to just run Linux or Windows apps from my phone attached to a monitor and kb. Reply
  • BedfordTim - Thursday, December 3, 2020 - link

    Samsung Dex might help you. Reply
  • abufrejoval - Thursday, December 3, 2020 - link

    To me that is the most exciting feature of the 888 and one where I'm not sure that Google's Android will pick up on.

    Today mobile phones (and increasingly desktops) are under the control of the ecosystem vendor, all trust and cryptography tied to Apple or Google (or Microsoft). Of course device vendors also want a piece (DeX) but really it's the owner who should be in charge.

    Going forward the number of stakeholders can only increase, there will be governments with vested interests and specific compliance concerns, corporate employers etc.

    So the ability to run a flexible number of enclaves that can be guaranteed not to step or spy on each other will eventually become critical, but also allow to break the stranglehold that Apple and Google currently have on the device you own, but don't control.

    We already have enclaves inside SIM cards and baseband controllers, but they are completely physical, secure that way, but not flexible and affordable to multiply.

    So while I would love to have more details, know if this is like SEV/MKTME on x86 or even better, I don't see how Apple, Google or Microsoft (Pluton!) or even the NSA for that matter, can be motivated to hand the supreme power to you and me, while they can now play in a walled garden we oversee, even if we can't sniff inside.

    In my book I should be able to block conversations between enclaves and their cloud controls, while Apple is pushing the envelope in the opposite direction, hiding their device/command & control-center conversations from owners.

    IHMO that needs to be made painfully illegal, before all the others jump on that bandwagon.
    Reply
  • ZolaIII - Wednesday, December 2, 2020 - link

    This actually looks like a horrible SoC, which QC didn't bother to improve much. Regarding Samsungs 7nm node things are actually other way around. First gen based on HD lib and with EUV whose both better & more efficient than TSMC 7nm without EUV but it cost more. The second gen is actually 7nm with UHD lib while TSMC 5 nm is a fully new node with around 2.2x higher density (to Samsung 7nm UHD) but sadly not significantly more power efficient while it cost get up more than duble. On the other hand Samsung cost went down as density went up for around 50% and thanks to maturity and good yields it's estimated as twice as much gates per a same price.
    I really wanted to see a new gen of Adreno's (for 4 SoC generations now) instead of that this is minor rework (couple new functions) in a same cluster configuration as it's predecessor. Now imagine 2x increase in logic with TSMC 5 nm process at 30% lower speed. I do think quoted 35% increase is in ideal conditions (utilising new stuff which we won't see in the next couple of years) while in reality we will see just a small insignificant increase.
    What's the use of single high performance CPU core which outruns the rest by working 50% IPC? Sure each & every first core on any cluster will be a one to bare the burden of the one where everything is started before workers are deployed but it only needs culpe % more capacity to balance that.
    Seams they didn't done nothing to increase cache coherence and efficiency.
    DSP had seen a real improvement but I don't look at QC Hexogen as something good (stiff, property, hard to get to & not flexible) to the point of thinking such things shouldn't exist. The biggest gain will be a integration of 5G modem which will cut power consumption in half.

    All in all a rather bad job.
    Reply
  • iphonebestgamephone - Wednesday, December 2, 2020 - link

    Its good as long as i get the the 25% and 35% imptovements. Reply
  • ZolaIII - Wednesday, December 2, 2020 - link

    Thing is you will only get around 20~25% improvement in a single (X1) core while in full all CPU core utilisation that will sink to only couple % (thanks to; bus, typology and memory coherence bottlenecks. Early Geekbench results already confirmed this. It will be same regarding GPU, it will get it's 30% advantage but only when new futures are used (which they won't be in a long time). Reply
  • iphonebestgamephone - Wednesday, December 2, 2020 - link

    Oh man, i forgot what they did with the 855, 45% cpu improvement over 845. But that was only on geekbench 4 single core. With maybe 15% on multi. Reply
  • Raqia - Wednesday, December 2, 2020 - link

    In addition to integrating the Modem, they also appear to be integrating WIFI (FastConnect 6900) and Bluetooth on die now. Prior year diagrams seem to have illustrated WIFI and Bluetooth off chip.

    "I really wanted to see a new gen of Adreno's (for 4 SoC generations now) instead of that this is minor rework (couple new functions) in a same cluster configuration as it's predecessor."

    Are you really sure about it being a minor rework? Variable rate shading may be possible or not give as much of a performance boost without some changes to the hardware. The rest is just marketing much like the moniker 888 is.
    Reply

Log in

Don't have an account? Sign up now