Triple ISPs: Concurrent Triple-Camera Usage

Interestingly enough, during yesterday’s keynote event, Qualcomm described themselves as a camera company, which is a funny way to see things, but actually somewhat makes sense given the large leaps in smartphone camera capture capabilities over the recent years.

The new Snapdragon 888 pushes the envelope in terms camera abilities by adding a whole new independent third ISP to the SoC, allowing the SoC to now run three independent camera modules concurrently, opening up new use cases for vendors and camera applications.

The new triple-ISP architecture now increases the overall pixel processing throughput by 35% to 2.7Gigapixels/s, allowing for concurrent usage of up to three 28MP sensors with zero shutter lag captures. Alternatively, you can use a combination of 64+25MP sensors with ZSL, or a single 84MP sensor with ZSL. There’s still also support for ultra-high-resolution sensors up to 200MP, but image captures here don’t support ZSL.

Allowing concurrent captures of three sensors now allows for the holy trifecta of ultra-wide-angle, wide-angle and telephoto modules to capture a scene at the same time, allowing for more interesting use-cases such as image stitching and image fusion to happen to seamlessly.

One interesting capability that Qualcomm was advertising is triple-stream 4K HDR video recording. That’s a bit of an odd-ball use-case as I do wonder about the practical benefits, but I do at least hope that the new triple ISP system allows for more seamless switching and zooming in and out between the various camera modules during video recording.

Video recording capabilities this year don’t seem to have changed, compared to the Snapdragon 865. This means 4K120 or 8K30 are still the peak capture modes, supporting also slow-motion of 720p960. Formats are also unchanged, with HEVC encoding in HDR formats such as HDR10+ or Dolby Vision being supported.

AV1 decoding didn’t make the cut this year unfortunately, which means wide-spread adoption in mobile for the codec will be delayed for another year.

While the video encoding formats haven’t changed, the image processing capabilities for HDR capture has. Thanks to the new ISPs and the raw performance throughput, the new Snapdragon 888 will be able to capture 4K HDR footage with the more advanced computational HDR processing being applied on each and every frame of the video.

Qualcomm claims that the new ISP in the Snapdragon 888 is the first to support new next-generation staggered HDR sensors.


Source: OmniVision

These are sensors that can have multiple rolling shutters, meaning sensor line readouts, active at the same time on the sensor. Instead of taking multiple exposures one at a time sequentially by scanning out the sensor matrix from start to finish, the sensor will start another exposure immediately after the completed line read-out, reducing the time in-between exposures greatly. This should allow for significant less motion ghosting between the exposures and a sharper resulting recombined HDR image capture than current generation sensors which only have a single active line readout on the sensor.

While we haven’t actually heard of such sensors from Samsung or Sony yet, Qualcomm is adamant that we’ll be seeing smartphones in 2021 employing this new technology.

A further improvement for still-picture captures is the advancement of the new multi-frame noise reduction engines inside of the ISPs. It’s said that the quality of the noise reduction has been improved this generation, allowing for even better low-light captures with the native capture mode (no computational photography).

Hexagon 780: A Whole new IP for AI & DSP; Adreno 660 GPU Conclusion & First Impressions
POST A COMMENT

122 Comments

View All Comments

  • RSAUser - Monday, December 7, 2020 - link

    It shouldn't be, it's about 10% denser than TSMC 7FFP (127 vs 114), a far cry from TSMC N5 though (173). Reply
  • Zeratul56 - Wednesday, December 2, 2020 - link

    Any word if/when this could end up in a 8cx platform. While probably short of the M1 it could make a windows on arm pc much more compelling Reply
  • domboy - Wednesday, December 2, 2020 - link

    I'm interested it see what the 8cx replacement based on the 888 will be like... Reply
  • colinisation - Wednesday, December 2, 2020 - link

    Andrei/Ian - Any thoughts on the suitability of a 1+3+4 core CPU configuration as against a 2+2+4 config. I recall you guys did a deep dive on the state of threading in Android and it seemed that you had one or two main threads and the rest were low performance.

    Do you think this is still the case or do you think SoC vendors should be looking at increasing the big core count, given more multitasking on phones.
    Reply
  • spaceship9876 - Wednesday, December 2, 2020 - link

    No AV1 video hardware decoding support? You have to be kidding! Reply
  • ikjadoon - Wednesday, December 2, 2020 - link

    Qualcomm has long loved the MPEG Industry. In 2017, Qualcomm was already hating on AV1: https://web.archive.org/web/20170611163031/https:/...

    In 2020, Qualcomm & Samsung are pushing for MPEG 5-EVC (presumably H.267?): https://www.qualcomm.com/media/documents/files/mpe...

    So the upcoming codec wars (again): AV1, VVC (H.266), and EVC (H.267?).

    However, most of the "big" hardware & software companies are backing AV1:

    NVIDIA
    AMD
    Intel
    Apple
    Arm
    Facebook
    Cisco
    Google
    Microsoft
    Netflix
    Samsung (many fingers in many pies)
    Adobe
    Hulu
    BBC
    Alibaba
    Broadcom
    Realtek
    Vimeo
    Xilinx

    Qualcomm already looking like chumps here. They literally use Arm's CPU cores on Google's operating system: Qualcomm's only serious work today is non-CPU IP: cameras, AI, etc. They're a little Texas Instruments?
    Reply
  • Raqia - Wednesday, December 2, 2020 - link

    CPUs, even high performance ones are now just much more commoditized; Qualcomm still does a lot of substantial work for the cache hierarchies and pre-fetchers which matter just as much as the core uArch. I'm always curious about CPU uArch and performance, but for a phone the other units matter more for the experience despite being less testable and prominent in most tech headlines. Reply
  • brucethemoose - Wednesday, December 2, 2020 - link

    "Apple"

    Not so sure about that... the M1 skipped AV1 too.
    Reply
  • RSAUser - Wednesday, December 2, 2020 - link

    No, the M1 has AV1 decode, it's a larger A14. Reply
  • halcyon - Wednesday, December 2, 2020 - link

    Yes, the lack of AV1 decode feels odd. Reply

Log in

Don't have an account? Sign up now