NVIDIA has upped its ISP game with Tegra 4, and includes the same ISP in both Tegra 4 and 4i, which they’ve dubbed Chimera. This new ISP includes a number of features which were already demonstrated at CES, but NVIDIA went into greater detail.

NVIDIA’s major new feature with Tegra 4 is inclusion of new APIs which leverage the GPU to enable computational camera features In addition to the traditional ISP pipeline. To begin, NVIDIA has made their own enhancements in the imaging chain which leverage this GPU-assisted architecture that sits atop the normal ISP. They’ve outlined a few features which work atop this — HDR panorama, HDR stills, and real time object tracking. The end result is that Chimera is one part programming model and APIs for future development efforts by third parties, another part first party software that NVIDIA will distribute for integration in OEM cameras which includes the HDR features and object tracking.

Chimera allows developers to build blocks around the traditional ISP data flow and get to manipulate either bayer image data or YUV space image data on the CPU or GPU using their own framework. There are still a lot of details to come about the programming model and how this will be exploited, but there clearly is space here for NVIDIA to try and make a case for their GPU being useful for image processing. At the same time, the elephant in the room is OpenCL (and its current absence on Tegra 4) and what direction the industry will take that to leverage GPU compute for some computational photography processing.

The novel new feature of note is something I was skeptical about upon seeing at CES — single frame HDR video. The traditional route for HDR video capture is to capture at twice the framerate of the intended output video, for example for 720p30 video with a two-frame HDR, that would mean capture at 720p60 with two different exposures. What NVIDIA does is a bit different. Instead of capturing two exposures and combining them after capture, NVIDIA has found a way to drive selected CMOS sensors (both IMX135 from Sony and AR0833 from Aptina have been called out specifically, but there are more) in some special fashion, and recombine interleaved high and low exposure images (the +1 stop, –1 stop) image into one equivalent frame. This is all done on the GPU using the same Chimera pipeline blocks, but with NVIDIA’s own algorithm. The results are actually pretty impressive, NVIDIA claims 3 stops of dynamic range (~24 dB), and no recombination artifacts.

Previous demonstrations of HDR video capture on a few other platform have had halos around moving elements because of the issues associated with taking two temporally different frames and recombining them. With NVIDIA’s HDR video capture I saw no halos even on very fast moving objects, with the same quality of HDR.

NVIDIA also demonstrated their panorama capture which includes HDR as well, using a GigaPan Epic 100. Admittedly a better demo might be hand held to demonstrate how well the ISP is able to compensate for change in perspective from the tablet moving around in the hand, the resulting image was the same quality of HDR as the still however. I also saw object tracking which essentially is face tracking (and thus AE / AF assist to that subject) but taken to arbitrary subjects. Paint a box around the object you want to track, and the camera will perform feature extraction and machine learn a model around the object. Rotating around complex geometries seemed to work decently well — the model learns slowly so slow movements can be accommodated, faster moving subjects might not be object tracked.

I also got a chance to tour NVIDIA’s camera tuning labs, where they characterize smartphone CMOSes for handset markers, and work with them on tuning their ISP and imaging experience. I saw one calibration room with many different scenes, and then another room with large test charts, and two sets of standard test scenes in lightboxes. The ever-familiar GMB color checker card and ISO12233 charts were everywhere as well.

The Modem - Icera 500 in Tegra 4i Hands on with the Phoenix, NVIDIA's FFRD
POST A COMMENT

75 Comments

View All Comments

  • tipoo - Sunday, February 24, 2013 - link

    Under 500 in Sunspider, about twice as fast as anything else ARM. But then again, it's a few months newer than that, and actually still not shipping. And as usual with Nvidia they're early to each party (first to dual core, first to quad core), but not always the best performing. We'll see if other Cortex A15 designs beat it.

    I'd love to see four of those cores paired with SGXs upcoming 600/Rogue series.
    Reply
  • jeffkibuule - Sunday, February 24, 2013 - link

    SunSpider is so software sensitive that a Tegra 3 @ 1.2 Ghz on Windows RT beats a Snapdraon S4 Pro @ 1.5Ghz on Nexus 4 using Chrome. It's a terrible benchmark because its so dependent on underlying kernel optimizations in the Android phone market. Reply
  • tipoo - Sunday, February 24, 2013 - link

    True, other benchmarks are similarly impressive though. Reply
  • karasaj - Sunday, February 24, 2013 - link

    Psh it has nothing on my desktop! 125ms on sunspider... Nvidia so behind.

    Anyways, still looks impressive. I really want to see some Krait 600/800 benchmarks.
    Reply
  • tipoo - Sunday, February 24, 2013 - link

    The fact that they're getting well below an order of magnitude slower than desktops is impressive in itself too. Even with iPad 2 level performance I still was reluctant to do most of my web browsing on a tablet for the performance. Maybe with Tegra 4 and beyond hardware speed that will change. Reply
  • Mumrik - Sunday, February 24, 2013 - link

    As someone with heavily tabbed browsing habits, I don't think I'll ever make that jump (and I own a tablet). Reply
  • tipoo - Sunday, February 24, 2013 - link

    Also true, that's my other thing. I like to open a bunch of background tabs and have them ready as I go through each one. Right now, tablets don't do background loading, as far as I know, and if they did they wouldn't be powerful enough to keep the main tab smooth while doing it. Reply
  • Tarwin - Monday, February 25, 2013 - link

    Tablets DO do background loading, as long as they're android. The only performance I've seen is from lack of RAM on my phone and lack of bandwidth on the phone and tablet but those things affect any computer as well. One observation to ne made, they do load in the background but things like audio and video playback will pause if you switch to another tab. Reply
  • von Krupp - Monday, February 25, 2013 - link

    Even Windows Phone 7.5 and 8 do background loading. I haven't used it, but I'd wager that RT does as well, if even the gimpy mobile OS can. Reply
  • tuxRoller - Sunday, February 24, 2013 - link

    As someone who had, until recently, over 40 tabs open on my chrome browser (Nexus 4), the critical problem has been memory. With enough memory, and good enough task management, these problems tend to go away.
    Of course, maybe you are than 0.00001% who has hundreds or thousands of tabs open in which case I pity any computer you are likely to own.
    Reply

Log in

Don't have an account? Sign up now