Camera Architecture

The One M9’s camera is probably one of the biggest departures for HTC in the past few years. Starting with the One M7, HTC took a rather bold path by avoiding the megapixel wars that were dominating the smartphone industry on the camera. Instead, they went back up the ladder to a 2 micron pixel size which gave us a four megapixel camera. From a design standpoint, this represents a bold trade-off as consumers have traditionally focused upon megapixel count as the primary indicator of camera quality. However, to really understand why HTC chose to use a larger pixel size we have to understand what the trade-off meant.

In short, a CMOS image sensor is fundamentally constrained by a sensor size for a given form factor. The size of each pixel establishes an upper bound on critical characteristics like the sensitivity of the pixel (how likely the pixel will count a photon) and the full well capacity of each pixel (how many photons a pixel can hold before saturating). Therefore, for the same sensor size, increasing resolution comes at a cost. This cost is a reduction in dynamic range, and a reduction in sensitivity on a per-pixel basis. Given that most people were unlikely to even view a photo at the full 4MP resolution of the HTC One’s camera, this seemed like a pretty reasonable trade-off. The problem with all of this was that the One M7’s camera sensor didn’t seem to be as technologically advanced as its rivals. As a result, the dynamic range of the sensor wasn’t quite as high as one would hope. The other problem was that the Bayer layout of the sensor reduced the realistic resolution of the photo from 4MP to 2MP. HTC’s design choice wasn’t necessarily wrong, as Apple followed HTC by moving to a 1.5 micron pixel size for the iPhone 5s. However, with 8MP the resolution issue wasn’t there and with a custom Sony sensor it seemed that they managed to avoid the problems that HTC had.

With the One M9, HTC seems to have left the Ultrapixel concept behind, as the rear camera has a Toshiba T4KA7 sensor with 1.12 micron pixels. With a 1/2.4” sensor format, this gives us 20.1 megapixels, or around five times as many pixels as the Ultrapixel sensor that we saw in the One M8 and One M7. It would be easy to assume that HTC made this decision simply because they caved to marketing pressure, but this really doesn’t present a realistic picture of HTC’s situation. Fundamentally, it’s clear that HTC is not in a position where they can get a custom camera sensor of extremely high quality like Apple. The other issue is that it’s patently clear that good post-processing, oversampling techniques, OIS, and other techniques can significantly reduce the impact of a small pixel size. This can be seen in phones like the Galaxy Note 4, LG G3, and Lumia 930 as they all manage to deliver competitive low light image quality when compared against something like the iPhone 6. A larger sensor can also significantly offset all of the issues that come with a smaller pixel size. For reference, by simply increasing sensor size from the One M8 to One M9 we get a 40% increase in sensitivity when ignoring other factors.

With this in mind, we’re led to the context of the One M9’s camera system, which represents HTC’s return to a more traditional camera system. The basic specs of the camera system can be seen in the table below.

HTC One Cameras
  HTC One M8 HTC One M9
Front Camera 5.0MP 4.0MP
Front Camera - Sensor Samsung S5K5E2
(1.12µm, 1/5")
OmniVision OV4688
(2.0 µm, 1/3")
Front Camera - Focal Length 1.83mm 3.82mm
Front Camera - Max Aperture F/2.0 F/2.0
Rear Camera 4.0MP 20.1MP
Rear Camera - Sensor OmniVision OV4688
(2.0 µm, 1/3")
Toshiba T4KA7
(1.12 µm, 1/2.4")
Rear Camera - Focal Length 3.82mm (28mm eff) 4.73mm (27.8mm eff)
Rear Camera - Max Aperture F/2.0 F/2.2
Secondary Rear Camera - Sensor OV2722
(1.4µm, 1/5.8")
-

As we can see, it seems that the optics stack has changed dramatically for this generation. The focal length of the camera is 27.8mm effective when multiplied by the crop factor. I’m sure that there are many that would claim to prefer a wider field of view for their camera, for an all-purpose camera I suspect the focal length should be closer to 30mm if possible. The aperture has also regressed somewhat from f/2.0 to f/2.2, which is likely necessary to reduce aberration, as the 4MP resolution of the One M7 and One M8 which hid optical aberrations to some extent with their lower resolution. The sensor choice is a bit surprising though, as the T4KA7 lacks some key features like PDAF which would allow for significantly faster autofocus. For a future device, it would definitely be preferable to see a longer focal length and wider aperture. It's also a bit disappointing that HTC didn't integrate OIS for this generation, as it's really a bit of a necessity for low light performance at this point.

Camera UX

At the end of the day, the quality of images and video that a camera can put out is often the most important aspect of the camera. However, there’s a lot that needs to happen before that can happen. The camera UI itself needs to be fast, fluid, intuitive, and powerful. A poorly designed UI can be a huge friction point, and it’s pretty much the easiest way to get me to hate using a phone. One of the classic mistakes here is using a 16:9 preview for a 4:3 image, which can seem to be a trivial problem but trying to frame something like an ISO chart properly rapidly shows the problems with this issue. The other aspects that can be easy to ignore include focus speed and capture speed, both of which are critical to capture a shot quickly.

If nothing else, HTC packages a custom camera application that is probably the best that I’ve used on any Android device. For those that are unfamiliar with HTC’s camera application, the Sense camera application keeps some critical camera controls in the right place. Flash, still images, video, and the gallery are all present with one touch, and the additional settings are split into either different cameras or additional settings.

The additional settings drawer holds additional scene settings, auto ISO max settings, exposure compensation settings, and white balance settings.

Within this drawer, there’s another settings drawer that allows for deep customization of the camera settings. It’s possible to switch between widescreen and regular crops of the camera sensor, in addition resolution adjustments. HTC also includes some settings to adjust post-processing when it comes to contrast, saturation, and artificial sharpening.

HTC also includes a full-manual camera mode, which allows adjustment of the exposure compensation, exact white balance temperature, manual ISO settings, shutter speed from 1/8000 seconds to 2 seconds, and manual focus control. All of these are controlled using sliders which can get somewhat messy, but it’s still well above anything other Android OEMs include with their standard camera applications. HTC has also fixed issues that came with this feature in the One M8, so there’s no longer a live preview for extremely long exposure times as a frame every 2-4 seconds is pretty much unusable.

Unfortunately, the rest of the experience isn’t as good as what we saw with the One M8. For one reason or another, the camera preview has a relatively low frame rate and resolution. The poor frame rate isn’t a function of shutter speed, as this is noticeable even in daytime when shutter speeds are well above 1/60 seconds for a 60 FPS refresh rate. The resolution of the preview is also a bit concerning as well, as it can be a bit hard to tell whether the camera is properly focused or not. I’m not sure what’s behind this change, as the ISP on the Snapdragon 810 should be more than capable enough to handle a 20MP camera if it’s designed to support a 55MP camera.

The final aspect of the camera experience is focus speed and capture latency, which are crucial to ensure that it’s possible to capture a fleeting moment before it’s lost. It’s pretty safe to say that it’s quite frustrating to try and take a quick photo and end up waiting on the phone to acquire focus, then wait even more for the phone to take the photo and save it. In order to properly test this, we use our standard ISO chart with strong lighting to ensure base ISO and a fast shutter speed, which should represent the ideal case for focusing and capture latency.

Camera Focus Latency (Shooting ISO 12233 Target)

Camera Shot Latency (Shooting ISO 12233 Target)

As one can see, the One M9 has a pretty stunning regression in autofocus speed and overall capture latency when compared to the One M8. However, it’s possible to understand this when in the context of the changes to the camera subsystem. There are two possible changes that could affect autofocus speed, which are the deletion of the Duocam and ImageChip 2 system. It’s hard to say the impact that each has had, but given that the One M8 had fast autofocus whether or not the second camera was covered it seems likely that the loss of the ImageChip 2 ISP is the bigger issue here. The change to capture latency isn’t nearly as significant, and seems to be somewhat reasonable given that the camera bandwidth requirements quadrupled from the One M8 to the One M9. The real issue here is the focus latency, which needs serious improvement. A sensor with phase-detection auto-focus will probably help a great deal here, in addition to improvements in the contrast auto-focus mechanism.

Introduction, Battery Life cont'd Camera Performance
POST A COMMENT

127 Comments

View All Comments

  • phoenix_rizzen - Wednesday, April 8, 2015 - link

    Once you start using double-tap-to-wake, and double-tap-status-bar-to-sleep (some launchers even let you set double-tap-on-home-to-sleep), you'll find yourself rarely using the power button for anything.

    However, if you use your phone a lot for music, you'll really come to love having the power button and volume controls on the back of the phone. You don't have to put your hand in your pocket with the phone to manage the media. Just slide the phone into your pocket with the screen along your leg ... and the buttons are now accessible through the outside of the pocket! So much nicer that way. Especially with the headphone jack on the bottom, which makes the buttons align correctly when the phone is in your pocket (move your finger "up" or straighten it for volume up).

    The other really nice thing about the G2 is that it has a 5.2" screen size ... but the overall phone size is smaller than most phones with only 5.0" screens. They're really done a great job keeping the overall phone sizes down since the Optimus G. I have no issues keeping my G2 in my front jeans pockets, front shorts pockets, or the inside pocket of my jackets.

    Compared to the last 3 HTC and Sony phones, the LG G2 can be considered a compact phone.
    Reply
  • melgross - Monday, April 6, 2015 - link

    I think a big part of the problem is that Apple surprised the industry with the A7 64 bit SoC. Everyone is trying to catch up. But it seems likely that no one was actively working on 64 bits except Apple. The new Tegra looks good, but has its own problems with thottling, etc.

    Qualcomm was caught with its pants down. That's for sure. The 810 seems rushed, and is far from optimized. Perhaps later in the year, new tapeouts will fix some of the problems. Samsung's SoC is a bit better, but is still behind
    Reply
  • aenews - Tuesday, April 7, 2015 - link

    Well final versions of products haven't been released for the X1. And take in mind that even though the K1 was released earlier than the Snapdragon 805, even the Snapdragon 810 doesn't come close to matching it. And the X1 is "twice" as fast potentially. Even if the final performance is reduced in mobile settings, it would still be far, far faster than anything else =). And the K1 has no issue with throttling. Nvidia wouldn't let default settings be too aggressive if it meant throttling. The Snapdragon 810 would probably perform more admirably in these new devices with the temperature throttling turned off or adjusted in terms of temperature or reduced clock speeds. Reply
  • sonicmerlin - Saturday, April 11, 2015 - link

    I would prefer a 32 bit A8X with my iPad Air 2 so I could have the increased RAM footprint of 64 bit apps back. Qualcomm was taken surprise by the public hype of 64 bit. Even the Nexus 9 suffers a bit with 64 bit and only 2 GB of RAM. Reply
  • JeffFlanagan - Monday, April 6, 2015 - link

    HTC has made trading my M8 in for a Samsung Galaxy S6 an easy choice. I was probably already going to do it for the VR, but it's clear that their's no benefit to sticking with HTC. Reply
  • melgross - Monday, April 6, 2015 - link

    Well, the consensus right now is that the S6 is the best Android phone for sure, but that the Edge is gimmicky and overpriced. Reply
  • jabber - Monday, April 6, 2015 - link

    Amazes me that in this day and age a premium smartphone can still screw up the camera. How hard can it be? Forget gimmicks just get the basics right. With so many good off the shelf options...just bizarre. Reply
  • J4ckb1ng - Monday, April 6, 2015 - link

    But the camera is a gimmick. A phone is designed to make and receive calls. How odd nobody seems to want a phone to do the one thing it is primarily designed to do. When I read HTC was fiddling with the camera on the M9, I heaved a sigh of relief. Reply
  • jabber - Monday, April 6, 2015 - link

    If you said that 5+ years ago I would have tended to agree but that's no longer the case. The camera for many has become more important than making calls.

    I bet if you asked a survey of 1000 smartphone users what was the top three reasons for having a smartphone I'd be surprised if making calls scraped in at 3.
    Reply
  • Refuge - Monday, April 6, 2015 - link

    I saw that you are on your first smartphone in another post of yours. So I understand if you haven't converted yet.

    But Smartphones aren't phones. I mean they are, but they aren't.

    They are hubs to the world. This is a communication device, that is meant to be mobile and contain every form of communication we could ever crave.

    Instagram, facebook, youtube, text messaging, instant messaging, skype, Uvuu, phone too.

    I use mine mostly for internet, email, and text. But thats just the nature of my business. Phone calls come through too, but only for things that can't be said one off in a text.
    Reply

Log in

Don't have an account? Sign up now