Camera Architecture

The One M9’s camera is probably one of the biggest departures for HTC in the past few years. Starting with the One M7, HTC took a rather bold path by avoiding the megapixel wars that were dominating the smartphone industry on the camera. Instead, they went back up the ladder to a 2 micron pixel size which gave us a four megapixel camera. From a design standpoint, this represents a bold trade-off as consumers have traditionally focused upon megapixel count as the primary indicator of camera quality. However, to really understand why HTC chose to use a larger pixel size we have to understand what the trade-off meant.

In short, a CMOS image sensor is fundamentally constrained by a sensor size for a given form factor. The size of each pixel establishes an upper bound on critical characteristics like the sensitivity of the pixel (how likely the pixel will count a photon) and the full well capacity of each pixel (how many photons a pixel can hold before saturating). Therefore, for the same sensor size, increasing resolution comes at a cost. This cost is a reduction in dynamic range, and a reduction in sensitivity on a per-pixel basis. Given that most people were unlikely to even view a photo at the full 4MP resolution of the HTC One’s camera, this seemed like a pretty reasonable trade-off. The problem with all of this was that the One M7’s camera sensor didn’t seem to be as technologically advanced as its rivals. As a result, the dynamic range of the sensor wasn’t quite as high as one would hope. The other problem was that the Bayer layout of the sensor reduced the realistic resolution of the photo from 4MP to 2MP. HTC’s design choice wasn’t necessarily wrong, as Apple followed HTC by moving to a 1.5 micron pixel size for the iPhone 5s. However, with 8MP the resolution issue wasn’t there and with a custom Sony sensor it seemed that they managed to avoid the problems that HTC had.

With the One M9, HTC seems to have left the Ultrapixel concept behind, as the rear camera has a Toshiba T4KA7 sensor with 1.12 micron pixels. With a 1/2.4” sensor format, this gives us 20.1 megapixels, or around five times as many pixels as the Ultrapixel sensor that we saw in the One M8 and One M7. It would be easy to assume that HTC made this decision simply because they caved to marketing pressure, but this really doesn’t present a realistic picture of HTC’s situation. Fundamentally, it’s clear that HTC is not in a position where they can get a custom camera sensor of extremely high quality like Apple. The other issue is that it’s patently clear that good post-processing, oversampling techniques, OIS, and other techniques can significantly reduce the impact of a small pixel size. This can be seen in phones like the Galaxy Note 4, LG G3, and Lumia 930 as they all manage to deliver competitive low light image quality when compared against something like the iPhone 6. A larger sensor can also significantly offset all of the issues that come with a smaller pixel size. For reference, by simply increasing sensor size from the One M8 to One M9 we get a 40% increase in sensitivity when ignoring other factors.

With this in mind, we’re led to the context of the One M9’s camera system, which represents HTC’s return to a more traditional camera system. The basic specs of the camera system can be seen in the table below.

HTC One Cameras
  HTC One M8 HTC One M9
Front Camera 5.0MP 4.0MP
Front Camera - Sensor Samsung S5K5E2
(1.12µm, 1/5")
OmniVision OV4688
(2.0 µm, 1/3")
Front Camera - Focal Length 1.83mm 3.82mm
Front Camera - Max Aperture F/2.0 F/2.0
Rear Camera 4.0MP 20.1MP
Rear Camera - Sensor OmniVision OV4688
(2.0 µm, 1/3")
Toshiba T4KA7
(1.12 µm, 1/2.4")
Rear Camera - Focal Length 3.82mm (28mm eff) 4.73mm (27.8mm eff)
Rear Camera - Max Aperture F/2.0 F/2.2
Secondary Rear Camera - Sensor OV2722
(1.4µm, 1/5.8")
-

As we can see, it seems that the optics stack has changed dramatically for this generation. The focal length of the camera is 27.8mm effective when multiplied by the crop factor. I’m sure that there are many that would claim to prefer a wider field of view for their camera, for an all-purpose camera I suspect the focal length should be closer to 30mm if possible. The aperture has also regressed somewhat from f/2.0 to f/2.2, which is likely necessary to reduce aberration, as the 4MP resolution of the One M7 and One M8 which hid optical aberrations to some extent with their lower resolution. The sensor choice is a bit surprising though, as the T4KA7 lacks some key features like PDAF which would allow for significantly faster autofocus. For a future device, it would definitely be preferable to see a longer focal length and wider aperture. It's also a bit disappointing that HTC didn't integrate OIS for this generation, as it's really a bit of a necessity for low light performance at this point.

Camera UX

At the end of the day, the quality of images and video that a camera can put out is often the most important aspect of the camera. However, there’s a lot that needs to happen before that can happen. The camera UI itself needs to be fast, fluid, intuitive, and powerful. A poorly designed UI can be a huge friction point, and it’s pretty much the easiest way to get me to hate using a phone. One of the classic mistakes here is using a 16:9 preview for a 4:3 image, which can seem to be a trivial problem but trying to frame something like an ISO chart properly rapidly shows the problems with this issue. The other aspects that can be easy to ignore include focus speed and capture speed, both of which are critical to capture a shot quickly.

If nothing else, HTC packages a custom camera application that is probably the best that I’ve used on any Android device. For those that are unfamiliar with HTC’s camera application, the Sense camera application keeps some critical camera controls in the right place. Flash, still images, video, and the gallery are all present with one touch, and the additional settings are split into either different cameras or additional settings.

The additional settings drawer holds additional scene settings, auto ISO max settings, exposure compensation settings, and white balance settings.

Within this drawer, there’s another settings drawer that allows for deep customization of the camera settings. It’s possible to switch between widescreen and regular crops of the camera sensor, in addition resolution adjustments. HTC also includes some settings to adjust post-processing when it comes to contrast, saturation, and artificial sharpening.

HTC also includes a full-manual camera mode, which allows adjustment of the exposure compensation, exact white balance temperature, manual ISO settings, shutter speed from 1/8000 seconds to 2 seconds, and manual focus control. All of these are controlled using sliders which can get somewhat messy, but it’s still well above anything other Android OEMs include with their standard camera applications. HTC has also fixed issues that came with this feature in the One M8, so there’s no longer a live preview for extremely long exposure times as a frame every 2-4 seconds is pretty much unusable.

Unfortunately, the rest of the experience isn’t as good as what we saw with the One M8. For one reason or another, the camera preview has a relatively low frame rate and resolution. The poor frame rate isn’t a function of shutter speed, as this is noticeable even in daytime when shutter speeds are well above 1/60 seconds for a 60 FPS refresh rate. The resolution of the preview is also a bit concerning as well, as it can be a bit hard to tell whether the camera is properly focused or not. I’m not sure what’s behind this change, as the ISP on the Snapdragon 810 should be more than capable enough to handle a 20MP camera if it’s designed to support a 55MP camera.

The final aspect of the camera experience is focus speed and capture latency, which are crucial to ensure that it’s possible to capture a fleeting moment before it’s lost. It’s pretty safe to say that it’s quite frustrating to try and take a quick photo and end up waiting on the phone to acquire focus, then wait even more for the phone to take the photo and save it. In order to properly test this, we use our standard ISO chart with strong lighting to ensure base ISO and a fast shutter speed, which should represent the ideal case for focusing and capture latency.

Camera Focus Latency (Shooting ISO 12233 Target)

Camera Shot Latency (Shooting ISO 12233 Target)

As one can see, the One M9 has a pretty stunning regression in autofocus speed and overall capture latency when compared to the One M8. However, it’s possible to understand this when in the context of the changes to the camera subsystem. There are two possible changes that could affect autofocus speed, which are the deletion of the Duocam and ImageChip 2 system. It’s hard to say the impact that each has had, but given that the One M8 had fast autofocus whether or not the second camera was covered it seems likely that the loss of the ImageChip 2 ISP is the bigger issue here. The change to capture latency isn’t nearly as significant, and seems to be somewhat reasonable given that the camera bandwidth requirements quadrupled from the One M8 to the One M9. The real issue here is the focus latency, which needs serious improvement. A sensor with phase-detection auto-focus will probably help a great deal here, in addition to improvements in the contrast auto-focus mechanism.

Introduction, Battery Life cont'd Camera Performance
POST A COMMENT

127 Comments

View All Comments

  • Despoiler - Monday, April 6, 2015 - link

    Nice find. I suspected that HTC just botched their settings based on the first release camera results and how they improved greatly on the second release. They need to keep working on it. Reply
  • rd_nest - Monday, April 6, 2015 - link

    Seriously how many 'usable' shots can you get at 1/2 sec shutter speed without OIS? Even with OIS, I doubt I can get many.

    http://forum.xda-developers.com/showpost.php?p=599...

    ^^ This is what is it supposed to show even with all tweaks and manual settings and good lighting condition.

    These above photos are littered with overexposed highlights, loss of detail, noise in sky and just wrong colour. They have a weird greenish tint.

    Above all, when you want to take a quick photo, you can't do so many changes in settings and try to find the right combination. If I had time, I would take out my DSLR - why worry about a phone camera?
    Reply
  • melgross - Monday, April 6, 2015 - link

    Agreed. I take close to 90% of my pictures these days with my phone. I've got expensive DSLRs and lenses. But for many purposes, the phone camera serves the purpose just as well. I never thought I'd say that, but it's true.

    But the phone camera must be a really good one.
    Reply
  • pjcamp - Monday, April 6, 2015 - link

    I had hopes HTC would finally have fixed their camera problems. Th design of this phone reflects the chaos in the company's upper management. They can't seem to make a camera to save their lives. Literally. Reply
  • smorebuds - Monday, April 6, 2015 - link

    Literally? Reply
  • pjcamp - Monday, May 11, 2015 - link

    Just to update you: upon introducing the M9, HTC experienced a 33% month to month decline in revenues and a 39% year to year decline. That shouldn't happen when your latest and greatest hits the market. So yes, they're circling the drain. Reply
  • Stuka87 - Monday, April 6, 2015 - link

    So you are saying they are all going to die because the camera is no good? That seems a bit harsh... Reply
  • pedromcm.pm - Monday, April 6, 2015 - link

    The camera (and other problems) will makes people not consider the M9. Why should they, on the first place? As such, HTC will be going down. Reply
  • LordConrad - Friday, April 17, 2015 - link

    I doubt it. I almost never use the camera in my phone, and when I do it is to take a quick snap of something funny or interesting I just saw. Any phone camera is good enough for that. Reply
  • pjcamp - Monday, April 6, 2015 - link

    HTC has been unprofitable or marginally profitable for a very long time. And yes, that will be the death of the company. They needed to do it right and they didn't. Reply

Log in

Don't have an account? Sign up now