Understanding Camera Optics & Smartphone Camera Trends, A Presentation by Brian Klug
by Brian Klug on February 22, 2013 5:04 PM EST- Posted in
- Smartphones
- camera
- Android
- Mobile
The Imaging Chain
Since we’re talking about a smartphone we must understand the imaging chain, and thus block diagram, and how the blocks work together. There’s a multiplicative effect on quality as we move through the system from left to right. Good execution on the optical system can easily be mitigated away by poor execution on the ISP for example. I put arrows going left to right from some blocks since there’s a closed loop between ISP and the rest of the system.
The video block diagram is much the same, but includes an encoder in the chain as well.
Smartphone Cameras: The Constraints
The constraints for a smartphone camera are pretty unique, and I want to emphasize just how much of a difficult problem this is for OEMs. Industrial design and size constraints are pretty much the number one concern — everyone wants a thin device with no camera bump or protrusion, which often leaves the camera module the thickest part of the device. There’s no getting around physics here unfortunately. There’s also the matter of cost, since in a smartphone the camera is just one of a number of other functions. Material constraints due to the first bullet point and manufacturing (plastic injection molded aspherical shapes) also makes smartphone optics unique. All of this then has to image onto tiny pixels.
Starting with the first set of constraints are material choices. Almost all smartphone camera modules (excluding some exceptions from Nokia) the vast majority of camera optics that go into a tiny module are plastic. Generally there are around 2 to 5 elements in the system, and you’ll see a P afterwards for plastic. There aren’t too many optical plastics around to chose from either, but luckily enough one can form a doublet with PMMA as something of a crown (low dispersion) and Polystyrene as a flint (high dispersion) to cancel chromatic aberration. You almost always see some doublet get formed in these systems. Other features of a smartphone are obvious but worth stating, they almost always are fixed focal length, fixed aperture, with no shutter, sometimes with an ND filter (neutral density) and generally not very low F-number. In addition to keep modules thin, focal length is usually very short, which results in wide angle images with lots of distortion. Ideally I think most users want something between 35 mm or 50 mm in 35mm equivalent numbers.
I give an example lens catalog from a manufacturer, you can order these systems premade and designed to a particular sensor. We can see the different metrics of interest, thickness, chief ray angle, field of view, image circle, thickness, and so on.
During undergrad a typical homework problem for optical design class would include a patent lens, and then verification of claims about performance. Say what you want about the patent system, but it’s great for getting an idea about what’s out there. I picked a system at random which looks like a front facing smartphone camera system, with wide field of view, F/2.0, and four very aspherical elements.
Inside a patent is a prescription for each surface, and the specification here is like almost all others in format. The radius of curvature for each surface, distance between surfaces, index, abbe number (dispersion), and conic constant are supplied. We can see again lots of very aspherical surfaces. Also there’s a doublet right for the first and second element (difference in dispersion and positive followed by negative lens) to correct some chromatic aberrations.
What do these elements look like? Well LG had a nice breakdown of the 5P system used in its Optimus G, and you can see just what the lenses in the system look like.
Facebook
Twitter
RSS
60 Comments
View All Comments
ltcommanderdata - Saturday, February 23, 2013 - link
http://www.sony.net/SonyInfo/News/Press/201208/12-...Sony's new Exmor RS sensors use a stacked structure which places the circuit section underneath the pixel section instead of beside it which should free up more room for more pixels or larger pixels within a given sensor module area or enable smaller modules. They also add a dedicated white channel for a RGBW coding which they claim improves low light performance. Any comment on the efficacy of these techniques?
Apple has been using Sony image sensors for both the iPhone 4S and iPhone 5 so that new Sony IMX135 with 13.13MP, 1/3.06", 1.12 μm pixel sensor looks like a prime candidate for the iPhone 5S.
slatanek - Saturday, February 23, 2013 - link
Well done, Brian. At last someone trying to explain and stress how much more is there to tiny smartphone cameras than just the megapixels. A few years ago I felt like we got over it and the race was over, but now few years later the race just continues but in the smartphone realm (previously compact cameras had the same issues where ultimate picture quality was compromised just to get to a higher number of megapixels on the box). Hopefully this will change in near future, but somehow I'm afraid that were stuck with it just as with the horrible quality displays in notebooks. And it's not that people don't want good quality, it's just that the behemot companys are not willing to take the risk.Anyways, thanks for a good read.
Shftup - Saturday, February 23, 2013 - link
Brain - Well done!This is a fanatastic article. Its well written, relatively easy to understand for any layman, and most importantly makes the reader coming back for more.
jabber - Saturday, February 23, 2013 - link
....for a smartphone to give me as good a picture quality as I got from my 3.2MP Nikon from 2004.When I find one I'll be happy.
I think a lot of the phone companies need to start poaching the optics and software specialists from the camera companies. It's all very well looking at the spec lists for components but its another thing entirely to make them all work together to produce a decent picture.
In the past for me near decent cams have been ruined by over zealous compression settings (Palm Pre2) or what appears to be zero configuration of the imaging processors (Nexus4).
No one yet has tweaked every part of the chain to provide a truly viable alternative to taking a $250+ camera along instead.
Never had anything to do with megapixels IMO. It's other factors that let them down.
I reckon in another 2 years we'll have it pretty much there.
MrSpadge - Sunday, February 24, 2013 - link
Same here, my Sony DSC-W5 from 2005 or so is still way better than my smartphone. If I could get at least similar quality in a new phone that would easily make it worth 50 - 100€ more for me.Tarwin - Saturday, February 23, 2013 - link
I enjoyed the article, it cleared up some doubts I had and taught me a few new things.In light of this article I hope you go into more detail in the One review when you eventually do it. I assume you will give special attention to the c amera due to how it goes against the trend and HTC's focus on it but I also hope you mention how it fits into all this a bit for those of us who read both articles. Thanks again.
DominicG - Saturday, February 23, 2013 - link
Hi Brian,much enjoyed your intro to camera optics. However you state:
"If we look at the airy disk diameter formed from a perfect diffraction limited ... we get a spot size around 3.0 microns"
You mentioned this is a back illuminated chip so that the light is focussing in a medium of refractive index ~3.5. Therefore the wavelength of red light inside the medium is ~700/3.5 = 200nm - still much smaller than the pixel. It is a bit more complicated than this since the optical resolution is of course determined not only by the wavelength but also the numerical aperture (NA) - the light refracts at the air-silicon interface so that a beam in air at 37 deg (2omega =75deg - your example lens), becomes a beam in silicon of only 10 deg. 10 deg in silicon gives a theoretical resolution (Rayleigh condition) of 0.7microns. Anyway, did you take all these factors into account?
fokka - Sunday, February 24, 2013 - link
thanks for this insightful article, brian, that's the sort of read i i visit this site for regularly!i have to say that i expected a little more (side-) content, though, like a more thorough look at the htc one initially pictured, one of the two reasons i clicked on the article. also a comparison with nokias pureview-approach would've been nice, since it's on the complete other side of the spectrum.
but this is just to nitpick, as i enjoyed the read none the less.
StormyParis - Sunday, February 24, 2013 - link
Great articleStormyParis - Sunday, February 24, 2013 - link
Seeing the interest everyone has in good pictures, and the relatively small size and cost of the lens apparatus, wouldn't it make sense to have 2 distinct cameras on a smartphone, and coordinate them to get better image quality ?