Camera Improvements

Arguably the second largest hardware change (with the A5 SoC being the first and largest) in the 4S is the inclusion of a much improved 8MP camera. In case you’ve forgotten, the iPhone 4 previously included a 5 MP camera. Back when the 4 was introduced, Apple talked for the first time about backside illumination, and pixel sizes. In a later update, the camera got even better with the ability to buffer three full size images and merge to HDR in real time. This time, Apple brought up F/# and backside illumination again, and added one more thing.

Though Apple never talked about any of their optical design for the iPhone 4 camera, to the best of my knowledge the design likely was close to reference designs reported on a few lens lists consisting of four plastic elements. For the 4S, Apple has mixed things up by including its own optical design front and center, and made special note of a five plastic element design. I’ve put together a table showing the 4 and 4S in comparison based on what information is available.

Note that many have speculated that Apple is dual sourcing the CMOS sensor which seems likely, and given the sensors out there the two most likely choices are Omnivision’s OV8830 and Sony’s IMX105. Both of these have almost identical specifications, including 1.4µm pixels, a 1/3.2“ format, and an improved backside illumination process over the previous generation wafer-scale process. Omnivision’s BSI–2 process cites some specifications that seem to line up with what Apple talked about in their presentation, including better quantum efficiency (ability to convert photons into electrons), low-light sensitivity, and larger well capacity (which translates to increased dynamic range). You’ll note that the 4S uses the same sensor format as the previous generation - 1/3.2”, and includes more pixels, which results in the pixel size going down from 1.75µm to 1.4µm.

iPhone 4 vs. 4S Cameras
Property iPhone 4 iPhone 4S
CMOS Sensor OV5650 OV8830/IMX105
Sensor Format 1/3.2" (4.54 x 3.42 mm) 1/3.2" (4.54 x 3.42 mm)
Optical Elements 4 Plastic 5 Plastic
Pixel Size 1.75 µm 1.4 µm
Focal Length 3.85 mm 4.28 mm
Aperture F/2.8 F/2.4
Image Capture Size 2592 x 1936 (5 MP) 3264 x 2448 (8 MP)
Average File Size ~2.03 MB (AVG) ~2.77 MB (AVG)

Everybody likes talking about sensors (and I see lots of attention given to them), but any good photographer knows that it’s a combination of optical system and sensor that matters to performance. Optical design is important, and having studied as an optical engineer I find it interesting that Apple would draw attention to having a custom design of their very own with an additional plastic element. For a while I’ve held off on really talking about smartphone camera optics, but while we’re here, let’s touch briefly on them.

The iPhone 4S camera module

Thus far this generation and the one before it have primarily used 4 plastic elements, and virtually everyone but Nokia uses nothing but plastic (Nokia famously uses Zeiss-branded designs, often with glass elements). Optical design is generally driven by material availability, and there are only a few optical grade (read: transmissive in the visible) thermoplastics out there - Styrene, Polystyrene, ZEONEX, PMMA (Acrylic) and so forth - the list is actually relatively short. Thankfully polystyrene and PMMA can be used to make something of an achromatic pair, with polystyrene as a flint, and PMMA as something of a crown. Plastic provides unique constraints as well though - coatings don’t stick well, not very many have great optical properties, they have a high coefficient of thermal expansion, high index variation with temperature (which oddly decreases with increasing temperature), and less heat resistance or durability among others. With all those downsides you might wonder why smartphone vendors use plastic, and that reason is simple - they’re cheap, but more importantly, they can be molded into complicated shapes. Those complicated shapes are aspheres, which are difficult to fabricate out of glass, and afford much finer control over aberrations using fewer elements, which is an absolute necessity when working with very little package depth.

Apple's 4S versus 4 infographic

So what does adding another element get you? Well, when you’ve faced with limited material choices, adding more surfaces gives you another opportunity to balance aberrations that start blowing up rapidly as you increase F/#. That said, there are tradeoffs as well to adding surfaces - more back reflections, increased cost, and a thicker system. In the keynote, Apple notes that sharpness is improved by 30% in their new 5 element design, and MTF is what they’re undoubtably alluding to.


Genius electronic optical - 5P lens. Compare to above.

Genius electronic optical has a page on their website with a lens system that seems likely to be what’s in the 4S, as the specifications include 8 MP resolution (same size), same sensor format, F/# (2.4), 5 plastic elements (5P) and looks basically like what’s in the 4S. Other than that, however, there’s not much more that I can say about this Apple specific design without destructively taking things apart. One thing is for certain however, and it’s that Apple is getting serious about camera performance, something that other handset vendors like HTC (with its F/2.2 systems) are also doing.

Apple made mention that it also included an IR filter in the 4S optical design. If you recall back to our Kinect story, I used the 4 camera to photograph the IR laser structured light projector that Kinect uses to build a 3D picture. The 4 no doubt has an IR filter (though not a great one), but it’s probably just a thin film rather than a discrete filter right before the sensor. The 4S includes what Apple has deemed a ‘hybrid IR filter’ right on top of the sensor, which is possibly just a combination of UV/IR CUT filter (UV is a problem too), and an anti-aliasing filter.

If you try and take the same Kinect (IR source) picture with the 4S, thankfully all those non-visible, IR wavelength photons get rejected by the filter. This doesn’t sound like much until you realize that silicon is transparent in the IR and will bounce around off the metal structures inside a CMOS or CCD and create lovely diffraction effects on fancy sensors. I digress though since that’s probably not what Apple was trying to combat here. On a larger scale, IR will generally just cause undesirably incorrect color representation, and thus people stick an IR filter either in the lens somewhere or before the sensor, which is what has been done in the 4S. The thin film IR filters that smartphones have used in the past also are largely to blame for some of the color nonuniformity and color spot (magenta/green circle) issues that people have started taking note of. With these thin film IR filters, rays incident on the filter at an angle (as we move across the field) change the frequency response of the filter and the result is that infamous circular color nonuniformity. I wager the other effect is some weird combination of vignetting and the microlens array on the CMOS, but when I saw Apple make note of their improved IR filter my thoughts immediately raced to this ‘hybrid IR filter’ as being their logical cure for the infamous green circle the iPhone 4 exhibits.

Another minor difference on the 4S is that the LED flash is improved. The previous LED flash had a distinctively yellow-green hue, the LED flash on the 4S seems slightly brighter and also has a temperature that’s subjectively much closer to daylight, though I didn’t measure it directly. I habitually avoided using LED illumination on the 4 and will probably continue to do so on the 4S (and use HDR instead), but it does bear noting that the LED characteristics are improved. Unfortunately the diffuser and illumination pattern still isn’t very uniform or wide. It also seems that all this talk of moving the LED flash to the other side of the device to combat red eye turned out wrong as well.

Display Improved ISP in A5
Comments Locked

199 Comments

View All Comments

  • metafor - Tuesday, November 1, 2011 - link

    When you say power efficiency, don't you mean perf/W?

    I agree that perf/W varies depending on the workload, exactly as you explained in the article. However, the perf/W is what makes the difference in terms of total energy used.

    It has nothing to do with race-to-sleep.

    That is to say, if CPU B takes longer to go to sleep but it had been better perf/W, it would take less power. In fact, I think this was what you demonstrated with your second example :)

    The total energy consumption is directly related to how power-efficient a CPU is. Whether it's a slow processor that runs for a long time or a fast processor that runs for a short amount of time; whichever one can process more instructions per second vs joules per second wins.

    Or, when you take seconds out of the equations, whichever can process more instructions/joule wins.

    Now, I assume you got this idea from one of Intel's people. The thing their marketing team usually forgets to mention is that when they say race-to-sleep is more power efficient, they're not talking about the processor, they're talking about the *system*.

    Take the example of a high-performance server. The DRAM array and storage can easily make up 40-50% of the total system power consumption.
    Let's then say we had two hypothetical CPU's with different efficiencies. CPU A being faster but less power efficient and CPU B being slower but more power efficient.

    The total power draw of DRAM and the rest of the system remains the same. And on top of that, the DRAM and storage can be shut down once the CPU is done with its processing job but must remain active (DRAM refreshed, storage controllers powered) while the CPU is active.

    In this scenario, even if CPU A draws more power processing the job compared to CPU B, the system with CPU B has to keep the DRAM and storage systems powered for longer. Thus, under the right circumstances, the system containing CPU A actually uses less overall power because it keeps those power-hungry subsystems active for a shorter amount of time.

    However, how well this scenario translates into a smartphone system, I can't say. I suspect not as well.
  • Anand Lal Shimpi - Tuesday, November 1, 2011 - link

    I believe we're talking about the same thing here :)

    The basic premise is that you're able to guarantee similar battery life, even if you double core count and move to a power hungry OoO architecture without a die shrink. If your performance gains allow your CPU/SoC to remain in an ultra low power idle state for longer during those workloads, the theoretically more power hungry architecture can come out equal or ahead in some cases.

    You are also right about platform power consumption as a whole coming into play. Although with the shift from LPDDR1 to LPDDR2, an increase in effective bandwidth and a number of other changes it's difficult to deal with them independently.

    Take care,
    Anand
  • metafor - Tuesday, November 1, 2011 - link

    "If your performance gains allow your CPU/SoC to remain in an ultra low power idle state for longer during those workloads, the theoretically more power hungry architecture can come out equal or ahead in some cases."

    Not exactly :) The OoOE architecture has to perform more tasks per joule. That is, it has to have better perf/W. If it had worse perf/W, it doesn't matter how much longer it remains idle compared to the slower processor. It will still use more net energy.

    It's total platform power that may see savings, despite a less power-efficient and more power-hungry CPU. That's why I suspect that this "race to sleep" situation won't translate to the smartphone system.

    The entire crux relies on the fact that although the CPU itself uses more power per task, it saves power by allowing the rest of the system to go to sleep faster.

    But smartphone subsystems aren't that power hungry, and CPU power consumption generally increases with the *square* of performance. (Generally, this wasn't the case of A8 -> A9 but you can bet it's the case to A9 -> A15).

    If the increase in CPU power per task is greater than the savings of having the rest of the system active for shorter amounts of time, it will still be a net loss in power efficiency.

    Put it another way. A9 may be a general power gain over A8, but don't expect A15 to be so compared to A9, no matter how fast it finishes a task :)
  • doobydoo - Tuesday, November 1, 2011 - link

    You are both correct, and you are also both wrong.

    Metafor is correct because any chip, given a set number of tasks to do over a fixed number of seconds, regardless of how much faster it can perform, will consume more energy than an equally power efficient but slower chip. In other words, being able to go to sleep quicker never means a chip becomes more power efficient than it was before. It actually becomes less.

    This is easily logically provable by splitting the energy into two sections. If 2 chips are both equally power efficient (as in they can both perform the same number of 'tasks' per W), if one is twice as fast, it will consume twice the energy during that time, but complete in half the time, so that element will ALWAYS be equal in both chips. However, the chip which finished sooner will then have to be idle for LONGER because it finished quicker, so the idle expense of energy will always be higher for the faster chip. This assumes, as I said, that the idle power draw of both chips being equal.

    Anand is correct, because if you DO have a more power efficient chip with a higher maximum wattage consumption, race to sleep is the OFTEN (assuming reasonable idle times) the reason it can actually use less power. Consider 2 chips, one which consumes 1.3 W per second (max) and can carry out '2' tasks per second. A second chip consumes 1 W per second (max), and can carry out '1' task per second (so is less power efficient). Now consider a world without race-to-sleep. To carry out '10' tasks over a 10 second period, Chip one would take 5 seconds, but would remain on full power for the full 10 seconds, thereby using 13W. Chip two would take 10 seconds, and would use a total of 10W over that period. Thus, the more power efficient chip actually proved less power efficient.

    Now if we factor in race-to-sleep, the first chip can use 1.3 for the first 5 seconds, then go down to 0.05 for the last 5. Consuming 6.75W. The second chip would still consume the same 10W.

    Conclusion:

    If the chip is not more power effficient, it can never consume less energy, with or without race-to-sleep. If the chip IS more power efficient, but doesn't have the sleep facility, it may not use less energy in all scenarios.

    In other words, for a higher powered chip to reduce energy in ALL situations, it needs to a) be more power efficient fundamentally, and b) it needs to be able to sleep (race-to-sleep).
  • djboxbaba - Monday, October 31, 2011 - link

    Well done on the review Brian and Anand, excellent job as always. I was resisting the urge to tweet you about the eta of the review, and of course I end up doing it the same day as your release the review :).
  • Mitch89 - Monday, October 31, 2011 - link

    "This same confidence continues with the 4S, which is in practice completely usable without a case, unlike the GSM/UMTS iPhone 4. "

    Everytime I read something like this, I can't help but compare it to my experience with iPhone 4 reception, which was never a problem. I'm on a very good network here in Australia (Telstra), and never did I have any issues with reception when using the phone naked. Calls in lifts? No problem. Way outside the suburbs and cities? Signal all the way.

    I never found the iPhone 4 to be any worse than other phones when I used it on a crappy network either.

    Worth noting, battery life is noticeably better on a strong network too...
  • wonderfield - Tuesday, November 1, 2011 - link

    Same here. It's certainly possible to "death grip" the GSM iPhone 4 to the point where it's rendered unusable, but this certainly isn't the typical use case. For Brian to make the (sideways) claim that the 4 is unusable without a case is fairly disingenuous. Certainly handedness has an impact here, but considering 70-90% of the world is right-handed, it's safe to assume that 70-90% of the world's population will have few to no issues with the iPhone 4, given it's being used in an area with ample wireless coverage.
  • doobydoo - Tuesday, November 1, 2011 - link

    I agree with both of these. I am in a major capital city which may make a difference, but no amount or technique of gripping my iPhone 4 ever caused dropped calls or stopped it working.

    Very much an over-stated issue in the press, I think
  • ados_cz - Tuesday, November 1, 2011 - link

    It was not over-stated at all and the argument that most people are right handed does not hold a ground. I live in a small town in Scotland and my usual signal strength is like 2-3 bars. If browsing on net on 3G without case and holding the iPhone 4 naturaly with left hand (using the right hand for touch commands ) I loose signal completely.
  • doobydoo - Tuesday, November 1, 2011 - link

    Well the majority of people don't lose signal.

    I have hundreds of friends who have iPhone 4's who've never had any issue with signal loss at all.

    The point is you DON'T have to be 'right handed' for them to work, I have left handed friends who also have no issues.

    You're the exception, rather than the rule - which is why the issue was overstated.

    For what it's worth, I don't believe you anyway.

Log in

Don't have an account? Sign up now