Camera Quality

So we’ve gone over the details of improvements on the sensor, optical system, and finally ISP, and now the question is what this translates to in terms of actual image quality. To get to the bottom of this, we’ve turned to our usual smartphone camera bench locations and taken photos with the iPhone 4S at those locales. In addition I’ve taken photos with the 4S in our light box test with the lights on and off.

Anand mentioned that the smartphone battery life test needs some tweaking, the same also applies to the smartphone camera bench. To that end, we’re doing a few new things here in addition to the same old stuff. I recently started taking video samples side by side with the device under test alongside another reference camera - we’re going to start doing that more for stills now as well, and I’ve done this with the 4S alongside the 4 and an SGS2. I’ve always felt that our photo tests could be a lot more rigorous, and we will be coming up with some much more objective tests, but for now I have a preview with three new controlled tests. These sample photos are taken in another smartphone camera mount on a tripod 1.5 feet away from a test target. If we were being really scientific we’d be using an optical track and such, this is just a tripod with the phone on top some distance away. The goal is to have some objective testing, some subjective/qualitative testing.

The first is a distortion grid which is pretty straightforward. We’re all familiar with distortion, which is a strongly field dependent magnification error. Put another way, rays at different field angles get different magnifications, and the result is either barrel or pincushion. It’s actually easy to qualitate how much distortion there is present based on a sample like this, which we’ll do in the future. In addition, it’s also possible to correct out distortion computationally.

The 4S subjectively has much less distortion than the 4, though both appear to have some barrel going on. Interestingly enough you can immediately tell that the 4S and 4 have close to the same field of view, with the 4S being just a tad wider. The SGS2’s wide field of view becomes readily apparent as well in the fourth image, and it suffers from a non-negligible amount of barrel distortion. I tossed in the other android handsets I have on hand as well as the 3GS to show how far things have come.

Another noteworthy thing is that the 4S minimizes but doesn’t entirely eliminate the colored spot in the center. The 4 had an incredibly distinctive green spot in the center that was so notable you could spot images online and instantly tell they came from an iPhone 4. You can see a magenta circle in the 4’s sample, but it’s a bit harder to detect on the 4S image. It’s an aberration that crops up whenever you’re not looking for it, but seems to elude me when I try and track it down. Oddly enough I managed to find it in my carpet, where the 4S has no such green circle, but the 4 does. We talked earlier about the improved IR filter possibly mitigating this issue, and that does seem to be the case somewhat.

iPhone 4S iPhone 4

Apple spent a lot of time in the keynote discussing their auto white balance functionality in the H4 ISP - the question is how well does it stack up? Onward on to our next test, which is a GMB/Xrite color checker classic card and the Kodak color control patches. I took a reference image with my D300s and configured white balance manually using an 18% grey card for comparison purposes, and then samples with the same phones again.

It’s interesting that the iPhone images have been getting less saturated with each generation. The 4 image looks absurdly oversaturated and almost cartoonish compared to either the reference, the 4S, or the SGS2. I’d say that the SGS2 and 4S are pretty much tied here and look close to but not exactly in line with the reference. Going down the line you can see some of the other vendors have a long ways to go with their own white balance.

The third new test for right now is an industry standard ISO 12233 chart test. I see a lot of people taking pictures of this, but then offer little interpretation of the results. There are a bunch of different components in this chart - the thick 30 degree lines are for use with very popular slant MTF analysis algorithms, there are some patterns for distortion computation, and finally line pairs with ever increasing spatial frequency. Analyzing these requires looking at the full size samples, then finding the point at which the contrast between line pairs goes to zero - essentially, the point where you can’t visually distinguish the pattern from irresolvable grey - in both vertical and horizontal. The numbers on the chart simply correspond to hundreds of line widths per picture height.

The gallery images of the full size charts are good but to really make conclusions you need to look at size crops of those tangential and saggital frequency regions. These are all taken with the same illumination, unfortunately the huge differences in white balance between cameras are just a reflection on how far the industry needs to go.

The difference between the frequency response of the 4S and the 4 is pretty huge. The disclaimer before you run into all of this data and start analyzing is that choosing the cutoff frequency is something of a science in and of itself (especially because there’s a contrast reversal that tricks people up), plus unfortunately all smartphone cameras export lossy compressed images (JPEGs) and not raw data, and there’s all kinds of sharpening and processing routines at play here. That said, I’d put the 4S cutoff in tangential around 17, and the 4 at 13. I threw the 3GS in as well to show how far things have come in such a short period - that camera barely makes it to 10. The results from the SGS2 put it around 15 or 16, though that camera is doing visible sharpening as evidenced by halos at the edge and an interesting spatial frequency response plot. If you look at contrast between 14 and 16 you can see the 4S narrowly edge out the SGS2 without any sharpening.

iPhone 4S iPhone 4 Galaxy S 2

I tossed in images from HDR mode from the 4S and 4 as well. These are made from computationally recombining three differently exposed images stored in a buffer. When you tap the capture button, those last three images get recombined into an image with more dynamic range. Interestingly enough, Apple does some sharpening in HDR Mode - again you can see halos and a big uptick on an SFR plot, but it isn’t subjectively that bad, and we can distinguish lines up to arguably between 17 and 18. In the saggital direction, we can see lines down to about 17.5 on the 4S, 13 on the 4, and around 16 on the SGS2. It’s hard to pick cutoffs when sharpening and nose reduction messes with the image, but clearly the 4S performance is very good. If you want to compare yourself, the original images cropped to tangential and saggital frequency areas is here (2 MB) and the full size charts images are here (19 MB).

Now for the subjective side by sides, which were taken in a bracket holding two smartphones at a time some distance apart horizontally. It’s impossible to take the same exact photo from the same place in space and time, but we’ve done our best to offer a comparison with only a small amount of horizontal shift.

iPhone 4S iPhone 4

The first 14 test images compare the iPhone 4 to the 4S and are at our test locations downtown, and from a few new angles. Then images 15–19 compare the 4 and 4S HDR performance, followed by 20–24 and 31 which compare the 4 and 4S in low light conditions. Last, 25–30 compare the 4S with the SGS2 which a lot of people have asked for.

The test charts we’ve shown so far tell the story, but seeing real world images makes the difference immediate. The 4S has vastly improved dynamic range over the 4 - you can see many more details in shadows and highlights, thanks in part to increased well depth between the two sensors. The difference between the 4 and 4S white balance is immediately visible as well in sample 4 where the 4S looks more like the actual bridge, and in 7 of a fountain.

In low light tests, the difference is even more dramatic and visible. The neon lit grill sign in 21 on the 4 is something of a blurry mess, while you can actually see the individual tubes in the 4S sample. I also shot a low light test in our light test box, number 31 where you can see a huge difference in noise between the 4S and 4.

The comparison with SGS2 is a little harder to make visually given the difference in field of view between the two. Aligning the two cameras also proved a challenge again thanks to this big difference in field of view, but you can see the effects of Samsung’s sharpening when you look for it. Subjectively, however, the two are quite close.

iPhone 4S iPhone 4 Galaxy S 2

I’ve also included the 4S images in our usual lightbox gallery with and without flash, and the normal smartphone camera bench.

I mentioned earlier that I intentionally avoided using the LED flash on the 4, and the same continues with the 4S. If you’re under 2 feet away from the subject, the tiny little fresnel lens on top of the LED flash continues to not be good enough at spreading light around evenly. There’s also a nice bleeding effect that happens on the white model that doesn’t affect the image but just casts a huge weird light pattern on the wall from light bouncing around inside the glass plate. In addition, Apple still doesn’t illuminate the scene when focusing in the dark, so often you’ll entirely miss focus if it’s too dark.

With the lights on inside the box you can see how much better the 4S’ white balance is than the 4, which has an almost red colored background in this test. Oversaturation is also gone, thankfully. I also tossed in images from HDR mode with the lights on. I haven’t touched on it as much but the 4S also shows a big improvement in brightness uniformity with essentially no unwanted vignetting.

The front facing camera on the 4S is unchanged from the 4 (still just VGA), however this offers an interesting opportunity to compare that ISP-enhanced whitebalance. If you look at the sample from the 4S and compare to the 4, the difference is pretty shocking.

To conclude the still image analysis section, we can safely say that Apple has gotten serious about image quality with the 4S. Before the iPhone 4, the iPhone camera largely felt like a commodity cameraphone solution tacked onto otherwise great hardware. With complete control over almost the entire imaging chain on the 4S - custom optics, ISP, and software - we’re starting to see some of Apple’s vertically controlled influence spill over into this arena as well. Nokia has built an incredible reputation for itself by using glass optics and building phones around high performance cameras (like the N8 which is still in a different category entirely), and for the first time we’re starting to see Apple venture into that category as well.

Improved ISP in A5 Video Capture Quality


View All Comments

  • metafor - Tuesday, November 01, 2011 - link

    Fair enough. But that really doesn't take away from the fact that the A5 is a relatively large chip and from the UV-scans of it, looks to use quite a bit of that die area for the GPU.

    I don't know if a similar scan has been done of Exynos but one can't safely say both chips are far bigger than SoC's traditionally used in this space.

    Though that trend appears to be moving forward with MSM8960 and Tegra 3.....
  • PeteH - Tuesday, November 01, 2011 - link

    That leads to an interesting question: will Apple always have the largest SoCs, and thus (most likely) the highest performance in the mobile space?

    The reason I could see this happening is that Apple doesn't have to sell their SoC's at a profit, so they're paying closer to cost for the chips (excluding the fab mark up). Other manufacturers (like NVIDIA) need to make a profit on their chips.
  • name99 - Thursday, November 03, 2011 - link

    "I'm not entirely sure why they had to use such a powerful GPU, though. "

    And you know EXACTLY how Apple use the GPU do you?
    Does Siri run some of its workload on the GPU? Does the faster camera stuff (eg fast HDR) run on the GPU? Does Apple already have OpenCL running (for internal use) on iOS?
  • doobydoo - Friday, December 02, 2011 - link

    He must be an Android fan.

    Androids new marketing campaign will offer a revolutionary 'new' feature - the ability to have a slower GPU than other phones!!!

  • InternetGeek - Monday, October 31, 2011 - link

    They might give AMD and nVidia a run for their money if they ever tried creating desktop products... Reply
  • sprockkets - Monday, October 31, 2011 - link

    Kyro 2 was a good chip, but obviously went to focus on the desktop market. Reply
  • tipoo - Tuesday, November 01, 2011 - link

    Maybe, but there's a reason such crossovers usually take so long. Look at Intel trying to get into this space, I don't doubt they will be good at it but it takes years of development. Imagination specializes in low power, it would take lots of development effort to get into the high power desktop game. Reply
  • _tangent - Tuesday, November 01, 2011 - link

    I think this might be intentionally ironic given they got out of that game a long time ago :P

    On point though, anyone would given AMD and nVidia a run for their money with the right up front cash and expertise. I imagine the barrier to entry into that market is truly colossal though. Point is, the SGX543 MP2 is no evidence one way or the other.
  • lurker22 - Monday, October 31, 2011 - link

    Before buying many people who got a 4s on AT&T told me how much better it was than their prior AT&T iphones.

    Anand, thanks for confirming and explaining the reasons.
  • LordSojar - Monday, October 31, 2011 - link

    Can't we have reviews as detailed as this for the really big name Android phones? They are always far less detailed and lack a lot of the testing put into this.... thing....

    Apple makes a few adjustments, tweaks a few things, adds in the same processor that's in the iPad 2, and we have a highly detailed, scientific review that covers every single aspect, even if said aspects are the same. Samsung releases a new phone that has overall better features, faster CPU, faster NAND, a different and arguably better (or at least equal) screen, and mums the word?

    The bias is getting a bit out of hand at this point... We get that you're big time Apple fans, but cmon... At least do a major review of this caliber for the Droid RAZR and the Samsung Galaxy Nexus and the Galaxy S2 Skyrocket (LTE on AT&T!). Even if you combine them into one review, just make it THIS detailed for once instead of giving Apple the huge, super detailed ultra review!

Log in

Don't have an account? Sign up now