Camera - Low Light Evaluation

In low-light scenarios, we should see the new iPhone XS showcase significant improvements thanks to the 50% better light capture ability of the new sensor. Apple’s still only employing a f/1.8 aperture lens on the XS - so while it will improve over past phones, at least on paper it’s still at a disadvantage to say Samsung’s latest phones, which have an extra-wide f/1.5 aperture available to them.

Click for full image
[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

In this first shot, we immediately see the new iPhone’s advantage over last year’s flagship. There is a lot more definition in the grass, less noise throughout the image, and less blown out lights in the scene.

Unfortunately, Apple is as expected still at a great disadvantage to Samsung here, as the latter is just able to give more light onto the whole scene, and the most evident, more colour to the grass. In terms of raw low light capture, the Huawei P20 Pro is still far ahead here, thanks to its massive sensor that is able to collect significantly more light.

Click for full image
[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

At first glance, the iPhone XS didn’t shoot a much brighter picture than the iPhone X in this construction scene. Opening up the full resolution images however shows that the new XS showcases much better details and lower noise. It’s not enough to compete with the S9+, and certainly not with the insane ISO25600 shot of the P20 Pro.

It’s interesting to see the improvements over the years from the iPhone 6S on – which barely manages to capture anything in this scene.

Click for full image
[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

The next shot is probably the only one that I found to be really problematic for Apple. Both on the iPhone X and the new XS, the resulting images weren’t consistent in consecutive shots. In four shots in a row, the iPhone XS kept changing the colour temperature. The same thing happened on the iPhone X, so I think this was part of Apple’s exposure / colour balance algorithm.

Colour balance aside, the exposure is similar between the X and the XS, and all the improvements of the new sensor go directly into improved detail and noise reduction throughout the scene, which is significantly better again compared to last year’s iPhone.

Here Apple is very close to Samsung, showcasing a bit better shadows, but still losing out in details in some parts of the scene. The P20 Pro is yet again the low-light kind here, as it just have that much more dynamic range work with.

Click for full image
[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

Again, the iPhone’s new sensor comes into play in these concrete trucks. The XS makes very good dealing of the blown highlights present in the iPhone X shot. Samsung is able to produce more vibrancy in the blue of the trucks. Huawei’s multi-exposure computational photography night mode is the best of all phones here as it’s just able to bring out that much more from the shadows.

Click for full image
[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

Apple's use of SmartHDR in this picture is extremely evident, as it really brings down the highlights of the lamp and brings out more shadows throughout the scene. The XS provides better detail, but it’s not as big of a difference as we’ve seen in other shots.

Apple’s usage of HDR here puts it ahead of the Samsung devices, trading blows with the P20 Pro, winning in some regards, while losing in others.

Click for full image
[ iPhone XS ] - [ iPhone X ] - [ iPhone 7 ] - [ iPhone 6S ]
[ Galaxy Note9 ] - [ Galaxy S9+ ] - [ Galaxy S8 ]
[ LG G7 ] - [ LG G6 ] - [ LG V30 ] - [ OnePlus 6 ]
[ Mi MIX2S ] - [ Pixel 2XL ] - [ P20 Pro ]

Finally, I wanted to test the iPhone XS to its limits and see what it can do in essentially impossible scenarios of low light.

Exposure-wise, the iPhone XS is no better than the X here. It provides better sharpness and less noise, however the image is still too dark to be of any use. I wish Apple would introduce a more innovative low light shooting mode, such as LG’s pixel binning mode. Huawei’s ISO51200 capture of this scene is just so beyond any other current phone, that it really raised the bar in what we’d normally expect to see in a smartphone.

Low-light conclusion

The new iPhone XS sensor is a great improvement to Apple’s lineup. Its advantages over the iPhone X are clearly evident in every single low-light shot, showcasing greater detail and sharpness while reducing noise. SmartHDR doesn’t seem to be something that’s solely for daylight shots, as Apple and the iPhone XS seem to make use of it in some low-light scenarios, giving the camera a further advantage over last year’s phones.

While Apple has showcased some really good progress, it’s can still lag behind low-light image quality of Samsung and Huawei’s P20 Pro. The former’s bigger aperture is just a sheer hardware advantage, while the latter enormous sensor makes use of innovative image processing to really raise the bar in terms of extreme low light photography. Here the iPhone XS is good; but it just can’t keep up.

Camera - Daylight - More HDR & Portrait Camera Video Recording & Speaker Evaluation
Comments Locked

253 Comments

View All Comments

  • Andrei Frumusanu - Friday, October 5, 2018 - link

    Pixels and Mate 20 are next in line.
  • name99 - Friday, October 5, 2018 - link

    Hi Andrei,

    A few comments/questions.

    - the detailed Vortex and GPU die shots seem to bear no resemblance to the full SoC die shot. I cannot figure out the relationship no matter how I try to twist and reflect...

    Because I can't place them, I can't see the physical relationship of the "new A10 cache" to the rest of the SoC. If it's TIGHTLY coupled to one core, one possibility is value prediction? Another suggested idea that requires a fair bit of storage is instruction criticality tracking.

    If it's loosely coupled to both cores, one possibility is it's a central repository for prefetch? Some sort of total prefetching engine that knows the usage history of the L1s AND L2s and is performing not just fancy prefetch (at both L1s and L2s) but additional related services like dead block prediction or drowsiness prediction?
  • Andrei Frumusanu - Friday, October 5, 2018 - link

    The Vortex and GPU are just crops of the die shot at the top of the page. The Vortex shot is the bottom core rotated 90° counter-clockwise, and the GPU core is either top left or bottom right core, again rotated 90° ccw so that I could have them laid out horizontally.

    The "A10 cache" has no relationship with the SoC, it's part of the front-end.
  • name99 - Friday, October 5, 2018 - link

    OK, I got ya. Thanks for the clarification. I agree, no obvious connection to L2 and the rest of the SoC. So value prediction or instruction criticality? VP mostly makes sense for loads, so we'd expect it near LS, but criticality makes sense near the front end. It's certainly something I'm partial to, though it's been mostly ignored in the literature compared to other topics. I agree it's a long shot, but, like you said, what else is that block for?
  • name99 - Friday, October 5, 2018 - link

    "The benchmark is characterised by being instruction store limited – again part of the Vortex µarch that I saw a great improvement in."

    Can you clarify this? There are multiple possible improvements.
    - You state that A12 supports 2-wide store. The impression I got was that as of A11, Apple supported the fairly tradition 2-wide load/1-wide store per cycle. Is your contention that even as of A11, 2 stores/cycle were possible? Is there perhaps an improvement here along the lines of: previously the CPU could sustain 3 LS ops/cycle (pick a combination from up to 2 loads and up to 2 stores) and now it can sustain 4 LS ops/cycle?

    - Alternatively, are the stores (and loads) wider? As of A11, the width of one path to the L1 cache was 128 bits wide. It was for this reason that bulk loads and stores could run as fast using pair load-store integer as using vector load stores (and there was no improvement in using the multi-vector load-stores). When I spoke to some Apple folks about this, the impression I got was that they were doing fancy gathering in the load store buffers before the cache, and so there was no "instruction" advantage to using vector load/stores, whatever instruction sequence you ran, it would as aggressively and as wide as possible gather before hitting the cache. So if the LS queue is now gathering to 256 bits wide, that's going to give you double the LS bandwidth (of course for appropriately written, very dense back to back load/stores).

    - alternatively do you simply mean that non-aligned load/stores are handled better (eg LS that crossed cache lines were formerly expensive and now are not)? You'd hope that code doesn't do much of these, but nothing about C-code horrors surprises me any more...

    BTW, it's hard to find exactly comparable numbers, but
    https://www.anandtech.com/show/11544/intel-skylake...
    shows the performance of a range of different server class CPUs on SPEC2006 INT, compiled under much the same conditions. A12 is, ballpark, about at the level of Skylake-SP at 3.8 to 4GHz...
    (Presumably Skylake would do a lot better in *some* FP bcs of AVX512, but FP results aren't available.) This gives insight across a wider range of x86 servers than the link Andrei provided.
    The ideal would be to have SPEC2006 compiled using XCode for say the newest iMac and iMac Pro, and (for mobile space) MacBook Pro...
  • Andrei Frumusanu - Friday, October 5, 2018 - link

    > Is your contention that even as of A11, 2 stores/cycle were possible?

    Yes.

    > - Alternatively, are the stores (and loads) wider?

    Didn't verify, and very unlikely.

    > - alternatively do you simply mean that non-aligned load/stores are handled better

    Yes.
  • remedo - Friday, October 5, 2018 - link

    Can you please review the massive NPU? It seems like NPU deserves a lot more attention given the industry trend.
  • Andrei Frumusanu - Friday, October 5, 2018 - link

    I don't have any good way to test it at the moment.
  • Ansamor - Friday, October 5, 2018 - link

    Aren't these (https://itunes.apple.com/es/app/aimark/id137796825... tests cross-platform or comparable with the ones of Master Lu? I remember that you used it to compare the Kirin 970 against the Qualcomm DSP.
  • Andrei Frumusanu - Friday, October 5, 2018 - link

    Wasn't aware it was available on iOS, I'll look into it.

Log in

Don't have an account? Sign up now