System Performance

As we’ve extensively covered in various over the last year and more, CPU performance alone doesn’t signify all that much if the system isn’t able to properly take advantage of it in order to achieve better user experience.

Software here plays and incredibly important role, and we’ve seen some devices fall face flat in this regard. Last year’s Exynos 9810 powered Galaxy devices in particular made it abundantly clear just how much of a user experience difference this can make, vastly overshadowing the actual hardware performance.

For the Galaxy S10, we expand on our initial MWC coverage regarding system performance between the two chipset variants. Here we saw clear indications that the Snapdragon 855 variant would again win in these set of benchmarks, and likely end up as the better device in terms of user experienced performance.

We’ve retested the numbers with our own in-house devices, so let’s take a look if and how things have changed:

PCMark Work 2.0 - Web Browsing 2.0

Starting off with the PCMark web-browsing workloads, the Snapdragon 855 variant of the Galaxy S10 leads the benchmark scores. Even though we’re seeing the Qualcomm chipset in a production device now, it still looks to lose out to the Kirin 980 devices. Both Samsung and Huawei now have “performance” modes in the device settings. For Huawei, as per company product management explanations, this is actually the intended full performance of the device while the default mode is slightly more battery friendly for the mass-market users who aren’t as performance sensitive.

It’s not yet exactly clear what Samsung’s performance mode changes compared to the “optimised” default setting, however I’ve also seen that this latter setting can result in throttled performance which actually results in Snapdragon performance numbers falling back to the range of the Exynos 9820 figures.

Samsung’s new “CPU Limiter” for the Galaxy S10 now works fundamentally differently than last year; it doesn’t actually limit the peak frequency the CPU can reach,  but rather limits the total CPU capacity to 70% in the scheduler, meaning multi-threaded workloads will be throttled. Frankly I find this silly as the device doesn’t inherently become more efficient as it still has access to the higher frequencies, and thus the battery advantage of this function isn’t nearly as great as what users will have experienced with the Galaxy S9.

Correction edit: The CPU Limiter does seem to work similarly and limiting CPU frequencies, however for some reason its activation and effect is seemingly delyed and isn't immediate following a change in settings.

PCMark Work 2.0 - Video Editing

The video editing workload is still something dominated by Qualcomm. The performance here seems to be dictated by the responsiveness of the little cores. In absolute terms, the differences aren’t big and this part of PCMark isn’t very indicative of overall system performance.

PCMark Work 2.0 - Writing 2.0

The one test that is actually most indicative of experienced device responsiveness is the Writing 2.0 workloads. Here the Snapdragon 855 falls in at the top of the performance charts along the Kirin 980. The Exynos 9820 Galaxy S10 also does relatively well here, but just falls short of the competition.

What is however most important is that this is now the first Exynos chipset in several years where S.LSI was able to deliver a scheduler and DVFS system that wasn’t absolutely abysmal, as evidenced by the massive performance difference between the Galaxy S10 and past Exynos devices at the very bottom of the chart.

PCMark Work 2.0 - Photo Editing 2.0

The Photo Editing workload again showcases similar improvements on the part of the Exynos. Here Qualcomm still has a very large performance advantage which simply might be due to better GPU acceleration for the RenderScript workloads which are part of the photo filter editing in this test.

PCMark Work 2.0 - Data Manipulation

Finally the Data manipulation workload sees the Snapdragon Galaxy S10 again at the top of the charts, with the Exynos 9820 slightly trailing it.

PCMark Work 2.0 - Performance

Overall, the Snapdragon 855 Galaxy S10 ends up as our top performing phone in PCMark. The Exynos 9820 unit trails behind, however does showcase significant improvements over previous generation Exynos phones.

Web JS Benchmarks

Switching over to web-browser based benchmarks, we’re testing inside a simple OS WebView shell as this is most representative of user-experience when browsing websites through third-party apps.

Speedometer 2.0 - OS WebView

In Speedometer 2.0, both Galaxy S10 devices are neck-in-neck within margins of error. The performance improvements of the Snapdragon 855 over last year’s Snapdragon 845 in this test seems relatively conservative, the Kirin 980 clearly is able to showcase a bigger performance lead.

The Exynos 9820 showcases dramatic performance gains over last year’s Exynos 9810. The removal of the abysmal hot-plugging mechanism means that the big CPU cores are able to actually run at full speed all while having secondary threads on the other big and middle cores.

WebXPRT 3 - OS WebView

In WebXPRT 3, the Snapdragon 855 manages to match the Kirin 980 while Exynos 9820 trails slightly behind.

JetStream 2 - OS WebView

Finally, I wanted to add in results of the brand-new JetStream 2 test that was released just two days ago and we’ll likely looking to adopt in the future (after more careful analysis). An interesting aspect of this test is that it’s using web workers to parallel workloads, something we usually never had in the past for browser-based JavaScript benchmarks.

Here the Exynos 9820 has a bigger disadvantage than the Snapdragon 855. The fact that the Kirin 980 scores identical to the Snapdragon means the performance shouldn’t be linked to the lower performance middle cores, but still strongly dependent on the big core performance. Unfortunately this seems to be another benchmark that doesn’t agree with Samsung’s CPU microarchitecture, with the Exynos S10 falling behind by 22%, even scoring less than the Snapdragon 845 of last year. This workload also doesn’t seem like a constant sustained test so it’s likely that scheduler responsiveness will play a role.

Scheduler & DVFS responsiveness

To investigate scheduler responsiveness and device DVFS settings, we fall back to our scaling performance ramp test. This is a fixed instruction chain workload with fine-grained timing collection every certain amount of instructions. By converting the time taken for every instruction block we can convert this into the frequency of the resident CPU that the workload is currently scheduled on, giving us detailed frequency information over time.

We’re looking at both Galaxy S10 units as well as the Mate 20 with the Kirin 980. For the new Qualcomm chipset what stands out is that the Galaxy S10 is indeed more aggressive in its scaling than what we’ve seen in January on the Qualcomm reference platform, reaching peak performance in 67ms rather than 95ms. It’s interesting that now even though the Qualcomm chipset has a clear scheduler and DVFS speed advantage over the Kirin 980, it still only is able to match or slightly lose out to the HiSilicon chip.

The results of the Exynos 9820 aren’t nearly as performant as on the Snapdragon Galaxy S10. Here we’re seeing that the chip first scales and resides on the small A55 cores for up to 83ms before it switches over to the Cortex A75 cores. What is extremely weird here is that the workload is staying on the middle cores for a mere 15ms before it switches over to the big M4 cores, finally reaching peak performance at around the 143ms mark, essentially twice as slow as the Snapdragon chipset.

We’ve seen a similar story last year with the Exynos 9810, and the issue is inherently tied to the scheduler. Make no mistake here, the Exynos 9820 behaves infinitely better than the Exynos 9810, however I expected more of the new chip. One of the things I did last year was to introduce two new big changes to the scheduler’s load tracking mechanism; halving the PELT half-life from 32ms down to 16ms which by itself doubled the responsiveness. On top of that I’ve added util_est to increase performance of short periodic workloads that otherwise would have lost their load utilisation faster.

For the Exynos 9820, what Samsung did was essentially also adopt these two important changes… and that’s about it. Although the new chipset comes with a new scheduler that does make efforts to schedule things around in an energy efficient way, the core issue of the scheduler load being slow hasn’t been further improved beyond what I myself was able to achieve last year. Currently the Exynos 9820 is the only flagship Android SoC that still uses PELT in its scheduler as a load tracking mechanism as both Qualcomm and HiSilicon are making use of WALT, which is massively more responsive. Google actually wants to drop WALT out of the Android Common Kernel, however this happened only after PELT was made to be as responsive as WALT. One very important patch to achieve this is unfortunately missing from the Exynos 9820’s BSP which means as a delivered product the Exynos Galaxy S10 just has lower responsiveness. In next year’s Exynos we’ll probably finally see things equal out, however this will by then be 4 generations and years of Qualcomm SoCs being superior and giving better user experience simply because they have the better software stack.

That being said, it’s not all doom and gloom for the new Exynos 9820 Galaxy S10. What the scheduler lacks is actually made up by touch boosting as well as Android framework integrated boosters which are triggered by activity switches. These mechanisms actually help out a lot the user experience of the Exynos Galaxy S10 beyond what we can actually measure in standalone benchmarks such as PCMark or the web tests. In my subjective experience with both phones, yes the Snapdragon unit was slightly faster, but if I didn’t have both devices side by side to compare, it would be have been something quite hard to notice. What is important is that the experience on the Exynos 9820 is leagues ahead of what we’ve seen in the Exynos 9810 devices and past chipsets. Some of these OS-side boosters seem to have made it into the Android P update of Exynos 9810 devices, so while the kernel as remained largely unchanged, at least this part benefits last year’s devices.

Overall, both the Snapdragon 855 and Exynos 9820 Galaxy S10 give among the very best performance experiences among current Android phones, even if the latter has some rough edges here and there.

Inference Performance: APIs, Where Art Thou? GPU Performance & Power
POST A COMMENT

228 Comments

View All Comments

  • GreenMeters - Monday, April 1, 2019 - link

    "If you’re a reader in the US or other Snapdragon markets, you can stop reading here and feel happy about your purchase or go ahead and buy the Galaxy S10+."

    Unfortunately, no, you can't feel happy about it, because once again the Snapdragon variant has its bootloader locked. So your expensive purchase that could easily have a 5+ year lifespan with an open source OS providing up-to-date security and features is now artificially limited to 2 years of Samsung's lousy support.
    Reply
  • XelaChang - Monday, April 1, 2019 - link

    Quite disappointing for Exinos, especially the audio. Going to look into Huawei P30 instead. Reply
  • Quantumz0d - Monday, April 1, 2019 - link

    Hello Andrei, huge thanks for the solid piece. I don't think there are any editors out who does this type of analysis. The most superb part was the battery analysis, just fantastic. I remember your piece on the Note 9 as well.

    Because smartphones with soldered/sealed batteries are a pain with 2 Yr EOL of cycles due to aggressive current/power/volts/cycles. I wish when you cover the LG. Maybe kindly have a look at their Qnovo. Replacing at end user is so bad, ruins the IP rating and hard to source. Samsung improvising this is a really good news.

    Next the Camera Hole points all are valid. Its worse than a notch with that absurdly thicc status bar and the stupid icons on the right side. An eye sore with dead pixels. Samsung showed in China for an under screen camera perhaps the Vertical integration you mentioned due to the Exynos applies here as well, perhaps the cost as well..

    One UI though perhaps feels polished but its too childish/kid friendly to me and excessively rounded like iOS instead of stock Pie/Q, that is bad IMO.

    Still have to read up on the Camera/Display. Also I think you should mention one great advantage that Exynos has - Bootloader Unlock. Without that QSD version is just a paperweight, zero ownership, zero tuning. IMO a brick.

    Also good to hear about the speaker system performance, apple mentions it always its surprising how they didn't yet offered a good quality, finally those AKG buds are very very bad. I heard them, their tiny driver is horrible in low end and mid range, its shameful. I'm not up to date with recent audio progress but at $100 we can get RHA MA 750/ Final Audio / iBasso IT01/ TFZ King II / Mee Audio P1 / FiiO F9 and Pro / Dunu Titan1 and ton of IEMs with far more superior quality.

    I hope they get the damn AKM chips into their phones and compete with LG ESS and take the Audio seriously, its a shame that LG doesn't advertise ESS anymore but Meridian collaboration.

    Finally the Audio DAC part, sometimes being 100% accurate doesnt necessarily mean best, my iPod 5.5G Wolfson DAC before CL merger many people say the iPod 6G+ Classics are better due to the ball roll off they mention on the Wolfson 5.5G DAC, I have both of them running same OS (Aftermarket stable Linux based Rockbox) and the Cirrus Logic G classic sounds fatiguing to me, metallic and lack of thump vs the 5.5G. I think maybe your impression is also similar. I heard the 835s Acoustic in my Car with my friend's Note 8 US version and it was hollow and lack of any texture and rumble. The iPod beats it by a HUGE margin both of th 6G and 5.5G and 5.5G being better, the V30s ESS sounds more balanced vs the 5.5G as in clear at the expense of soundstage (in car more significant) and sharpness being higher but retains excellent sub bass. All this is subjective. Just to let you know..

    Thanks.
    Reply
  • Quantumz0d - Monday, April 1, 2019 - link

    Correction.

    > Apple mentions it always its surprising how they didn't yet.

    Apple stands at top as one of the best speakers on an iPad/iPhone.
    Reply
  • Andrei Frumusanu - Tuesday, April 2, 2019 - link

    The iPhone XS improved it, but the S10 beats it handily in speaker quality. Reply
  • Quantumz0d - Tuesday, April 2, 2019 - link

    Wow, that's really surprising and great news. Thank you for the information. I'll stop by to a Best buy near to me and check it out.

    Perhaps they'll improve on their new S5 855 Tablet (hopefully with jack, unlike S5e) because the Tab S4 is outright beaten to pulp by iPad Pro 2017.
    Reply
  • s.yu - Friday, April 5, 2019 - link

    Wow beating Apple at audio is definitely something special. Reply
  • Quantumz0d - Monday, April 1, 2019 - link

    Damn another typo

    >Ball roll off

    Its Bass Roll off. And 6G missing before "classic sounds"
    Reply
  • watersb - Tuesday, April 2, 2019 - link

    What an incredible opportunity to compare two leading SoC architectures. Reply
  • TechieSandy - Tuesday, April 2, 2019 - link

    The difference in the Dynamic Range between the Snapdragon and the Exynos variant may have less to do with processing than the triggering by the camera. In one of the pics the Exynos proxessor has better details in the shadows and more dynamic range.
    In this photo:
    https://images.anandtech.com/galleries/6981/S10PS_...

    It may be something triggering the HDR algorithm. It may have to do with it.
    Reply

Log in

Don't have an account? Sign up now