System Performance

As we’ve extensively covered in various over the last year and more, CPU performance alone doesn’t signify all that much if the system isn’t able to properly take advantage of it in order to achieve better user experience.

Software here plays and incredibly important role, and we’ve seen some devices fall face flat in this regard. Last year’s Exynos 9810 powered Galaxy devices in particular made it abundantly clear just how much of a user experience difference this can make, vastly overshadowing the actual hardware performance.

For the Galaxy S10, we expand on our initial MWC coverage regarding system performance between the two chipset variants. Here we saw clear indications that the Snapdragon 855 variant would again win in these set of benchmarks, and likely end up as the better device in terms of user experienced performance.

We’ve retested the numbers with our own in-house devices, so let’s take a look if and how things have changed:

PCMark Work 2.0 - Web Browsing 2.0

Starting off with the PCMark web-browsing workloads, the Snapdragon 855 variant of the Galaxy S10 leads the benchmark scores. Even though we’re seeing the Qualcomm chipset in a production device now, it still looks to lose out to the Kirin 980 devices. Both Samsung and Huawei now have “performance” modes in the device settings. For Huawei, as per company product management explanations, this is actually the intended full performance of the device while the default mode is slightly more battery friendly for the mass-market users who aren’t as performance sensitive.

It’s not yet exactly clear what Samsung’s performance mode changes compared to the “optimised” default setting, however I’ve also seen that this latter setting can result in throttled performance which actually results in Snapdragon performance numbers falling back to the range of the Exynos 9820 figures.

Samsung’s new “CPU Limiter” for the Galaxy S10 now works fundamentally differently than last year; it doesn’t actually limit the peak frequency the CPU can reach,  but rather limits the total CPU capacity to 70% in the scheduler, meaning multi-threaded workloads will be throttled. Frankly I find this silly as the device doesn’t inherently become more efficient as it still has access to the higher frequencies, and thus the battery advantage of this function isn’t nearly as great as what users will have experienced with the Galaxy S9.

Correction edit: The CPU Limiter does seem to work similarly and limiting CPU frequencies, however for some reason its activation and effect is seemingly delyed and isn't immediate following a change in settings.

PCMark Work 2.0 - Video Editing

The video editing workload is still something dominated by Qualcomm. The performance here seems to be dictated by the responsiveness of the little cores. In absolute terms, the differences aren’t big and this part of PCMark isn’t very indicative of overall system performance.

PCMark Work 2.0 - Writing 2.0

The one test that is actually most indicative of experienced device responsiveness is the Writing 2.0 workloads. Here the Snapdragon 855 falls in at the top of the performance charts along the Kirin 980. The Exynos 9820 Galaxy S10 also does relatively well here, but just falls short of the competition.

What is however most important is that this is now the first Exynos chipset in several years where S.LSI was able to deliver a scheduler and DVFS system that wasn’t absolutely abysmal, as evidenced by the massive performance difference between the Galaxy S10 and past Exynos devices at the very bottom of the chart.

PCMark Work 2.0 - Photo Editing 2.0

The Photo Editing workload again showcases similar improvements on the part of the Exynos. Here Qualcomm still has a very large performance advantage which simply might be due to better GPU acceleration for the RenderScript workloads which are part of the photo filter editing in this test.

PCMark Work 2.0 - Data Manipulation

Finally the Data manipulation workload sees the Snapdragon Galaxy S10 again at the top of the charts, with the Exynos 9820 slightly trailing it.

PCMark Work 2.0 - Performance

Overall, the Snapdragon 855 Galaxy S10 ends up as our top performing phone in PCMark. The Exynos 9820 unit trails behind, however does showcase significant improvements over previous generation Exynos phones.

Web JS Benchmarks

Switching over to web-browser based benchmarks, we’re testing inside a simple OS WebView shell as this is most representative of user-experience when browsing websites through third-party apps.

Speedometer 2.0 - OS WebView

In Speedometer 2.0, both Galaxy S10 devices are neck-in-neck within margins of error. The performance improvements of the Snapdragon 855 over last year’s Snapdragon 845 in this test seems relatively conservative, the Kirin 980 clearly is able to showcase a bigger performance lead.

The Exynos 9820 showcases dramatic performance gains over last year’s Exynos 9810. The removal of the abysmal hot-plugging mechanism means that the big CPU cores are able to actually run at full speed all while having secondary threads on the other big and middle cores.

WebXPRT 3 - OS WebView

In WebXPRT 3, the Snapdragon 855 manages to match the Kirin 980 while Exynos 9820 trails slightly behind.

JetStream 2 - OS WebView

Finally, I wanted to add in results of the brand-new JetStream 2 test that was released just two days ago and we’ll likely looking to adopt in the future (after more careful analysis). An interesting aspect of this test is that it’s using web workers to parallel workloads, something we usually never had in the past for browser-based JavaScript benchmarks.

Here the Exynos 9820 has a bigger disadvantage than the Snapdragon 855. The fact that the Kirin 980 scores identical to the Snapdragon means the performance shouldn’t be linked to the lower performance middle cores, but still strongly dependent on the big core performance. Unfortunately this seems to be another benchmark that doesn’t agree with Samsung’s CPU microarchitecture, with the Exynos S10 falling behind by 22%, even scoring less than the Snapdragon 845 of last year. This workload also doesn’t seem like a constant sustained test so it’s likely that scheduler responsiveness will play a role.

Scheduler & DVFS responsiveness

To investigate scheduler responsiveness and device DVFS settings, we fall back to our scaling performance ramp test. This is a fixed instruction chain workload with fine-grained timing collection every certain amount of instructions. By converting the time taken for every instruction block we can convert this into the frequency of the resident CPU that the workload is currently scheduled on, giving us detailed frequency information over time.

We’re looking at both Galaxy S10 units as well as the Mate 20 with the Kirin 980. For the new Qualcomm chipset what stands out is that the Galaxy S10 is indeed more aggressive in its scaling than what we’ve seen in January on the Qualcomm reference platform, reaching peak performance in 67ms rather than 95ms. It’s interesting that now even though the Qualcomm chipset has a clear scheduler and DVFS speed advantage over the Kirin 980, it still only is able to match or slightly lose out to the HiSilicon chip.

The results of the Exynos 9820 aren’t nearly as performant as on the Snapdragon Galaxy S10. Here we’re seeing that the chip first scales and resides on the small A55 cores for up to 83ms before it switches over to the Cortex A75 cores. What is extremely weird here is that the workload is staying on the middle cores for a mere 15ms before it switches over to the big M4 cores, finally reaching peak performance at around the 143ms mark, essentially twice as slow as the Snapdragon chipset.

We’ve seen a similar story last year with the Exynos 9810, and the issue is inherently tied to the scheduler. Make no mistake here, the Exynos 9820 behaves infinitely better than the Exynos 9810, however I expected more of the new chip. One of the things I did last year was to introduce two new big changes to the scheduler’s load tracking mechanism; halving the PELT half-life from 32ms down to 16ms which by itself doubled the responsiveness. On top of that I’ve added util_est to increase performance of short periodic workloads that otherwise would have lost their load utilisation faster.

For the Exynos 9820, what Samsung did was essentially also adopt these two important changes… and that’s about it. Although the new chipset comes with a new scheduler that does make efforts to schedule things around in an energy efficient way, the core issue of the scheduler load being slow hasn’t been further improved beyond what I myself was able to achieve last year. Currently the Exynos 9820 is the only flagship Android SoC that still uses PELT in its scheduler as a load tracking mechanism as both Qualcomm and HiSilicon are making use of WALT, which is massively more responsive. Google actually wants to drop WALT out of the Android Common Kernel, however this happened only after PELT was made to be as responsive as WALT. One very important patch to achieve this is unfortunately missing from the Exynos 9820’s BSP which means as a delivered product the Exynos Galaxy S10 just has lower responsiveness. In next year’s Exynos we’ll probably finally see things equal out, however this will by then be 4 generations and years of Qualcomm SoCs being superior and giving better user experience simply because they have the better software stack.

That being said, it’s not all doom and gloom for the new Exynos 9820 Galaxy S10. What the scheduler lacks is actually made up by touch boosting as well as Android framework integrated boosters which are triggered by activity switches. These mechanisms actually help out a lot the user experience of the Exynos Galaxy S10 beyond what we can actually measure in standalone benchmarks such as PCMark or the web tests. In my subjective experience with both phones, yes the Snapdragon unit was slightly faster, but if I didn’t have both devices side by side to compare, it would be have been something quite hard to notice. What is important is that the experience on the Exynos 9820 is leagues ahead of what we’ve seen in the Exynos 9810 devices and past chipsets. Some of these OS-side boosters seem to have made it into the Android P update of Exynos 9810 devices, so while the kernel as remained largely unchanged, at least this part benefits last year’s devices.

Overall, both the Snapdragon 855 and Exynos 9820 Galaxy S10 give among the very best performance experiences among current Android phones, even if the latter has some rough edges here and there.

Inference Performance: APIs, Where Art Thou? GPU Performance & Power
POST A COMMENT

228 Comments

View All Comments

  • Wardrive86 - Monday, April 1, 2019 - link

    Look forward to that! Reply
  • s.yu - Monday, April 1, 2019 - link

    "HDR10 non plus"
    lol
    That looks like French.
    Reply
  • JTBM_real - Monday, April 1, 2019 - link

    Samsung Galaxy S7 Edge - It is usable now. At the start the phone part did not work - it froze after a few hours and you did not receive calls and couldnt call anyone. Software issue. Headphone had an unbearable noise - software issue. Edge screen made it impossible to use it with one hand - could not receive a call in my car. Restart took ages and had to restart every few hours. There were other issues too. By today the phone is actually ok. There is a new sh*t on it "Smartthings running" - I don't care, just not show on the screen.

    I will never buy a new Samsung phone ever again. It is unusable in the first 2 years. The hardware is good though. No scratch until today. Huwai had two large ones after 1 week - my wife bought one. It had on paper the same gorilla glass as my phone. Yeah...
    Reply
  • twinuser01 - Monday, April 1, 2019 - link

    Would an unlocked device from the US or Canada work in the EU (Germany)? Reply
  • Andrei Frumusanu - Monday, April 1, 2019 - link

    Yes it would, although things like VoLTE or WiFi calling will not. Reply
  • nuttyprof1991 - Monday, April 1, 2019 - link

    Hi guys,
    I noticed a weird effect related to the gamma.
    So I installed the app called display tester from play store and ran the gamma test.
    During the test i found he gamma values to be very low and found it a bit strange.Then again I checked after one day and noticed it was around 2.2 most of the time.

    During both the tests the only difference I noticed in my setting was the resolution which affected the tests soo much. Is this possible? The accuracy was perfect in max resolution of 3040 by 1440.
    Reply
  • Andrei Frumusanu - Monday, April 1, 2019 - link

    The visual gamma testers such as that app rely on correct 1:1 pixel mapping for the display panel and the comparison background interleaved pattern in the app, on smartphones this is hard to achieve due to how things are resolution independent. The app will not work correctly on pentile display nor will it work correctly in the resolution change scenario you describe. Reply
  • nuttyprof1991 - Monday, April 1, 2019 - link

    that makes sense. I was wondering if this could have resulted in the gamma problems explained in the review. Guess not Reply
  • agn53 - Monday, April 1, 2019 - link

    Unless I'm missing something, I don't see anything related with phone quality and performance (call stability and sound, signal strength, network speed, quality and range) or the UI (aesthetics and ergonomics, speed and load, functionality) or even health related and other features.
    Isn't phone usage and graphical design a important requisite for this type of tech?...
    It's a phone, after all. But nevertheless a very comprehensive and factual review, almost perfect but yet so flawed ... :-)
    Reply
  • Andrei Frumusanu - Monday, April 1, 2019 - link

    We don't have any good methodology on things like signal, network (does any site test this *accurately*?).

    As for the UI bits, it's something I wanted to have in the piece but also didn't want to further delay the article another week. In general OneUI is Samsung's by far best user interface and has fantastic features without them feeling like gimmicks. It's currently in my opinion the best variant of Android, though I'm sure some Google users will get angry at me for saying that.
    Reply

Log in

Don't have an account? Sign up now