The Snapdragon 8 Gen 1 Performance Preview: Sizing Up Cortex-X2
by Dr. Ian Cutress on December 14, 2021 8:00 AM ESTMachine Learning: MLPerf and AI Benchmark 4
Even as a new benchmark in the space, MLPerf has been made available that runs representative workloads on devices and takes advantage of both common ML frameworks such as NNAPI as well as the respective chip libraries for each vendor. Using this benchmark on retail phones to date, Qualcomm has had the lead in almost all the tests, but given that the company is promoting a 4x increase in AI performance, it will be interesting to see if that comes across all of MLPerf’s testing scenarios.
It should be noted that Apple’s CoreML is currently not supported, hence the lack of Apple numbers here.
Across the board in these first four tests Qualcomm is making a sizable lead, going above and beyond what the S888 can do. Here we’re seeing up to a 2.2x result, making an average +75% gain. It’s not quite the 4x that Qualcomm promoted in its materials, but there’s a sizable gap with the other high-end silicon we’ve tested to date.
The only non-lead is with the language processing, where Google’s Tensor SoC is almost 2x what the S8g1 scores. This test is based on a mobileBERT model, and either for software or architecture reasons, it fits a lot better into the Google chip than any other. As smartphones increase their ML capabilities, we might see some vendors optimizing for specific workloads over others, like Google has, or offering different accelerator blocks for different models. The ML space is also fast paced, so perhaps optimizing for one type of model might not be a great strategy long-term. We will see.
In AI Benchmark 4, running in pure NNAPI mode, the Qualcomm S8g1 takes a comfortable lead. Andrei noted in previous reviews with this test that the power consumed during this test can be quite high, up to 14 W, and this is where some chips might be able to pull ahead an efficiency advantage. Unfortunately we didn’t record power at the same time as the test, but it would be good to monitor this in the future.
Facebook
Twitter
RSS
169 Comments
View All Comments
Fulljack - Wednesday, December 15, 2021 - link
resale value is cool and all but not everyone bought a phone to be sold later down the road. just like how some people simply didn't like iOS and it's limitations.michael2k - Thursday, December 16, 2021 - link
Some people use the extended life of an iPhone for other things; that’s part of why they have high resale value.In other words people hand down three year old iPhones to kids or spouses. Some people want to keep their iPhones. Those people, as an example, refusing to let their iPhones go for sale increase the resale price by reducing supply in the face of demand.
iphonebestgamephone - Tuesday, December 14, 2021 - link
Iphones are pretty bad for gaming for more than a few minutes unless you have very low ambient temps or are willing to use a plugged to the wall phone cooler. Something like the redamgic or legion are better for mobile 'gamers'.hemedans - Wednesday, December 15, 2021 - link
I disagree, if you play games get android gaming phones, they can sustain perfomance compare to iphones which throttle a lot and mislead in lot of metrics (like operate in 120fps when you are on menu and drop to 60fps when you fight)Also with Android you get more games options, like now Ps2 emulator has been released, you wont find something like this in ios, also lot of pc games have been ported to Android, you can enjoy them on go.
yankeeDDL - Tuesday, December 14, 2021 - link
As an Android user, the results in Spec CPU 2017 are depressing.In every single test the X2 is far behind A15 (sometimes by nearly 2x) and yet in every single one the A15 uses less power. It is a bloodbath.
I realize the synthetic tests are not necessarily representative of use case, but it sure looks dismal for Qualcomm. They are extraordinarily behind.
Imagine if the next Galaxy S22 would use the A15: it would be 2-3 years ahead of all competition.
The lack of competition in high-end ARM-based CPU is just mind-boggling.
Wrs - Tuesday, December 14, 2021 - link
Yeah it seems very incremental on the CPU. They need a revolution just to catch up. But their GPU and NN seem like bigger leaps. Unfortunately both are more limited use cases - gaming and speech recognition primarily.Kangal - Sunday, December 19, 2021 - link
With the way Qualcomm has followed with the Wattage, they've only been achieving better and better scores at the expense of battery and heat. Basically they've embraced throttling. And that's not an acceptable solution for phones.There's little to no performance difference between the QSD 855 to its siblings*. And that seems to be the case with the new Snapdragon 8gen1 for 2022. The biggest difference I can see is at the highest-end emulation, which is to run the latest console, with the highest graphical title, and scale up resolution as appropriate. Currently, this is the Nintendo Wii, running Metroid 3 Prime, and upscaled to x4 (1440p). On the fastest Android Phones (eg ASUS RoG 5) it runs around 50-60fps depending on the scene. On something like a Samsung S10+ you're instead getting 40-50fps. And the bigger improvement can come from the natural progression of the emulator itself (ie 2018 vs 2021), and by running Mods on it like MMJ. So these things can/do bridge the performance difference that I stated above.
What might be a good argument to be made, is when we get other more powerful/demanding use cases. Such as comparing Nintendo Switch emulation, running desktop programs, or even getting a new AAA-game that was developed for the handheld (eg GTA 6). Anyways, we will see a separation from current performance tier, at the earliest with 2023 using second-generation ARM Europe microarchitecture. Android hasn't quite caught up to Apple A13 levels of performance yet, and that's what I think when we will surpass it (Apple's A14 and A15 aren't that much better than their A13 either).
*855+, 860, 778, 780, 865, 865+, 870, 888, 888+.
Wereweeb - Tuesday, December 14, 2021 - link
When I can use it as a desktop replacement I'll start to care. Until then, all you get with a "muh faster CPU" is your apps take three nanoseconds less time to open.My next phone will be the GAA-FET (3-2nm) successor to the 780G, for the better energy-efficiency. If it takes five years, so be it. I don't game so I don't have a use for X2 cores.
1_rick - Tuesday, December 14, 2021 - link
"When I can use it as a desktop replacement I'll start to care."DeX, dude. It's already good enough for light work.
TheinsanegamerN - Tuesday, December 14, 2021 - link
Also limited to samsung devices. Perhaps Wereweeb has no interest in owning one of their devices. There are many that have no interest in owning a samsung based on past experiences.