A few days ago we published our article addressing Huawei and Honor’s inaccurate benchmarking behaviour. In a nutshell, we had found out that this year’s devices had introduced a new thermal limiting behaviour that quickly throttled power consumption to ~4-4.5W in 3D workloads. While this in itself wasn’t an issue, the problem is that the firmware did not apply this new behaviour to a specific list of whitelisted 3D benchmarks.

Sidenote: Earlier this year, Futuremark (The 3DMark creators) had been acquired by UL, joining the global safety consulting and certification company as a benchmarking division.

We work closely with all benchmark vendors, and UL isn’t an exception. We had first given UL note of the behaviour two weeks ago and had been sharing our early results with the development team. Today UL published their independent confirmation of our results on their own devices, and have subsequently decided to remove the tested devices from their results database.

UL stresses that the kind of detection and optimization performed by Huawei infringes the company’s rules for manufacturers. Unfortunately this isn’t the first instance of a vendor being delisted, as most famously a slew of phone manufacturer had been delisted in 2013 after a more in-depth investigation of ours resulted in quite embarrassing results for a lot of vendors.

The UL team further explains that they’ve also been in touch with Huawei, and that the Chinese vendor is planning to address the behaviour by introducing a new “performance mode” that disables the new thermal throttling behaviour. In essence this mode would revert back to the behaviour we’ve seen in the past such as the Mate 9 – where the SoC is allowed higher peak performance figures at a cost of high power.

While this is still problematic for representative benchmarking for devices with Kirin 960’s and 970’s (due to the very large peak power at the highest performance states of those SoCs), I expect this to be a non-issue for the new Kirin 980, as it’s projected to have significantly better sustainable peak power figures. We're still looking forward to the new SoC in the upcoming Mate 20, and believe it to be an outstanding performer.

Source: UL News Release

POST A COMMENT

33 Comments

View All Comments

  • bji - Thursday, September 06, 2018 - link

    Actually, you are seeing *fewer* and *fewer* posts without capitals. Less/fewer misuse is also annoying!

    "Less" is for uncountable things, like water. So you can have "less water", but not "fewer water".

    "Fewer" is for countable things, like "cups of water". So you can have "fewer cups of water", but not "less cups of water".
    Reply
  • grahad - Monday, September 10, 2018 - link

    Philosoraptor: Can you truly count the number of posts on the internet? Kappa Reply
  • damianrobertjones - Thursday, September 13, 2018 - link

    When I visit the MS Word thesaurus, to view the word 'less', the first description is 'fewer' (A reduced amount).

    Good info though. Thanks.
    Reply
  • Morawka - Thursday, September 06, 2018 - link

    Says the guy who can't even capitalize his own sentences. Also "who the hell do you do this" makes no sense. Reply
  • thexile - Thursday, September 06, 2018 - link

    Clearly you are the dumb one. Reply
  • Manch - Friday, September 07, 2018 - link

    Zmeul bitches about the title, as he writes a comment that would bleed red if corrected. LOL Reply
  • spartaman64 - Sunday, September 09, 2018 - link

    i missed you Reply
  • casperes1996 - Thursday, September 06, 2018 - link

    I love the fact that so far, literally every single one of the 9 comments here is about grammar, and correcting what I presume to be a troll, and nobody talks about the actual article Reply
  • bubblyboo - Thursday, September 06, 2018 - link

    Diversion created by a Huawei employee? Reply
  • Holliday75 - Thursday, September 06, 2018 - link

    Seems par for the course in today's social media environment. Reply

Log in

Don't have an account? Sign up now