Last week we detailed an article covering MediaTek’s seemingly widespread default inclusion of a benchmark whitelist in their chipset BSP (board support package) – a mechanism that enables more aggressive performance tuning of a device’s power management once it detects that a benchmark application is running.

Yesterday, UL, the developers of the PCMark and 3DMark benchmarking suites, have followed up on our investigation and analysed a wider range of devices, and have made the decision to temporarily delist all devices powered by a wide range of MediaTek SoCs, a list of over 50 devices from over 25 different vendors.

We had worked with UL early on in the investigation, with them providing us alternative anonymised versions of the benchmarks which bypass the whitelist detection, thus exposing the cheating behaviour.

The UL news post states:

Using hidden mechanisms to detect benchmarking apps by name and make app-specific performance optimizations is not an "accepted industry standard." It is, in fact, the very opposite of the accepted standard.

Likewise, benchmark scores based on hidden app-specific optimizations and settings that are enabled by default and not available to the user do not accurately reflect a device's true performance in everyday use. 

Simply put, a device must run a benchmark as if it was any other application. Performance gains must come from reacting to the nature of the workloads in the test rather than the name of the app itself.

As it has with similar cases in the past, we hope this delisting will help persuade MediaTek to change its approach and join the rest of the industry in adopting benchmarking best practices.”

The statement is a harsh rebuttal of MediaTek’s public response to our article, again pointing out that the practice is anything but an “industry standard” – especially damning since it’s coming from one of the major benchmark developers in the industry.

As UL stated in their blog post, and what we also pointed out in our original piece, we hope that the negative reactions to the matter will convince MediaTek to abandon such practices and reconsider their view of what the “industry standard” is.

Related Reading:

Source: UL News Post

POST A COMMENT

19 Comments

View All Comments

  • zmatt - Thursday, April 16, 2020 - link

    Ouch. Good to know Underwriter's Labs still isn't anyone's fool. Reply
  • Jorgp2 - Thursday, April 16, 2020 - link

    Is it actually Underwriters Lab? Reply
  • FunBunny2 - Thursday, April 16, 2020 - link

    yes Reply
  • shabby - Thursday, April 16, 2020 - link

    I preferred madonion... Reply
  • JKflipflop98 - Friday, April 17, 2020 - link

    To be fair, I'm sure they have more than one lab. . . Reply
  • katherinepdix52 - Saturday, April 25, 2020 - link

    Make 6150 bucks every month... Start doing online computer-based work through our website. I have been working from home for 4 years now and I love it. I don't have a boss standing over my shoulder and I make my own hours. The tips below are very informative and anyone currently working from home or planning to in the future could use this website.........www.iCash68.Ⅽom Reply
  • Hamm Burger - Thursday, April 16, 2020 - link

    Two thumbs up, Andrei! Reply
  • ballsystemlord - Thursday, April 16, 2020 - link

    ++ Reply
  • Makaveli - Thursday, April 16, 2020 - link

    Shame them into changing I like it. Reply
  • NICOXIS - Thursday, April 16, 2020 - link

    Nice, a good old slap in the hands Reply

Log in

Don't have an account? Sign up now