Last week we detailed an article covering MediaTek’s seemingly widespread default inclusion of a benchmark whitelist in their chipset BSP (board support package) – a mechanism that enables more aggressive performance tuning of a device’s power management once it detects that a benchmark application is running.

Yesterday, UL, the developers of the PCMark and 3DMark benchmarking suites, have followed up on our investigation and analysed a wider range of devices, and have made the decision to temporarily delist all devices powered by a wide range of MediaTek SoCs, a list of over 50 devices from over 25 different vendors.

We had worked with UL early on in the investigation, with them providing us alternative anonymised versions of the benchmarks which bypass the whitelist detection, thus exposing the cheating behaviour.

The UL news post states:

Using hidden mechanisms to detect benchmarking apps by name and make app-specific performance optimizations is not an "accepted industry standard." It is, in fact, the very opposite of the accepted standard.

Likewise, benchmark scores based on hidden app-specific optimizations and settings that are enabled by default and not available to the user do not accurately reflect a device's true performance in everyday use. 

Simply put, a device must run a benchmark as if it was any other application. Performance gains must come from reacting to the nature of the workloads in the test rather than the name of the app itself.

As it has with similar cases in the past, we hope this delisting will help persuade MediaTek to change its approach and join the rest of the industry in adopting benchmarking best practices.”

The statement is a harsh rebuttal of MediaTek’s public response to our article, again pointing out that the practice is anything but an “industry standard” – especially damning since it’s coming from one of the major benchmark developers in the industry.

As UL stated in their blog post, and what we also pointed out in our original piece, we hope that the negative reactions to the matter will convince MediaTek to abandon such practices and reconsider their view of what the “industry standard” is.

Related Reading:

Source: UL News Post

POST A COMMENT

18 Comments

View All Comments

  • mode_13h - Friday, April 17, 2020 - link

    Great news, guys! Thanks for blowing the whistle on these cheaters. Reply
  • chipped - Saturday, April 18, 2020 - link

    Good on your guys! Reply
  • eastcoast_pete - Saturday, April 18, 2020 - link

    Great job, Andrei! Result! MTK might finally see the light and stop this stupid software-juicing. Especially as they now seem to finally have SoCs that won't need to cheat to perform well (D1000). Reply
  • GreenReaper - Saturday, April 18, 2020 - link

    I fear UL has overplayed its hand. MediaTek doesn't need them to sell to its customers. But I guess we'll soon see who has the real power in their relationship. Reply
  • tommythorn - Sunday, April 19, 2020 - link

    They don't have a choice. If they hadn't delisted MediaTekCheat they would have lost all credibility which arguably their most valuable asset. Reply
  • mode_13h - Saturday, April 25, 2020 - link

    Agreed. UL's reputation extends well beyond this benchmark and is something worth protecting. Reply
  • Koenig168 - Sunday, April 19, 2020 - link

    Well done, AT & UL. Good to know that I can continue to trust my 3DMark suite. Reply
  • Sivar - Monday, April 20, 2020 - link

    Good for UL, doing the right thing for the integrity of the industry (and their reputation).

    Also, this is not the first time this has happened. Never forget:
    https://www.anandtech.com/show/7384/state-of-cheat...
    Reply

Log in

Don't have an account? Sign up now