The HTC One (M8) Review
by Anand Lal Shimpi & Joshua Ho on March 26, 2014 7:00 PM EST- Posted in
- Smartphones
- HTC
- Mobile
- HTC One
Subtle Cheating: New Benchmark Optimizations
We’ve been tracking the state of benchmark optimizations among Android OEMs for a while now. It wasn’t too long ago that we published a piece calling out nearly all Android OEMs for optimizing for benchmarks. The optimizations are pretty crude. Upon detecting a whitelisted benchmark by APK identifier alone, most OEMs would enable a performance mode that would:
1) Plug in all CPU cores
2) Run all cores at max frequency
3) Raise thermal limits to eliminate/reduce throttling
Defeating the optimization was just as simple: thorough renaming of the benchmark and all internal application IDs. For the past several months we’ve been working with benchmark, silicon and handset vendors to curb the behavior. Although we found the optimizations to have minimal impact on our test results, it’s still a messy practice that isn’t worth doing.
We’ve seen early (encouraging) indications that some vendors have reconsidered their position on benchmark optimizations. Unfortunately HTC isn’t quite there yet.
The M8’s Android 4.4.2 build includes a new, more subtle form of benchmark optimization that we hadn’t seen in previous devices. Benchmarks are still detected according to their application identifier, but instead of hot plugging in all CPU cores and driving them to max frequencies, everything appears to be normal at launch.
Here’s the state of the CPU cores after launching the Play Store version of any optimized benchmark:
Everything looks just fine. But look at what happens if we monitor CPU frequency over time on the Play Store and a special renamed version of 3DMark:
Average CPU frequency is about 15% higher while running the Play Store version of 3DMark. I still need to run some thermal analysis on the device but I don’t think HTC is raising thermal limits. Instead what appears to be happening is HTC is simply more aggressively tuning the governor response to performance demands, allowing for higher frequencies. Note that the frequency response latency is now so low that I couldn't even grab the 300MHz screenshot above in the Play Store version of 3DMark. As soon as the device detected a button press it would ramp up to 1.7GHz.
The impact on performance goes hand in hand with the increase in average clock speed. I measured performance during 3DMark’s Physics test (which is CPU bound). The difference was about 15%.
I also tracked GPU clock speed over time. Thankfully the optimization seems limited to CPU frequencies alone:
The list of optimization targets has also expanded since we last looked at HTC. The latest versions of GFXBench, BaseMark X and BaseMark OS II are now included in the benchmark whitelist.
HTC made one small concession - it’s allowing users the ability to run their device in this high performance mode at all times. Under developer tools (tap on the build number 5 times in Settings > About > Software information > More), you’ll find an option to enable high performance CPU mode. Checking that box will put your device in the same mode that’s enabled when a whitelisted benchmark is detected.
 |
 |
I do appreciate that HTC is exposing the optimization control, the only thing missing is the ability to toggle the benchmark optimization off (not to mention that I’d prefer if it was disabled to begin with). I fear that HTC’s justification in all of this is that everyone else is doing it so why opt out. The reality seems to be trending the other direction however. We’ll have to see what Samsung does with the Galaxy S 5, but I have a feeling that HTC is going to end up on the wrong side of history with this move. All of our benchmarks are already immune to the optimization, so it’s really a matter of sacrificing integrity for no real gain. There’s nothing more to say other than I’m disappointed.
Facebook
Twitter
RSS
222 Comments
View All Comments
thedenti5t - Thursday, March 27, 2014 - link
Got this phone today and absolutley love it. It is the best phone Ive ever used hands down. To answer 2 questions I keep seeing. 1) It does have blinkfeed but you can remove it from you screens. 2) The 3 buttons have been moved to the screen but dissapear when actually using an app or watching a movie. I dont care about the camera as much as others, but it does take nice pictures. You can also crop which is another point I keep seeing people make.vv007420 - Friday, March 28, 2014 - link
Ok I have HTC One (M7) which is around 9 months old.....its still fast and zippy but do you guys think it will be a worthy upgrade to M8...(mind you Im in India and there are no contracts here...we have to pay full price for the phone downright)HangFire - Friday, March 28, 2014 - link
I'm sticking with my M7. It gets down to, is there feature of the M8 you can't live without?asaini007 - Saturday, March 29, 2014 - link
I wouldn't say it's worth the upgrade. Wait till the M9 imosynaesthetic - Friday, March 28, 2014 - link
I'm going to reserve judgment until iFixit posts a teardown. The last HTC One was virtually impossible to disassemble without destroying it. I realize I probably sound silly, but I don't buy anything I can't repair myself.I feel very strongly about our electronic waste problem and I believe a good way to help reduce e-waste is to encourage manufacturers to make things that can be (relatively) easy to take apart and repair. Unfortunately this flies in the face of "planned obsolescence," so it'll take a lot of people to actually make it happen... :/
HangFire - Friday, March 28, 2014 - link
Do you rebuild your own automatic transmissions? Just curious.synaesthetic - Tuesday, April 1, 2014 - link
I don't own a car. :P When I did own a car, I had a manual and yes, I either did the work on it myself or had someone else repair it for the stuff I couldn't fix. Cars are still something of a durable good (less so than they used to be, though) and aren't nearly as steeped in planned obsolescence as consumer electronics...Anyway it's just a personal preference, especially since I tend to keep the same phone for two years on average. It's nice to be able to, at the end of those two years, clean everything up, replace the battery, wipe the device and sell it to someone. Keeps the device out of a landfill for a while longer and helps me pay for a new one. :)
Alexey291 - Friday, March 28, 2014 - link
I am really confused.When sammy (we all dislike sammy so fair play i suppose) makes a rehash - its boring and crap and so on.
When HTC essentially makes a larger, less comfortable version (top button? really?) of last years phone (which didn't sell) - it's doing things to design that nobody else does! It's great and amazing! Righto.
And if one remembers that last year's One (m7 or whatever) scratched off the metal finish and turned ugly in literally a month... Yeah design...
Essentially: Camera's meh. Screen's pretty standard, battery pretty normal for the recent crop of devices. Benchmarks are "optimised". Ergonomics are worse. This is going to be a boring year for droid smartphones....
And I love a bit of anecdotal evidence at the end. Very fitting for a serious review. /s
HangFire - Friday, March 28, 2014 - link
You're wrong about the "metal finish". You must be thinking of one of the One variants, because the One doesn't have a finish to be scratched off, it's solid metal (over a plastic sandwich), anyone who read the M7 review knows that. I've had mine for over 3 months and it still looks new.Personally I prefer a top button, I can silence a ringing phone in my breast pocket without even looking at it, my pen prevents me from doing that with a side button, but I realize this is a matter of personal preference.
Alexey291 - Saturday, March 29, 2014 - link
My mate's one was black (you know the usual anodized bs) and yup it scratched off in literally 2 or 3 weeks.And yeah I agree the silver-metallic version is certainly difficult to scratch (well at least not via rubbing it lol) but I personally would have gone for the black one. If I was going to support a dying company that is :)
As for the top button well that really is a matter of preference naturally. Except ofc after however many years of using a side power button one gets used to it. (And that's aside from having to really REACH for it whenever u want to shut the screen off.)
Either way. Not enticing. Neither is S5. Z2 looks ok but is likely meh too. Good thing I still have a year on my contract xD