POST A COMMENT

108 Comments

Back to Article

  • quarpronuet - Tuesday, July 30, 2013 - link

    "BenchmarkBooster"..?
    what the..
    Reply
  • piroroadkill - Tuesday, July 30, 2013 - link

    Underhand. Real classy, Samsung. Reply
  • FATCamaro - Tuesday, July 30, 2013 - link

    Optimization? What the fsck. Just call it cheating like it is. Reply
  • nutgirdle - Tuesday, July 30, 2013 - link

    The file system does not need to be checked. Reply
  • MrCommunistGen - Tuesday, July 30, 2013 - link

    +1 ROFL! Reply
  • Dman23 - Tuesday, July 30, 2013 - link

    Samsung doing what they do best. Manipulating coverage and cheating to artificially inflate their success. You would think paying students / people to post fake negative reviews of competing products i.e. HTC and iPhone, is bad enough --->http://www.bbc.co.uk/news/technology-22166606 but apparently there nefarious ways knows no bounds. You would think they would have some confidence in their products not to result to these childish / cheating ways but apparently not. Soooo sad Reply
  • StiHson - Thursday, August 01, 2013 - link

    Cheating? Inflate? Am I the only one who finds this article describes that Samsung unlock full power on their phone for (certain) benchmarking applications (they're not inflating the results, the processor *IS* that fast and powerful) and basically limits the phone for everything else. The only cheating here is that the users get to use the slower phone while full potential is only available to benchmarks. I do understand that battery life would be significantly lower with that, but isn't that what these dual-mode processors are for? Reply
  • lin2log - Thursday, August 01, 2013 - link

    Yeah. You ARE the only one. Catch a clue. Reply
  • ghgidug9w - Thursday, August 01, 2013 - link

    Well, it is cheating. What is the point of having some chip if you cannot fully use it (might as well put in a cheaper chip and save $). At the same time, they want you to believe you have access to it. If I sold you an i7 PC at price, boasting of its performance, but then underclocked it so that it is only as powerful as an i3, would you be OK with that? Reply
  • erephus - Friday, August 02, 2013 - link

    What is most important to you?

    Have a smooth and fast experience when running a beenchmark software or when using the phone for what you bougth it for?
    Reply
  • darwinosx - Tuesday, July 30, 2013 - link

    Samsung is a scumbag company that is investigated for illegal behavior even in South Korea where they have a lot of power. Nothing they do should surprise anyone. Reply
  • Lonyo - Tuesday, July 30, 2013 - link

    Apple is a scumbag company that is investigated for illegal behaviour even in the USA where they have a lot of influence/mindshare. Nothing they do should surprise anyone.

    See how easy it is?
    Reply
  • eallan - Tuesday, July 30, 2013 - link

    Point me to a story where apply pays reviewers or does this. Reply
  • RichDavis - Tuesday, July 30, 2013 - link

    Like what? The ebook issue? That was decided by one person. Personally, I've read through the same thing they were and I wouldn't think that Apple was doing anything illegal. All Jobs was doing was to make it a fair market. What companies TYPICALLY do is set a retail price, set a wholesale price to resellers and let the resellers sell it for whatever they want. This is just trying to create a equal playing field. That's all Jobs was suggesting. he wasn't doing anything more than that, and since Jobs wasn't there to defend his statements, a judge is just trying to squeeze money out of Apple because Apple has lots of money. Now, in terms of routing profits from doing business in other countries into Ireland. Other companies such as Microsoft, Google and others do the same thing. There is NOTHING wrong about it. The reason why they do it is because the profits they made in other countries have already been taxed by the various countries they did business in. To constantly bring money back and forth between other countries and the US, they have to pay taxes on it and it's very difficult to manage their finances. BUT, Apple does invest the profits they make in other countries and that is brought into the US and it is taxed. Apple pays a VERY reasonable percentage in taxes. If you look at Apple's income statements, they pay a decent percentage of their gross profits in taxes. Last year they paid out about $14 BILLION in taxes. That's more than Google made in Net Profits for the year.

    Apple have never been sued for Monopolistic business practices like MIcrosoft got continually nailed for. Nor did Apple get nailed for price fixing, which Samsung has been convicted of TWICE, once for memory price fixing and another time for monitors. They should get nailed again for raising prices to Apple on the processors they mfg because Apple went to other suppliers for other components. samsung also should be nailed for conflict of interest and unfair business practices since they sell components to various other companies and then they turn around and compete with similar products. That's COMPLETELY wrong. Nothing right about doing that. That's like Intel going around selling Intel computers competing against their own customers.
    Reply
  • DigitalFreak - Tuesday, July 30, 2013 - link

    Oh do shut up, you twat. Reply
  • PeteH - Tuesday, July 30, 2013 - link

    What a compelling argument. The wisdom of your words has undoubtedly opened many eyes to the rightness of your position. Bravo to you sir, bravo! Reply
  • TedKord - Tuesday, July 30, 2013 - link

    The ebook situation WAS price-fixing, and they did get nailed for it. Reply
  • psuedonymous - Wednesday, July 31, 2013 - link

    "What companies TYPICALLY do is set a retail price, set a wholesale price to resellers and let the resellers sell it for whatever they want. This is just trying to create a equal playing field. That's all Jobs was suggesting."
    Actually, the opposite is the case. Amazon were previously using that model for ebook sales, but Apple colluded with publishers to move to a different model, where the publisher sets the final sale rice, and gives the reseller a fixed cut of that final price. This, combined with agreements requiring Apple always have the lowest (or equal to lowest) sale price meant that Amazon was prevented from selling ebooks from those publishers for less than Apple was, and publishers got to see prices higher than Amazon's previously established baseline.

    THAT was why Apple was slapped down in every court that has seen the case (US and EU), and why all the publishers involved settled to avoid more costly losses.
    Reply
  • RichDavis - Tuesday, July 30, 2013 - link

    Samsung pays much more in lobbying money in the US than Apple does. In fact, Apple spends the least for most of these large computer/software companies. Microsoft spends more in lobbying, so do other high tech companies, but generally Apple doesn't like greasing the palms of politicians by contributing to political campaigns. Reply
  • BC2009 - Tuesday, July 30, 2013 - link

    Seeing the replies to your comment, it apparently wasn't as "easy" as you say. You've been pretty much shut down. Reply
  • RichDavis - Tuesday, July 30, 2013 - link

    Lonyo, it seems as though you don't know how to read. You obviously get suckered into the Misleading crap. Apple is a LOT more upstanding than most of their competitors. Apple usually sets trends that other try to copy or emulate. And since they are successful, idiots such as yourself try to damage their reputation. While they are't perfect, they are certainly a lot better at how they conduct business than a lot of others. You might want to go to college and learn how to read and question what you are reading. something that a lot of people can't seem to do. Reply
  • melgross - Tuesday, July 30, 2013 - link

    Except that what you said is wrong, and what he said is right. Reply
  • drenBS - Wednesday, July 31, 2013 - link

    an example of someone paid by The Chaebol. Reply
  • M_F - Wednesday, July 31, 2013 - link

    You might want to read the following article by The Independent.

    http://www.independent.co.uk/news/world/asia/tax-e...
    Reply
  • hhh333 - Saturday, August 17, 2013 - link

    yea it is easy as samsung/ to copy things others create Dumb@ss Reply
  • Dman23 - Tuesday, July 30, 2013 - link

    Samsung doing what they do best. Manipulating coverage and cheating to artificially inflate their success. You would think paying students / people to post fake negative reviews of competing products i.e. HTC and iPhone, is bad enough --->http://www.bbc.co.uk/news/technology-22166606 but apparently there nefarious ways knows no bounds. You would think they would have some confidence in their products not to result to these childish / cheating ways but apparently not. Soooo sad Reply
  • DanNeely - Tuesday, July 30, 2013 - link

    You'd've thought they'd've learned from the gfx industry. They all tried writing special drivers to cheat on benchmarks. They all got caught repeatedly and were intensely mocked and derided for doing so. Eventually they figured out all they were getting was bad PR and stopped cheating. Reply
  • CBone - Tuesday, July 30, 2013 - link

    IIRC, didn't everyone (amd, nvidia, 3dmark) agree to stop talking about it as cheating and allow optimization paths? Reply
  • name99 - Tuesday, July 30, 2013 - link

    Samsung HAS been caught cheating repeatedly --- and the internet has apologized repeatedly for its behavior. Are you completely unaware of the tech arguments of the past few years?

    Presumably they felt that "since our fan base not only already accepts everything we do, but provides plenty of excuses for it, what's the downside?"
    Reply
  • darwinosx - Tuesday, July 30, 2013 - link

    Exactly.
    Cheap plastic case? Buy another one to go over it.
    Only 9 GB of storage? Buy an SD card.
    Lousy screen in sunlight with shimmering while scrolling? Buy a screen protector.
    Lag, lag, lag? Deny it exists.
    Touchwiz a dog with terrible interface, speed issues, and features basically don't work? Like all the eye stuff they advertise on TV? Deny again.
    Apparently owning a Samsung smartphone requires some sort of Stockholm syndrome.
    Reply
  • Bob Todd - Tuesday, July 30, 2013 - link

    Sadly I think it is just as likely that their response will be to update the benchmark booster functionality to cover new versions and other apps vs. removing it all together. Reply
  • Senti - Tuesday, July 30, 2013 - link

    Stopped cheating? Who, where? Both NV/AMD drivers are full of special hacks for popular games and benches. But write your own program and you find out that it often doesn't even work properly because the (standard) features you used are not widespread in games. Reply
  • dishayu - Wednesday, July 31, 2013 - link

    Well, very few people buy GPUs by synthetic benchmarks. Most people look at fps numbers in games, and it doesn't matter if NV/AMD cheat to get a higher number in games because that cheat will eventually generate higher fps in-game for the end user as well. Reply
  • psychobriggsy - Tuesday, July 30, 2013 - link

    Can you measure the power consumption difference between 480MHz GPU and 532MHz GPU? Reply
  • Shadowmaster625 - Tuesday, July 30, 2013 - link

    Can someone write an app+GUI that changes that file to enable./disable a turbo boost for any chosen app? Reply
  • r3loaded - Tuesday, July 30, 2013 - link

    Wow, just like the Nvidia fiasco a few years back where they whitelisted benchmark exes so that the drivers ran with lower quality settings for them.

    Despite all these "optimisations", the S4 can't touch the responsiveness of the Nexus 4.
    Reply
  • CBone - Tuesday, July 30, 2013 - link

    I doubt qualcomm would put out a new chipset that's slower than their old one. My experience has been that the s4 is close enough in responsiveness and dominates the n4 in Reply
  • CBone - Tuesday, July 30, 2013 - link

    eveything else.

    Gods, this comment system sucks.
    Reply
  • rd_nest - Tuesday, July 30, 2013 - link

    Don't know about S4. But my N4 "lags" (interpret the way you want) while scrolling the settings menu. Almost 50% of the times, it will stutter.
    Also on random occasion, it will stutter in widgets page. Running 4.3 stock.
    Issue was same on 4.2.2
    Reply
  • jonup - Tuesday, July 30, 2013 - link

    Try Trinity! it appears counterproductive to choose the phone with the largest development support and not you it. Reply
  • rd_nest - Tuesday, July 30, 2013 - link

    That's a valid argument.

    Every phone has some amount of development community. My choice of N4 was not because of development community. It was merely a matter of bang-for-buck.

    I just don't have the "patience" now to check through various kernel or ROM. I know it sounds weird. Now I just root and run stock most of time.
    Reply
  • thekingofmean - Tuesday, July 30, 2013 - link

    <i>Every phone has some amount of development community.</i>

    That's an invalid statement.
    Reply
  • rd_nest - Tuesday, July 30, 2013 - link

    Maybe. I am not going to run around and check for each and every phone out there.

    I am sure you knew that was not the point of discussion. But if it makes you feel better, I will accept that argument.
    Reply
  • ChuckRox - Wednesday, July 31, 2013 - link

    All Androids lag. You guys need a real OS that doesn't need a mega-core processor to run smoothly. I'm talking, of course, about BB10 which runs like a racehorse and never crashes or lags. Reply
  • markthema3 - Tuesday, October 15, 2013 - link

    >Never crashes Reply
  • Jmaxku - Tuesday, July 30, 2013 - link

    Very true. Which is why I purchased the Samsung Galaxy S4 GPe which bests the responsiveness of the Nexus 4—not really, but it certainly isn't any worse and the gaming performance is better. Reply
  • jonup - Wednesday, July 31, 2013 - link

    Well, I do not how you measure your gaming/responsiveness experience and I cannot comment on the Octa core model, but as far as the Qualcomm version is concern I highly doubt you can tell the difference in real life. Given the similarity of the SoC, (yes the on in S4 is faster, but faster than plenty fast) and the fact that I have been running my phone on only 2 cores and max freq of 1G without noticing any negative effect on the user experience, I find your claim that S4 performance is better than a not so finely tuned N4 false. The biggest benefit of S4 would actually be sot battery life. AMOLED is just so more energy efficient. Reply
  • Ralos - Tuesday, July 30, 2013 - link

    Wow. This is worst than cheating. Anandtech is more than polite to call this "optimisation".

    These are not optimisations; this is a boost pure and simple, activated uniquely during detected benchmarks and that's way worst.

    If the GPU could reach 533MHz during other applications, then it would be plain cheating. The fact that this speed is never available elsewhere makes it worst that cheating; this is deception.

    It's gonna take awhile before I buy anything Samsung again.
    Reply
  • darwinosx - Tuesday, July 30, 2013 - link

    Yes, they should call it what it is. Unless they are afraid of hurting advertising revenue like all the other sites. Reply
  • dishayu - Wednesday, July 31, 2013 - link

    If they were really afraid of anything, they would just shut up and not post anything about it at all. Although most of us (me included) will agree this is shameless cheating, Samsung can say that they never claimed any clock speeds. Plus there is a legal risk of samsung suing them for defamation. Right now, there is nothing Samsung can do but suck it up and make amends. Reply
  • ancientarcher - Tuesday, July 30, 2013 - link

    Is this a hatchet job commissioned by Intel?
    While this is bad, nothing can touch the depths that your beloved Intel dropped to with AnTuTu and publicising with the ABI Research report. At least, it doesn't skip certain steps and completely dupe the buying public...
    Maybe you should start taking some money from Samsung/ARM, your reporting will become relatively unbiased then...
    Reply
  • darwinosx - Tuesday, July 30, 2013 - link

    Ah here is the Samsung apologist..what took you so long? Reply
  • ancientarcher - Tuesday, July 30, 2013 - link

    Samsung is cheating, no doubt! All I am saying is, it is a far lesser sin than that of Intel which skips certain steps altogether and then publicises it. Overclocking the GPU only for certain benchmarks is wrong. And hopefully Samsung and the others listen. Samsung loses a lot more than it gains through these shenanigans.... not worth it sammy!!! It is worth a lot more to put your time and effort to improve your chips, not game the system... Reply
  • SydneyBlue120d - Tuesday, July 30, 2013 - link

    What about the Google Experience version of the device? Same behavior or it doesn't cheat? Reply
  • rd_nest - Tuesday, July 30, 2013 - link

    It's for Exynos.
    GE version uses S600.
    Reply
  • boris81 - Tuesday, July 30, 2013 - link

    From the above article:
    "Interestingly enough, the same behavior (on the CPU side) can be found on Qualcomm versions of the Galaxy S 4 as well."
    Reply
  • sherlockwing - Tuesday, July 30, 2013 - link

    It is different. The 532mhz GPU clock for Exynos is only avaliable for certain benchmarks, what they do with S600 is force the CPU to run at full clock(1.9Ghz) and all cores(4) regardless of load. But that 1.9Ghz/quad core performance is available(not always needed) for all apps and benchmarks unlike the 532mhz GPU clock. Reply
  • boris81 - Tuesday, July 30, 2013 - link

    Thanks for clarifying that! Holding all cores on the S600 at the highest clock speed may still produce better benchmark scores than if they were allowed to change speed as they normally do. I wouldn't bet that it's significant but I think it's worth investigating. Reply
  • name99 - Tuesday, July 30, 2013 - link

    This is misleadingly optimistic. It gives users an unrealistic view of the ACTUAL performance/power tradeoffs made by the device during use.

    Assume the Samsung power manager actually worked the way it should. It would detect that the CPU was working hard and boost performance, it would detect that the CPU was idling and would lower performance.
    If this were the case, there would be no need to diddle the benchmarks in this way because things would just work.

    The fact that Samsung IS diddling the benchmarks this way suggests they have no confidence in their power management --- they don't trust it to rapidly (or ever) kick in under conditions of load and/or they don't trust it to rapidly throttle under conditions of idle. Both these behaviors are ESSENTIAL in a phone.
    WTF is the point of having 4 fast cores and all this big.LITTLE infrastructure if your OS is either incapable of using them (when speed is needed) or using them drains the battery so rapidly you notice?
    Reply
  • danbob999 - Tuesday, July 30, 2013 - link

    Well to be fair with Samsung, it's possible that many of these benchmarks are poorly written and depend on very short compute time. In these cases, loosing a little time to ramp up the CPU speed can be fatal. These benchmarks also tend to produce different results on every run. Reply
  • WaltFrench - Tuesday, July 30, 2013 - link

    That's not “being fair” to Samsung, it's fabricating an excuse for them that might not even be true. If they want to claim some reason like that, they have many forums where they could tell their story.

    I, for one, would be delighted to have a Samsung engineer—even their Top Dog Marketing Guy—explain why they suggested the benchmark performance was consistent with their battery claims.
    Reply
  • Abelard - Tuesday, July 30, 2013 - link

    Sneaky. Reply
  • boris81 - Tuesday, July 30, 2013 - link

    So if these "optimizations" are a part of TouchWiz it would explain why the TW GS4s benchmark better than the stock Android ones. Would you please compare the Qualcomm CPU clocks of TouchWiz and stock Android GS4s running benchmarks? Reply
  • darwinosx - Tuesday, July 30, 2013 - link

    Yeah that never made any sense to me. Reply
  • et20 - Tuesday, July 30, 2013 - link

    It would be great if everyone, starting with you as highly valued reviewers stopped focusing on benchmark scores and spec lists.
    Your wrongheaded stubborness of looking at PCs, tablets and phones as "computing devices" is a disservice to consumers.
    If you like that kind of stuff you should cover servers and supercomputers.

    Raw computing power is irrelevant in consumer applications. I don't care what it could do, I only care what I can do with it. If it's difficult, tedious, slow, annoying, unreliable or boring, I don't care how much or how fast it can compute; I still won't buy it or use it.
    So tell what it is easy, fun, fast and pleasureable to do with it and stop overanalyzing the internals.
    Reply
  • Death666Angel - Tuesday, July 30, 2013 - link

    If you are not interested in the reviews AT publishes, look at the Verge or Engadget who use pretty much no hard evidence and only go by "feeling" to evaluate the products they test. I like some good hard numbers to go with the subjective analysis AT provides, thank you very much. Reply
  • et20 - Tuesday, July 30, 2013 - link

    I would but they also focus on benchmarks, specifications and price because they are too lazy and feel the need to cater to their tech elite wannabe audiences.
    They are spineless because they want to keep on good terms with and have access to everyone.
    So all their reviews are useless for informing buying decisions because they dare not clearly praise or criticize anything.

    I'm sure AnandTech can publish plenty of interesting analysis outside of product reviews if there's demand for it.
    Reply
  • kyuu - Tuesday, July 30, 2013 - link

    Benchmarks are useful, and if you don't like them I suggest you stay away from tech blogs in general. They don't tell the whole story, of course, but no one (legitimate) ever claimed they do.

    It's impossible to objectively quantify how a device "feels" in use.
    Reply
  • WaltFrench - Tuesday, July 30, 2013 - link

    While you're at it, please draft a note to Samsung, with a cc to the FTC. AnandTech didn't promote these deceptions, they exposed them. Reply
  • jwcalla - Tuesday, July 30, 2013 - link

    Another reason not to take benchmarks too seriously. Reply
  • UpSpin - Tuesday, July 30, 2013 - link

    As you said in your article, it was predictable that such a thing will happen. NVidia and ATI used tricks with their drivers, and Intel does it similar (Antutu and Atom) or simply by crippling the competition:
    Not related to ARM and mobile, but AMD vs. Intel and specifically the ICC:
    http://www.agner.org/optimize/blog/read.php?i=49
    so maybe you better also revise your server benchmarks to take care of this spoofing, because I doubt you've considered it yet.
    Reply
  • flashbacck - Tuesday, July 30, 2013 - link

    This is really fantastic. Reply
  • takeship - Tuesday, July 30, 2013 - link

    As I recall, they stopped marking out code path optimizations as cheating, since everyone does that for every game, but effects & downsampling to improve fps was (and is) still cheating. I.e. no derez to 16bit floats in the background when the benchmark is asking for 32bit floats. This though, is something different. As though Nvidia (or AMD) would lift their PowerTune/Powermizer/Boost speeds explicitly for benchmarking purposes, then enforce the normal limits during gaming.

    Imagine if Intel & AMD started shipping procs with only the boost frequency listed. Sheesh.

    Samsung should be called on this. Every performance claim they make is now b*ll***t until proven otherwise.
    Reply
  • gnx - Tuesday, July 30, 2013 - link

    This confirms suspicions about vendors "optimizing" for bench marks, and I doubt Samsung is the only culprit.

    What interests me more is the fact that this might be done cause of heat problems. The raw power of modern flagship smartphones seem to outstrip their cooling faculties, so despite the need to advertise and brag about high frequency processing power, the smartphones are too dangerous to allow any app to run their processors at full power, hence this "cheating" on benchmark apps.

    For all the amazing development in the mobile cpu/GPU sphere the last three years, there is still a long way to go, it seems.
    Reply
  • Den - Tuesday, July 30, 2013 - link

    Can you bypass it by renaming the benchmark to quack.exe? (That was one of the first I remember on the PC side where you got better numbers out of quake.exe than you did if you renamed it to quack.exe before benching,) Reply
  • mike55 - Tuesday, July 30, 2013 - link

    I've been looking for a CPU usage overlay like that. Where can I get this System Monitor? Reply
  • BugblatterIII - Tuesday, July 30, 2013 - link

    I don't care whether it's been done before or whether everyone does it. This is fraud. It's providing misleading information upon which a consumer may base their decision to buy. That's fraud, and it's illegal in the UK at least.

    And I'm not a Samsung hater; I'd say the same for any company doing this.
    Reply
  • BC2009 - Tuesday, July 30, 2013 - link

    So, what I want from Anandtech is the ACTUAL benchmarks after they disable the benchmark booster so we can see how the GS4 really stacks up agains the HTC One and iPhone 5. Reply
  • WaltFrench - Tuesday, July 30, 2013 - link

    Go back and re-read. Now about power, is t it roughly true that increasing voltage and speed by 10% should increase GPU power by about 20%—maybe more? Assuming that your games wouldn't shut down for thermal protection during play, the full-bore speed might mean an hour's less play. Reply
  • jeffkibuule - Wednesday, July 31, 2013 - link

    The relationship with voltage, speed, and performance you have is WAY off. At higher frequencies, you end up needing to scale voltage quite a bit to maintain stable clocks, and that draws a LOT more power, and the performance gains are often negligible. In physics, P = V^2/R. And because voltage has such a tremendous effect on power, endlessly scaling frequency is not a good way to save battery life. Reply
  • Excors - Tuesday, July 30, 2013 - link

    "Nothing for GLBenchmark 2.5.1 though, despite its similar behavior."

    TwDVFSApp has some hard-coded package names, but it also gets a list from android.os.DVFSHelper.PACKAGES_FOR_BOOST_ALL_ADJUSTMENT. That's set in /system/framework/framework2.odex and contains:

    com.aurorasoftworks.quadrant.ui.standard
    com.aurorasoftworks.quadrant.ui.advanced
    com.aurorasoftworks.quadrant.ui.professional
    com.redlicense.benchmark.sqlite
    com.antutu.ABenchMark
    com.greenecomputing.linpack
    com.glbenchmark.glbenchmark25
    com.glbenchmark.glbenchmark21
    ca.primatelabs.geekbench2
    com.eembc.coremark
    com.flexycore.caffeinemark
    eu.chainfire.cfbench
    gr.androiddev.BenchmarkPi
    com.smartbench.twelve
    com.passmark.pt_mobile

    In my opinion, the fundamental problem is that almost everybody who reviews smartphones has a very juvenile approach - they want to run a benchmark and get a single number, because that lets them say "phone X has a bigger number than phone Y, so it is better" without engaging their brains. (And those are the more advanced reviews, that go beyond simply counting cores.)

    But phones aren't designed for a single performance number - they're designed for power *and* performance, and it's impossible to separate one from the other.

    I think a good review would analyse two things:

    * The power/performance curve. If a phone has well-designed hardware and software, it will give better performance at the same power as another phone, across the whole curve. You could make an attempt at comparison by tweaking the kernel's cpufreq settings, to force it to run at a particular speed, then measuring benchmark scores and battery current. Do the same for GPU speeds if possible. Repeat at every available speed and draw the curve. That's hardly ideal (you probably need to compensate for display brightness etc), but it's a start, and it'd allow more meaningful comparisons than a single number per benchmark per phone.

    * The tuning. A good phone will switch sensibly to higher-performance modes to give a responsive user experience. A bad phone will use too much power in mostly-non-interactive scenarios (on the home screen, recording a video, playing Angry Birds, etc), or it will take too long to ramp up when the user interacts (e.g. taking a photo or switching between apps will be slow and jerky). I think this is especially a concern with big.LITTLE - switching on the A15 cores is a relatively big commitment in time and power, so the software will have to be very careful to use them at the right times. It would be great if someone could at least attempt to measure this kind of thing quantifiably.

    Looking at those two aspects would help understand how well a phone has chosen its power/performance tradeoffs.

    But while reviewers keep comparing SunSpider and AnTuTu scores, it's no surprise that phone developers will spend a lot of effort on maximising those scores - sometimes subtly (like tweaking GPU driver optimisations for a benchmark's unique usage patterns), sometimes blatantly (like Samsung here) - because that gets good reviews and sells more phones. There's little point in kindly asking Samsung to focus less on benchmarks and accept worse reviews and lower sales and less money - if you want them to focus more on the user experience, that's what has to be measured in reviews.
    Reply
  • yhselp - Tuesday, July 30, 2013 - link

    It's hard not to attack your comment since all you say is generally true. However, it seems that you've never read an AnandTech smartphone review -- the whole of it, or you've somehow missed what it says (and many accompanying pipeline stories). At AT they never say X is better than Y because of some benchmark number, on the contrary -- they always say benchmarks are subjective and are only a part of the story. I've read some shockingly objective pieces on this matter here on AT; of what a benchmark's role is, what it represents, how it should be read, what significance is has -- all benchmarks are useful as long as you read them 'right' and AT does an excellent job of stressing that very point in each review. Also, they were the first to say that comparing smartphones cross-platform is a very subjective thing.

    So you should've probably familiarized yourself with AT better before posting over-generalized comments like this one. I'm not hating on you, just saying.
    Reply
  • abdealiv - Wednesday, July 31, 2013 - link

    you missed his point completely!! Sure AT has different suits for benchmarking & also mentions that benchmarks are subjective but 1st thing rest of 99.999% tech sites , reviewers, bloggers & general users do is benchmarks this new phones with those mentioned apps & compares instantly with other flagships ..so its not surprise that samsung feels the need to get positive reviews from those majority reviewers. this is why the guy above mentioned to change the review system as whole ..not for AT but for others out there. Reply
  • yhselp - Wednesday, July 31, 2013 - link

    Wait a second, my reply was strictly aimed at Excors "the guy above" and not this article so I don't believe I've missed any point. I am not defending benchmark optimizations, on the contrary -- I've always been against this game of cat and mouse between silicon makers; I'm all for focusing on improving the hardware and user experience.

    It seems you've missed the point of my reply as a response to Excors' comment completely!!1 While I agree that many other tech sites evaluate smartphones (and other products) solely by comparing benchmarks, however, Excors wrote his generalized comment (which I agreed with) here on AT, and not on other tech sites. Wouldn't you agree that it's pointless to leave feedback and advice at the wrong place, unless you believe AT doesn't review properly which is not the case. That's what my comment was about.
    Reply
  • kurkosdr - Tuesday, July 30, 2013 - link

    When will benchmark developers learn to RANDOMIZE any strings that identify the app and the fize size?

    Google "quack ati" for similar hilarity.

    PS: What if you run the benchmark again and again for hours and the phone dies a death from overheating? Will the samsung warranty cover it?
    Reply
  • tcool93 - Tuesday, July 30, 2013 - link

    Anandtech is an idiot for making such a big deal out of this. Now you got everyone trashing Samsung claiming they are "cheating". Which is total BS, because its not any different than what EVERY other company does, which is write drivers so that certain games or whatever perform better with their hardware. And that includes benchmark programs, and Anandtech knows it.

    We are also talking about a measly 53mhz... big deal. This is just stupid, now this biased BS will get spread everywhere, thanks to Anandtech.

    I believe Android tablets even have an overclock option in their settings.
    Reply
  • varad - Wednesday, July 31, 2013 - link

    "biased BS"?? - So how much does Samsung pay you for writing comments like this? Reply
  • Scannall - Wednesday, July 31, 2013 - link

    It smells like....... Astroturf. Reply
  • tcool93 - Tuesday, July 30, 2013 - link

    It would be funny to see Samsung sue Anandtech for spreading this fud. Reply
  • varad - Wednesday, July 31, 2013 - link

    It would be funnier if Anandtech also tracked your a/c to Samsung Reply
  • Nathillien - Tuesday, July 30, 2013 - link

    Me: Hey Exynos, I need you to finish this task faster than usual.
    Exynos: No problemo Boss, right away.
    iPhan: Hey A6 I need you to finish something a bit faster than usual.
    A6: No can do, screw you. I'm the Boss of you.

    Oh LOL
    Anandtech preparing ground for the iPee5S release and GLBench comparison with GS4 Exynos devices? Noooooo.
    I can just imagine future iPhan benchmark enthusiasts comments.
    " ... well, they tweaked their chipset ...whine ..."
    Reply
  • ratte - Tuesday, July 30, 2013 - link

    Me: Hey Exynos, I need you to finish this task faster than usual.
    Exynos: Can't do that Boss, unless it's a benchmark.
    Reply
  • varad - Wednesday, July 31, 2013 - link

    now that is hilarious :) Reply
  • Musafir_86 - Tuesday, July 30, 2013 - link

    -Hmm, so how about what Intel's (allegedly) doing? http://www.eetimes.com/author.asp?section_id=36&am... (via http://legitreviews.com/news/15752/).

    Regards.
    Reply
  • Musafir_86 - Tuesday, July 30, 2013 - link

    Correction: http://legitreviews.com/news/15752/ Reply
  • ViRGE - Tuesday, July 30, 2013 - link

    Brian Klug has already commented on that one on twitter: https://twitter.com/nerdtalker/status/355445625518...

    Which goes along with the fact that AT doesn't even use AnTuTu.
    Reply
  • everydae - Tuesday, July 30, 2013 - link

    It's so shame that Samsung did this. As a Korean-American, I know how 'ethically' bad Samsung is doing in Korea (Another nickname of Korea, "Samsung Republic," explains quite a lot. Here in America, at least, the USA doesn't stand for the United States of Apple), but at least I believed they would be 'technically' honest with what they have to offer and what they show to US media. But guess I am wrong. Reply
  • shm224 - Wednesday, July 31, 2013 - link

    Well, not yet, but we are getting there, the United States of Apple.

    Sure, no doubt this is unethical, but I don't think this is egregiously bad, having overclock/jailbreak'ed my smartphones in the past. I'm not a big gamer, so never noticed any difference. I agree with the author that Samsung should either open up the setting for all to use or keep it at 488Mhz.
    Reply
  • rd_nest - Wednesday, July 31, 2013 - link

    What do you say about Intel episode then? Reply
  • batongxue - Tuesday, July 30, 2013 - link

    I'm not surprised at all.
    It is the sort of things Shamesung do.
    Reply
  • lgmagic - Wednesday, July 31, 2013 - link

    Thanks for the wonderful post. Can you also post similar results from other manufacturer devices? I'm pretty sure they would have similar tweaks. All the mobile manufacturers are simply showcasing their best through these benchmarks. what is wrong in that? Obviously you will find every mobile with such things to boost their performance. Reply
  • DanaGoyette - Wednesday, July 31, 2013 - link

    Hmm, I wonder if this means anything for overclocking and warranty... Reply
  • shoomkloom - Wednesday, July 31, 2013 - link

    non-insignificant = significant :-) Reply
  • yhselp - Wednesday, July 31, 2013 - link

    Untrue, they have different degrees of implication. Significant is stronger than non-insignificant. These subtle differences are key for the diversity of a language. Reply
  • toyotabedzrock - Wednesday, July 31, 2013 - link

    I see a string that looks designed to raise the thermal limits for the Soc. Reply
  • arch2008 - Wednesday, July 31, 2013 - link

    Watch this video about test of Samsung galaxy S4 to make sure for everyone that the Samsung Company false and are misleading the customers ..I bought a Samsung galaxy S4 a month ago and fell from the table height of 50 cm and with regret no longer works, and when I went to the Samsung's agency in Egypt , they told me the screen need to change, and that the price is 1900 LE, not covered by the warranty .. and when I told them about test they told me: (they do not know anything about these tests and they are not responsible for that, and they told me to go to Samsung Egypt) !!!! ...
    When I went to the Samsung Egypt, I have directed a question to a customer service representative, and the question as follows: (Is Samsung S 4 against bumps and breakage?), the answer was: (Yes,sir), and when i faced him with my problem,he stripped of his answer and repeated the same words that I heard in the Agency.
    waiting your comments,Thanks for all
    http://www.youtube.com/watch?v=K4f9gQ7b6Gg
    Reply
  • Gondalf - Wednesday, July 31, 2013 - link

    Too much money around and in a too short time sometimes give strange effects on brain. Samsung is trying to dominate the handset with it's phones. I have the suspect that once Merrifield will be out on a Contender top of the line phone, the mission will become a lot harder, looking at the absurd trick used to fake the benchmarking. The market share can goes up or down quickly in this world.
    Good luck Samsung, you don't look good at all.
    Reply
  • Zibri - Thursday, August 01, 2013 - link

    Strings? LOL! Can't you decompile and analyze what it really does? Reply

Log in

Don't have an account? Sign up now