Updated Smartphone & GPU Bench DBs Live!
by Anand Lal Shimpi on December 24, 2011 12:08 AM EST- Posted in
- Site Updates
- Smartphones
- Mobile
- GPUs
We've been running our public performance database tool, Bench, for a few years now. Bench allows you to have direct access to the performance data we use in our reviews, segmented according to product category. Today we have five major product categories in Bench: CPU, GPU, Smartphone, Mobile and SSDs.
CPU Bench is the longest running without significant changes. You can actually compare Intel's most recently released Sandy Bridge E CPU to an old Core 2 Duo E6550 from four years ago in a number of tests using the tool.
GPU Bench has always posed a more significant challenge. Drivers and test platforms change very frequently, which often precludes us from keeping a running tally of benchmarks similar to what we do on the CPU side. We used our Radeon HD 7970 review as the kicking off point for our brand new PCIe 3.0 testbed for GPUs going forward - a platform we hope to keep in place for at least the next 12 months. While all of this data is already in the 7970 review, if you want to do any head to head comparisons between GPUs you can now do so using our updated GPU Bench 2012 database. The previous version of the database (GPU Bench 2011) is still available.
Similarly, we've finally updated our Smartphone Bench database. Smartphone Bench 2011 features all of the tests and test data we've been using in our most recent smartphone reviews (e.g. Droid RAZR). We opted against calling it Smartphone Bench 2012 because we plan on updating some of the benchmarks in the new year (e.g. you may have seen one of our new battery life tests in our Eee Pad Transformer Prime Follow-up).
We are in the preliminary stages of planning AnandTech's site refresh for 2012, including updates to Bench. If you have any suggestions you'd like to see implemented, please feel free to leave them as comments here. We already have a number of ideas of things we're trying to implement, but your opinions are the most important here so I'm always eager to listen if you're willing to share.
Facebook
Twitter
RSS
63 Comments
View All Comments
Brian Klug - Saturday, December 24, 2011 - link
A few important clarifications that I definitely should have made:1 - Adding which OS is used to produce results is a top priority. I have that data (and it's been reported in a few reviews), it's just difficult to express right now with some of the presentation fields allowed in the engine (this is something that will be fixed).
2 - I see people saying this a lot, but I have a UK unlocked SGS2 and in the stock browser never see numbers like what I occasionally read in comments. I just re-ran and got close to that number again. This is updated to the latest official ROM using Kies as well.
One thing I want to be absolutely clear on is that we aren't ever going to pick and choose third party browsers to run for smartphone benchmarks. While some combos definitely post better numbers, the purpose is to make relative comparisons (inside the domains of each WebKit + V8 combo for Android, so forth for the other OSes) and also gauge what kind of performance comes stock.
3 - Absolutely agreed, we do re-run things and to the best of my knowledge what's live right now represents the latest data we have. Unfortunately devices do go back after anywhere between 2 weeks (which is 1 week longer than most other reviewers use these phones for smartphone reviews - yes, seriously) or a month. I'll absolutely run things after platform updates though, since bench ultimately has to be helpful to everyone :)
-Brian
augustofretes - Saturday, December 24, 2011 - link
I've just ran Browsermark on my SGS II and it had 94,295. Running Stock Browser on Samsung's stock ROM (but rooted): Android 2.3.6 (KK5).Brian Klug - Saturday, December 24, 2011 - link
I should note that latest I can see is still XWKI4, though this wouldn't be the first time that Kies/OTA has lied to me about what the latest update is. :PInstalling XWKK2 now.
-Brian
KaarlisK - Saturday, December 24, 2011 - link
How do you install HWKK2 when the latest you see is XWKI4?Brian Klug - Saturday, December 24, 2011 - link
Just flash using Odin basically. It's a Samsung ROM but not pushed out through Kies.I'm a believer now: http://imgur.com/a/7zGpp
-Brian
KaarlisK - Saturday, December 24, 2011 - link
Thank youDrazick - Saturday, December 24, 2011 - link
MATLAB would be so easy to test and evaluate.Many of the readers of the site work with it.
It would be great if you added it to the list of applications you test.
Oldskool71 - Saturday, December 24, 2011 - link
I would appreciate a greater level of consistency regarding what gadgets you choose to include in your test suites.As an example, the OpenGl ES tests were run across several android phones for easy comparision. However, when the same tests were run on iOS phones, only the 720p offscreen results are reported, not the actual results with built-in phone screen itself. Since the very high resolution of the iPhone4 negatively affects the real graphics perforance, this glaring ommision may give rise to suspicions of giving the Apple platform an undue advantage.
This bench may provide a tool for unbiased comparisions across devices; However such suspicion about biased tests must be countered with a transparent policy on what devices to include in your test suits.
Sorry for my poor english, Regards, an international reader.
bplewis24 - Saturday, December 24, 2011 - link
Yeah, because Anandtech has a reputation for being biased against Apple./sarcasm
exostrife - Saturday, December 24, 2011 - link
GPU Bench (and the others) are all a great idea, and I'm sure they are a lot of work to be as good as they are. However GPU bench just isn't big enough to really be useful in its current state, and promising to keep the rig for GPU 2012 together for at least 12 months just isn't much of a promise.I try to use bench when I see deals on video cards to evaluate how they compare to my current card, something I'm sure many do as these are lean years for the world and many of us are on a budget.
Gamer or not upgrades do not necessarily come on a annual basis.The beauty of bench is a common platform to run cards against and compare the benchmarks. If you don't have a common platform that goes back a couple years, then the usefulness of Bench as upgrade tool is limited.
I had thought the original idea of Bench was to take a fast system, benchmark all the cards in it from that point on, and then have common numbers for all the cards you tested from that point forward. Obviously you would probably have to update the test system every few years to keep it getting behind, but doing it every year and not retesting everything on the new system...you are crippling the usefulness of bench and making more work for yourself.