Introducing the 2012 Mobile Benchmark Matrix
by Jarred Walton on January 6, 2012 8:02 PM ESTBenchmarking the Matrix
With the updated test suite, we’re also losing some points of reference to our back catalog of laptops. Obviously, the biggest change is in the gaming results, and we decided to take one of our recently reviewed laptops for a spin using the new benchmark suite. (We may look at adding a couple more lower end laptops from late 2011 to the charts as well in the near future.) ASUS was kind enough to let us hang onto the G74SX until the new suite was complete, and given the reasonably high-end hardware and continued availability, it makes for a good starting point for our 2012 laptop results. We updated to the latest NVIDIA drivers (290.56 at the time of testing) and ran through all of our gaming tests. You can find the complete results in Mobile Bench, and the games are all grouped under the Mobile Gaming 2012 category; since we only have one laptop tested right now, we’ve summarized the gaming scores below.
In our 2011 gaming suite, the ASUS G74SX—and NVIDIA’s GTX 560M—proved capable of handling the majority of games at our Enthusiast settings and 1080p while still breaking 30 FPS. With some of the latest titles at similar “maxed out” settings, frame rates now drop below 30FPS in five of the seven titles, but remember that our new Enthusiast is equivalent to last year’s “Ultra”. There are certainly other games that will tax the GTX 560M, and our recommendation is that you consider disabling antialiasing or dropping the quality down a notch if you want higher frame rates, but in general the GTX 560M is still a good solution for notebook gamers.
Closing Thoughts
As a sci-fi buff, it’s pretty exciting to see the rapid pace of advancement over the last few years. Today’s smartphones pack about as much power in a small portable device as the PCs we used less than a decade ago. If you’ve ever dreamed of real-world tricorders and holodecks—or maybe cyberspace and Ono-Sendai decks—they’re getting tantalizingly close. Maybe we won’t have exactly what the sci-fi writers of 20 or 30 years ago envisioned, but we’re definitely shedding the wires and I look forward to seeing where we will be in another ten years!
Back on topic, no benchmark suite can ever (reasonably) contain every performance metric, and we do understand that mobile gaming is still a small piece of the larger mobility pie. Even so, it’s still important to consider mobile GPU performance, and with the improving nature of integrated graphics we felt it was time to finally ditch the 2006-era graphics quality settings and shoot for something more visually appealing. Our mobile gaming suite now represents some of the latest DX11 titles, and even at our Value settings all of the games look quite good. If you’re looking for basic gaming capabilities, all you really need is a mobile GPU that can hit 30 FPS at our Value settings in all seven titles and you should be set. If you’re after higher quality and higher resolutions, you’ll want something more than midrange GPUs, but be prepared to pay the price—both in terms of cost as well as in terms of notebook size.
With the updated laptop benchmarks now in place, we’re still early enough in 2012 that if you can make a good case for other benchmarks that we haven’t included we’re willing to consider adding a couple more tests. Remember that the goal is to provide a reasonable test suite from which you can estimate performance in other similar benchmarks, so adding three more video encoding tests isn’t really going to add much; on the other hand, if there’s a class of application you don’t feel our test suite adequately covers, sound off in the comments.
As a final thought, I’ve been the head laptop tester at AnandTech since early 2006. While we have frequently heard about the increasing importance of laptops in the overall computer market, the past two years have really shown tremendous growth. We had seven mobile articles on AnandTech in 2006, 15 in 2007 and 2008, and 32 in 2009. That’s pretty reasonable, but then in 2010 we had a whopping 107 mobile articles and 2011 eclipsed that with 166 articles. Wow! Granted not all of the articles in the past two years are about laptops, and we've had a lot of shorter articles in the past two years, but however you want to view it one thing is eminently clear: mobile devices are now well and truly established and our increased coverage reflects that. It’s also worth noting that Intel’s Sandy Bridge and AMD’s Llano launches were both more about the mobile sector than about desktops, and the upcoming Ivy Bridge, Trinity, and Haswell appear to continue that trend.
Here's looking forward to another awesome year in the mobile space, kicking off with CES next week. Hint: besides the usual plethora of large displays and 3D demonstrations, CES is all about smartphones, tablets, and laptops. (I almost feel sorry for Brian...almost.)
Facebook
Twitter
RSS
48 Comments
View All Comments
losonn - Saturday, January 7, 2012 - link
A few thoughts for someone who went out and bought an XPS 15 L502x immediately after reading your L501x / L502X reviews.I've got to agree with an earlier poster that I always skim the mobile reviews here at AnandTech, but frequently find myself referring to laptop reviews elsewhere (including notebookcheck.net) for better formatted specs, more helpful product pics, more dense information, specific noise / heat info, power adapter weight / pics, build quality opinions, etc.. Your current article format is perfect for pro hardware and new tech reviews (7970 review, vertex 3 review, etc.) but your consumer / mobile reviews could be a lot more dense ( http://www.notebookcheck.net/Review-Samsung-305U1A... ) or at least a little bit less ugly (see http://www.theverge.com/2012/1/4/2677801/hp-envy-1... ).
This may have more to do with your page layout formatting than anything else though, even on my 1080p screen I have to do a -lot- of scrolling on your site and the dropdown navigation for your articles is a bit clunky. This site has a solid reputation and I trust the recommendations of Anand more than any other tech site, but you could really use a facelift that was a little bit less all-business-all-the-time a la the asus transformer prime :)
As for benchmarks a sleep, hibernate, wake from sleep, wake from hibernate could be really helpful.
coolhardware - Sunday, January 8, 2012 - link
Nice points here.+Notebookcheck definitely does a great job. :-)
-To me though, theverge layout doesn't read/flow nearly as nicely as Anandtech does right now... :-(
colonelclaw - Saturday, January 7, 2012 - link
Ok, it's very niche, but if you would like to try a real ball-busting CPU test I would recommend a combination of 3DSMax 2012 and VRay. It's the most popular combination in the 3D world at the moment and nothing sucks the life out of a PC better than a VRay render.Plus these days VRay is available for Max, Maya, Sketchup, Rhino, Softimage and probably more. It also runs on OSX under the apps available for that platform. Something to think about.
RoninX - Sunday, January 8, 2012 - link
I'd like to request a benchmark for battery life when gaming. This wouldn't need to be the full set of gaming tests, but just one or two games run at value, mainstream, and enthusiast settings.I own a high-end gaming desktop for gaming at home, but I also do a lot of business travel, and I like to having a laptop that can run games during long waits at the airport. Right now, I have a Dell XPS 15, which does pretty well, but in the future, I'd be interested to know the tradeoffs in battery life and performance for dedicated gaming laptops (e.g. Alienware) as well as high-end mainstream laptops (e.g. XPS).
I realize that laptop battery life is pretty poor across the board when running demanding 3D games, but for me, there's a big difference between 90 minutes (which I get from my XPS 15), and say, 20 minutes. With a spare battery, the former gives you 3 hours of gaming time -- more than enough for typical flight delays.
JarredWalton - Sunday, January 8, 2012 - link
Besides tripling the number of battery life tests (ouch!), there are other factors to consider. First, you need a "game test" that's repeatable. If you play Batman for a couple hours on a laptop, and then play it again for a couple hours, I'm not sure the load will be the same. Then again, I doubt that the difference between Value/Mainstream/Enthusiast and even different games will be much -- they all activate the GPU and put a decent load on the CPU, so they're pretty much the worst-case scenario.How about this one: start battery life timer, open Skyrim save, and let the laptop just sit there until it runs dead. (Note that Skyrim will have the camera start circling the player in third person mode after a minute or two of inactivity.) That's about as consistently repeatable as I can come up with, though not necessarily a realistic test case. I would also be willing to check battery life on one laptop at our three detail settings to verify whether there's a difference in battery life or not. I might also check out some other games to see if they show variance in battery life (and if I can find a good looping test). Let me see if we can come up with something reasonable -- and we'd also want to test performance on battery power, as most higher end GPUs really clamp down on maximum clocks when on DC power.
RoninX - Sunday, January 8, 2012 - link
That Skyrim test sounds good to me, and testing performance on battery power would also be very useful.Thanks!
joelypolly - Monday, January 9, 2012 - link
Would be nice to have a web based comparison table so you can compare things like viewing angles, general specs and performance.JamesAnthony - Thursday, January 12, 2012 - link
I was wondering if you would be able to add to the video tests for games more notebooks with Nvidia Quadro and ATI FireGL video cards, as Dell specifically & several other vendors tend to use those cards as the only option available in business level or professional level laptops & it would be nice to know how well you could expect your $1500 Business laptop $3,000 CAD laptop to play some games when you get home with it.For Example
AMD FirePro M8900 Mobility Pro
NVIDIA Quadro 3000M
NVIDIA Quadro 4000M
NVIDIA Quadro 5010M
AMD FirePro M5950 Mobility Pro
NVIDIA Quadro 1000M
NVIDIA Quadro 2000M
NVIDIA NVS 4200M