Benchmarking the Matrix

With the updated test suite, we’re also losing some points of reference to our back catalog of laptops. Obviously, the biggest change is in the gaming results, and we decided to take one of our recently reviewed laptops for a spin using the new benchmark suite. (We may look at adding a couple more lower end laptops from late 2011 to the charts as well in the near future.) ASUS was kind enough to let us hang onto the G74SX until the new suite was complete, and given the reasonably high-end hardware and continued availability, it makes for a good starting point for our 2012 laptop results. We updated to the latest NVIDIA drivers (290.56 at the time of testing) and ran through all of our gaming tests. You can find the complete results in Mobile Bench, and the games are all grouped under the Mobile Gaming 2012 category; since we only have one laptop tested right now, we’ve summarized the gaming scores below.

In our 2011 gaming suite, the ASUS G74SX—and NVIDIA’s GTX 560M—proved capable of handling the majority of games at our Enthusiast settings and 1080p while still breaking 30 FPS. With some of the latest titles at similar “maxed out” settings, frame rates now drop below 30FPS in five of the seven titles, but remember that our new Enthusiast is equivalent to last year’s “Ultra”. There are certainly other games that will tax the GTX 560M, and our recommendation is that you consider disabling antialiasing or dropping the quality down a notch if you want higher frame rates, but in general the GTX 560M is still a good solution for notebook gamers.

Closing Thoughts

As a sci-fi buff, it’s pretty exciting to see the rapid pace of advancement over the last few years. Today’s smartphones pack about as much power in a small portable device as the PCs we used less than a decade ago. If you’ve ever dreamed of real-world tricorders and holodecks—or maybe cyberspace and Ono-Sendai decks—they’re getting tantalizingly close. Maybe we won’t have exactly what the sci-fi writers of 20 or 30 years ago envisioned, but we’re definitely shedding the wires and I look forward to seeing where we will be in another ten years!

Back on topic, no benchmark suite can ever (reasonably) contain every performance metric, and we do understand that mobile gaming is still a small piece of the larger mobility pie. Even so, it’s still important to consider mobile GPU performance, and with the improving nature of integrated graphics we felt it was time to finally ditch the 2006-era graphics quality settings and shoot for something more visually appealing. Our mobile gaming suite now represents some of the latest DX11 titles, and even at our Value settings all of the games look quite good. If you’re looking for basic gaming capabilities, all you really need is a mobile GPU that can hit 30 FPS at our Value settings in all seven titles and you should be set. If you’re after higher quality and higher resolutions, you’ll want something more than midrange GPUs, but be prepared to pay the price—both in terms of cost as well as in terms of notebook size.

With the updated laptop benchmarks now in place, we’re still early enough in 2012 that if you can make a good case for other benchmarks that we haven’t included we’re willing to consider adding a couple more tests. Remember that the goal is to provide a reasonable test suite from which you can estimate performance in other similar benchmarks, so adding three more video encoding tests isn’t really going to add much; on the other hand, if there’s a class of application you don’t feel our test suite adequately covers, sound off in the comments.

As a final thought, I’ve been the head laptop tester at AnandTech since early 2006. While we have frequently heard about the increasing importance of laptops in the overall computer market, the past two years have really shown tremendous growth. We had seven mobile articles on AnandTech in 2006, 15 in 2007 and 2008, and 32 in 2009. That’s pretty reasonable, but then in 2010 we had a whopping 107 mobile articles and 2011 eclipsed that with 166 articles. Wow! Granted not all of the articles in the past two years are about laptops, and we've had a lot of shorter articles in the past two years, but however you want to view it one thing is eminently clear: mobile devices are now well and truly established and our increased coverage reflects that. It’s also worth noting that Intel’s Sandy Bridge and AMD’s Llano launches were both more about the mobile sector than about desktops, and the upcoming Ivy Bridge, Trinity, and Haswell appear to continue that trend.

Here's looking forward to another awesome year in the mobile space, kicking off with CES next week. Hint: besides the usual plethora of large displays and 3D demonstrations, CES is all about smartphones, tablets, and laptops. (I almost feel sorry for Brian...almost.)

All New Gaming Test Suite
POST A COMMENT

48 Comments

View All Comments

  • codedivine - Friday, January 06, 2012 - link

    Can you include the Chromium compilation test that you to do for CPU tests?
    And I would very much like to see Blender too if possible.
    Reply
  • JarredWalton - Friday, January 06, 2012 - link

    I need to see if I can get Anand (or someone else) to walk me through the entire Chromium compilation process. I tried to get that working at one point and after wasting several days with nothing to show for it I gave up. LOL. (Yes, I followed a guide, but even then it didn't work for me.) If someone has a step-by-step guide that they: A) know works, and B) is free (open source) for all needed tools, and C) works with Windows 7 64-bit... post a link. Remember, not only do I need to figure out how to run the compile test, but so does Dustin and anyone else we have doing laptop reviews. :-) Reply
  • Conficio - Friday, January 06, 2012 - link

    I'd second that. A compilation test would be great.

    I'd even love to see some sort of Java test, may be an Eclipse Import a large project from zip and wait until it has finished updating the workspace.
    Reply
  • coolhardware - Friday, January 06, 2012 - link

    I realize it is not as demanding as many of the other games, but so many people play it (and will play it for years to come) that is sure seems like it deserves a spot.

    :-(

    Still, thanks for keeping your benchmark suites fresh and relevant!

    PS if anybody has tips/suggestions on the best <a href="http://www.jdhodges.com/2011/06/best-starcraft-2-g... for SC2</a> please let me know, thanks! [not talking about bells and whistles...but rather comfort, performance and reliability]
    Reply
  • coolhardware - Friday, January 06, 2012 - link

    Sorry about that malformed URL. Here's the correct one: http://www.jdhodges.com/2011/06/best-starcraft-2-g...

    PS any other games people are missing or games that you are excited to see WERE added?
    Reply
  • JarredWalton - Friday, January 06, 2012 - link

    Ultimately, Dustin and I decided to drop SC2 from the list because it:

    1) doesn't scale at all with more than 2 CPU cores
    2) is rather a pain to benchmark
    3) is DX9 only
    4) runs more than fast enough at all but the most demanding settings

    Basically, it's more of a CPU benchmark in most cases, as even with lots of stuff happening on screen the two-core nature of the engine doesn't allow it to scale that well. Sure, at Ultra quality with 4xAA forced on in your drivers (another pain to deal with), it drops FPS down to around 23 on the G74SX, but that still doesn't address the other items I'm concerned with. I figure if Civ5 and TWS2 run on a laptop, SC2 will run fine as well.
    Reply
  • coolhardware - Friday, January 06, 2012 - link

    Cool, thanks for listing those factors.

    It makes a lot more sense to me now to drop it, since it sounds like a real PITA to benchmark :-) Also, you raise a good point that if higher end games play okay on a particular laptop then SC2 should be good to go...

    Appreciate all the hard work you guys do!
    Reply
  • zebrax2 - Friday, January 06, 2012 - link

    Maybe you could include at least rage? Reply
  • JarredWalton - Friday, January 06, 2012 - link

    We had intended to include Rage, right up to the point where it was released and we discovered that it is largely useless as a benchmark:
    http://www.anandtech.com/show/4970/rage-against-th...
    Reply
  • ltcommanderdata - Saturday, January 07, 2012 - link

    Could you consider including Brink? Admittedly it's id Tech 4, but heavily modified to make use of OpenGL 3.1. It also uses virtual texturing like RAGE. The upcoming Prey 2 also uses id Tech 4 so performance in Brink does have forward looking relevance. Reply

Log in

Don't have an account? Sign up now