Designing Our Next GPU Test Suite

240 Comments

View All Comments

• vectorm12 - Sunday, March 14, 2010 - link

  Now I for one would like to see the very compute intensive Football Manager 2010 make it into your suite.
  
  As it's more or less completely CPU based it's not very good for seperating one GPU from another but on the other hand it's shown to be fairly well threaded in later versions suggesting it would be an interesting choice when determening CPU performance.
  
  Other than that I'm gonna agree with most other here. No more synthetic benchmarks.

• Baron Fel - Sunday, March 14, 2010 - link

  BFBC2
  Napoleon Total War
  Metro 2033
  
  all intensive, system stressing games.
  
  Mass Effect 2 and TF2 would be relevant but they dont need much power at all.

• pensuke89 - Monday, March 15, 2010 - link

  I'd suggest relevant and popular games to be benched in feature article. That way viewers will be able to see the performance of the game and know which is the minimum required GPU to play the game at satisfactory settings.
  
  Just leave those graphical intensive games to the GPU benchmark suite.

• pensuke89 - Sunday, March 14, 2010 - link

  3DMark should always be included in the bench. Why? Because its free and provide a good base on the graphic card performance.
  1) Sure, graphic card vendor optimize the drivers for those synthetic benchmarks, but if both nVIDIA and ATI optimizes it, then its fair.
  2) Its available for free to download. If the benchmark comes out with games like Crysis and someone don't actually own Crysis, then how do they compare their cards and the benched cards?
  
  I second the suggestion to include some function to auto-detect the user's graphic card and display the scores for the card in the benchmark.
  
  Its also important to keep using the top-end CPU to avoid bottlenecking. If I want to check for CPU performance, I'd go to the CPU section. If they include too many processors, that would consume a lot of their time.
  
  If its possible, I would like for Anandtech to give viewer to be able to download benchmark files (so user can run it themselves using the same suite).
  
  Yes
  1) S.T.A.L.K.E.R.: Call of Pripyat
  2) Crysis or Crysis 2
  3) Programs that use GPU for other tasks beside gaming (Its GPGPU era now, so I think this should be included.)
  4) DIRT2 (DX11)
  5) Metro 2033
  6) Graphic Card power usage (not whole system)
  7) Battlefield BC2
  8) Synthetic benches (read above)
  
  No
  1) Basically any UE3 engine games (OK, maybe keep 1 for representing the much used UE3 engine)
  2) Source games
  3) COD4MW, COD4MW2
  4) API specific programs (like Badaboom CUDA)
  5) enabling PhysX (unless its a dedicated section, say, Physics GPU)

• Dracusis - Sunday, March 14, 2010 - link

  The beauty of PC gaming is that we get a massive variety of games to play on our video cards. We get a lot of the big budget blockbuster cross-platform titles that also hit the 360/PS3 and we also get lots of nice exclusive RTS, MMOs, RPGs and heaps of Indie titles too.
  
  2 x MMO
  2 x FPS
  2 x RTS
  2 x RPG
  
  You'd want games that stress a machine, work with all features if possible (AA ect) and you'd want to cover some of the more popular 3D engines like Unreal Engine 3, VALVe Source ect.
  
  For RTS you could use Supreme Commander II, Napoleon Total War or Starcraft II if it's released before your benchmark update and Sins of a Solar Empire to cover the Indie aspect of PC gaming.
  
  For RPGs you'd probably want to mix subgenres, Bioshock/Bordelrands Action RPG paired with Fallout/DragonAge.
  
  For MMOs you'd want to stick to solid players that have been around for a while like WoW, Warhammer and Age of Conan or maybe risk a newer Cryptic game like Star Trek Online (same engine as Champions Online).
  
  For FPS you'd want to test both ends of the scale in terms of scope, Bad Company 2 and Left 4 Dead 2 would make good choices here.
  
  
  

• Paulman - Monday, March 15, 2010 - link

  +1 agree
  
  Although I would weight the FPS-genre a little more heavily, since so many more people play it. Maybe at the expense of RTS and RPG benchmarks, if necessary.

• OzoZoz - Sunday, March 14, 2010 - link

  I'd like to see more OpenGL benchmarks. As an OpenGL developer, I would like to see how stable and optimized the different GPUs are with respect to OpenGL. If there are not enough good OpenGL games, you could use some real-time simulation softwares, such as something based on OpenSceneGraph or Gizmo3D.
  
  Also, I would like to see some OpenGL driver compliance analysis (with respect to the OpenGL specification), as well as performance comparisons for precise OpenGL features that are not typically used by popular games (ex: glReadPixels, glTexSubImage, arb_multisample, complex GLSL shaders, etc.).
  
  In other words, I would like to see how these graphics cards (and drivers) can be used for OpenGL development of real-time, performance-oriented applications. And at the same time, push these companies into providing better OpenGL support.

• Luminair - Monday, March 15, 2010 - link

  I endorse the recommendation of including an OpenGL game benchmark.

• Alastayr - Sunday, March 14, 2010 - link

  I'll try and keep it short.
  
  Resolutions to bench:
  - 1680x1050
  - 1920x1080
  - 2560x1600
  
  Don't include Eyefinity / the SLI alternative yet, make it a separate one-off test. At the moment it's way too niche to be useful. Enthusiasts can already gather a lot of easily available data from various forums and other sites if they're interested.
  
  
  Games to bench:
  
  - keep Crysis: Warhead (just for the sake of it)
  
  - maybe drop Far Cry 2; the engine has had no impact on the market and it's a forgotten game already
  
  - keep Dragon Age: Origins; it's a bit less taxing than The Witcher but more contemporary
  
  - keep Dawn of War II; maybe upgrade to Chaos Rising if it has an in-built benchmark especially since DoW II was a TWIMTBP and Chaos Rising is ATi sponsored now (implications for CF?)
  
  - add Dawn of Discovery (Anno 1404); it's a beautiful game and it's taxing as hell
  
  - add Battlefield: Bad Company 2; Frostbite is finally on PC and it's here to stay (MoH MP; BF3)
  
  - keep Batman: Arkham Asylum; it's representative of UE3 games (Mass Effect 2, Mirror's Edge etc.)
  
  - consider the GTA IV - Episodes; the stand-alone addon should prove quite taxing to many GPUs
  
  - keep the Source Engine and go with L4D2; especially viable for low-end and mid-range
  
  - add DiRT 2; you guys need a racing game, the engine is the basis for the new F1 game and it's DX11
  
  - add Metro 2033 or Stalker: Call of Pripyat; exotic, possibly less polished engines and overall good indicator for new unusual games as they're heavily into DX11
  
  - consider Just Cause 2 when it comes out; it looks like a benchmark waiting to happen...
  
  
  One really nice feature that I'd appreciate would be sound clips of different cooling solutions, maybe in a YouTube video that includes talking so we'd get a good measure of relative loudness and tone. The current dB(A) measurements are not very meaningful in my opinion.
  
  I really love the idea of a comprehensive overview and I'm happy with the current benchmark suite but only half of the games in there seem indicative to me. And good call on excluding synthetics; I don't want this to turn into the early noughts again. Many people seem to have forgotten that time period already.

• rjc - Sunday, March 14, 2010 - link

  Hello,
  
  Sorry being late to this...
  
  Can anandtech please consider improving their measurment of graphic card power usage? Have a look at this German site for an example:
  http://ht4u.net/reviews/2009/power_consumption_gra...">http://ht4u.net/reviews/2009/power_consumption_gra...
  
  Also, as well as higher power use cards are now running at higher frequencies. This gives the potential that not carefully designed cards (especially if overclocked) will start emitting excessive EMI causing errors and/or failure in other components(ie they irradiate the other components in the computer). At the moment any failure of the computer is just assumed to be overheating. This needs to be studied more, sadly nobody seems to be checking this at present. If graphic cards are used increasingly for long running computational tasks which require high accuracy(ie Cuda or OpenCl applications) then reliability becomes much more important.
  
  The above 2 issues fit into basic requirements, i think they should come ahead of any performance related testing.
  
  Thankyou for considering the above.