Designing Our Next GPU Test Suite

240 Comments

View All Comments

• Targon - Monday, March 15, 2010 - link

  There is a place to see how well a video card/GPU will run with limited CPU power though. At the least, a Phenom 2 X4 955 is considered a "fast enough" CPU to see the performance of a video card in most situations. This is why I also posted that it would be good to see GPU performance on different chipsets from time to time. If every test is done on an Intel i7 with Intel chipset, how do we know that a given GPU doesn't have issues with the Intel chipset?
  

• piroroadkill - Monday, March 15, 2010 - link

  I agree with this. It's a GPU review, not a CPU review.

• dubyadubya - Sunday, March 14, 2010 - link

  +1

• Crusse - Sunday, March 14, 2010 - link

  I'd be interested in that too, but I guess it'd add quite a lot of time to the benchmarking process. Maybe only add one or two last generation CPU's.
  
  I wonder if monitoring GPU load in different games is a reliable way to measure the relative size of a bottleneck...

• jimhsu - Monday, March 15, 2010 - link

  Is there a way to generate a 2 dimensional contour plot of GPU performance vs. CPU performance?
  
  Let's say you have GPUs ranked from last to first on the x-axis, and CPUs ranked last to first on the y-axis. Color resulting performance by blue (worst) to red (best). The result, assuming perfect scaling, should be blue in the bottom left corner, and red in the top right corner. (example: http://yfrog.com/ja33420199j">http://yfrog.com/ja33420199j )
  
  In contrast, a GPU limited game would look like this: http://yfrog.com/jagputj">http://yfrog.com/jagputj
  
  The difference is subtle, but it's there - in the 2nd image, notice that improvements to CPU speed don't affect performance all that much, while improvements to GPU speed matter a lot.
  
  There's probably a much easier way to explain this, but no site that I know if has ever attempted a multidimensional visualization like this before. I don't know just how much sample data you have to make this possible, but it would be an exceedingly interesting feature.
  
  
  
  
  

• ET - Sunday, March 14, 2010 - link

  I agree that there's no need for synthetic benchmarks. Regarding games, I think that a suite (and a set testing rig) is good for a comparison over several reviews, but I think that adding some other games here and there won't hurt. In particular, I'd be interested in games that particularly favour a specific graphics card architecture. I'd like Anandtech to test a lot of games on a limited number of cards, and if there's a game that's interesting (GPU limited but performs well on specific cards), test it on more cards.
  
  In general, though, just have a test suite that's made of best selling games which are GPU bound.

• ComputerGuy2006 - Sunday, March 14, 2010 - link

  I agree with no synthetic benchmarks, keep it with real games.
  
  More game benchmarks the better, I do understand time is an issue. Maybe pick 4-5 main games (with 2-3 resolutions) then have about 5-6 EXTRA games were you only bench at a single resolution (such as 1920x1200).
  
  This way we not only know how the cards work on the new games on different resolutions, but we also get a sense (with 1 res) on how they perform on other/older games compared to other cards.
  
  And I highly recommend you keep working on the "Bench (beta)". I think it has potential for interesting feature such as: The ability to customize review pages based on the user reading it. So for example right now I have an "4850 512MB". If this site was dynamic enough, it would be possible to have that video card included as part of the benchmarks in the review. This way on every article GPU review release id instantly be able to compare it with my cards (or cards on my wishlist) while reading the review.

• JonnyDough - Monday, March 15, 2010 - link

  Also in agreeance with game benchmarks only. Also would love to add that Fallout 3 or Oblivion needs to be in there - or any game that uses these types of engines which is heavy on shaders and textures.

• JonnyDough - Monday, March 15, 2010 - link

  Maybe you could "group" games somehow according to what strain they put on certain components. Such as Oblivion/Fallout/Dragon Age? - find a way to make a "group" of tests and then label it "heavy on textures and shaders RPGs" or something easier. That way when we want to play a type of game we have a list of games that gives us some idea of how a card should perform. Again as I listed before, all we really need to know is will this card suit my needs or won't it? If a card does well in FPS but not in other types of games, but all I play is FPS then I just want a card that will do well in games. Recommendations for cards that might perform well in future games would be helpful as well - according to where you see current developers going with PC gaming.

• jimhsu - Monday, March 15, 2010 - link

  I actually think Fallout/Oblivion aren't all that heavy on the GPU side per se (they depend a lot on CPU also). I would recommend splitting "game" benchmarks into CPU focused (RTSes like Supreme Commander, and I would include Fallout/Oblivion in here), and GPU focused (Crysis, most FPSes, etc) benchmarks. Or to make this easier, you could implement general categories: FPS (Crysis, L4D2, MW2), RTS (Supreme Commander, Starcraft 2 (when it comes out), Battleforge), RPG (Mass Effect, Dragon Age), Simulation (be creative here), etc.