Designing Our Next GPU Test Suite

240 Comments

View All Comments

• Paulman - Monday, March 15, 2010 - link

  I agree with bdunosk! WoW has the largest player base in the world for any game. Yes, it's typically more CPU limited than GPU limited, so benchmarks might be more applicable to mid-range to lower-end cards. Thus, perhaps WoW benches should only be featured in those performance classes of GPU reviews. However, I think benchmarking something more intensive like boss raids or Eyeinifinity setups (probably a lot more repeatable than boss raids) at high resolutions and high AA could present a more GPU-limited scenario that WoW players would be interested in.
  
  I would also like to take this opportunity to reiterate that for performance/mid-range to lower-end cards, it would be BRILLIANT if you also paired them up with mid to low-end CPU's as well, since that is more realistic user rig than pairing a Core i7 Extreme with a Radeon HD 5650 or even a 5770, for example.
  
  Thanks for listening!

• moep - Sunday, March 14, 2010 - link

  Call of Duty: Modern Warfare 2 (FPS - everyone seems to play this)
  Team Fortress 2 (FPS - Source engine)
  Batman: Arkham Asylum (Action/Adventure - UE3 engine)
  Starcraft 2 beta (the RTS - bench using a taxing reference replay, use same beta version until release and then switch)
  DiRT 2 (Racing - DX11, tesselation)
  Crysis - Warhead (FPS - …but will it run crysis?)
  
  

• faxon - Sunday, March 14, 2010 - link

  im going to start off by posting a link to toms hardware since they did a review on the game's performance which i found rather puzzling. http://www.tomshardware.com/reviews/star-trek-onli...">http://www.tomshardware.com/reviews/star-trek-onli... . in this review, nvidia cards literally walked all over ATI cards (it's a TWIMTBP game btw), but the general over all performance in the game is comparable to crysis performance across the GPU bench test. the game is definitely heavily GPU limited, and i want to see what anandtech's expert testers can do to pound out how the game performs. would be interesting to see a separate test set for space and ground combat as well, since im sure the 2 perform differently (ground combat is definitely laggier than space combat)

• faxon - Sunday, March 14, 2010 - link

  ooh i almost forgot to add, you need to post max, avg, and min FPS for each benchmark. i base my card choice off min, avg, and % min FPS, since i want it to always perform at minimum of a certain level in my games for framerate stability. i also would be interested in seeing each video card tested on an amd and intel CPU based platform, as i know it has been proven here and elsewhere that some games behave differently with different cpu/gpu combinations even in games that werent CPU limited. it would also be good to test with different speeds of CPU within a certain generation to give a baseline for comparison since not everyone owns an EE i7 CPU. a lot of the users out there are using dual cores, and while im not saying that you should test with them, im saying that an i7 extreme isnt exactly a real world representation of what most users are playing with. most users are playing on stock clocked dual cores or cheap ass quads still, so if you really wanted real world data you would at least toss in an AMD and Intel bench, and at least 1 underperforming CPU so people can see how their CPU can effect their gaming experience. if anything, at least include the data in the anandtech bench at a later date, with a link to the bench in the article. this would get more users to use the bench when looking for information on products and it would be extremely useful for everyone to make recommendations based on an individual users level of hardware.

• iwodo - Sunday, March 14, 2010 - link

  Splitting GPU into 5 categories, Enthusiast, Performance, Mainstream, Entry, and IGX.
  
  Every GPU review would have an representative ( or averages ) from other Categories presents. In the Case where some would ruling the graph. Just simply mention its results. And Other then Numbers, % would be nice as well.
  
  So in a Benchmark, I could see, how this Performance GFX ( 100%), Igx is 10% ( Meaning Performance GFX is 10 times faster then IGFX in general.
  
  Other then Games, 2D Benchmark would be nice as well, as well as performance on Windows Aero Acceleration. CAD, Rendering, OpenCL, Photoshop, etc...
  
  All in all, i dont think there is anything we need to add to the test. We just need a different way of presenting these data to consumers and let them easily find the answer they want.

• derrida - Sunday, March 14, 2010 - link

  ...because gpus are not only for gaming.

• Ryan Smith - Sunday, March 14, 2010 - link

  GPGPU benchmarks are something we want to do, but don't expect to see any kind of comprehensive benchmark for it in the near future. We've been poking in to this one looking for an OpenCL/DirectCompute benchmark and thus far have come up empty handed.
  
  There's a somewhat wider range of programs that can use Stream or CUDA, but that means we have to throw out cross-vendor comparisons. So if we used such benchmarks, it would be sparingly.

• derrida - Sunday, March 14, 2010 - link

  Have not tried it but it could be a starting point:
  http://www.sisoftware.net/?d=news&f=opencl_rel...">http://www.sisoftware.net/?d=news&f=opencl_rel...
  
  For me the choice of a new gpu depends solely on the available flops [single|double] as I do work on hpc. Don't know if I am a minority here.

• falc0ne - Monday, March 15, 2010 - link

  agree with no synthetic benchmarks and pointing out CPU/GPU bottlenecks where they appear
  regards

• Holly - Sunday, March 14, 2010 - link

  I'd like to see fraps based benchmarks dropped. There are way too many "random" differences brought in to one run compared to other run of the test if the game is not running in benchmark mode on its own (ie. always exactly the same scene).