Introduction

For those who follow our Linux reviews, we have made a lot of headway in the last two months. Our benchmarking has improved, our graph engine is top notch and we are working closely with all the major manufacturers to bring a definitive resource for Linux hardware to our readers. Today, we want to introduce everyone to our GPU Linux benchmarks and how we will run them in the future. This isn't a comparative analysis yet, but we won't keep you waiting long for that.

The adherent flaw with any benchmark is that you, the reader, only receives a sampling of data - engineers and statisticians alike call this "data compression". When we sample data from a timedemo and format it into an average frames per second, we lose all sort of valuable data, such as what the lowest frames per second was, what the highest was, when the largest dip in FPS had occured, what the image looked like, and the list goes on. There have been a few attempts to convey more than just an average FPS in video benchmarks, most notably with FRAPS. FRAPS does not entirely address the issue of reproducibility and FRAPS runs on Windows only.

Fortunately, we have been graced with some very talented programmers who worked with us to build a benchmarking utility similar to FRAPS (on Linux) that we may eventually port over to Windows as well. Consider this to be our experiment in advancing our benchmarking methods while using Linux as our guinea pig. Eventually, we anticipate releasing the benchmark complete with source to the public.

Here is how our utility works, as explained by the lead developer, Wiktor Kopec.
"The program computes frames per second for an application that uses OpenGL or SDL. It also takes screenshots periodically, and creates an overlay to display the current FPS/time.

"This is accomplished by defining a custom SwapBuffers function. For executables that are linked to GL at compile time, the LD_PRELOAD environment variable is used to invoke the custom SwapBuffers function. For executables that use run-time linking - which seems to be the case for most games - a copy of the binary is made, and all references to libGL and the original glXSwapBuffers function are replaced by references to our library and the custom SwapBuffers function. A similar procedure is done for SDL. We can then do all calculations on the frame buffer or simply dump the frame at will."
You can read more about SDL and OpenGL. SDL is a "newer" library bundled with most recent Linux games (Medal of Honor: AA, Unreal Tournament 2004). In many ways, SDL behaves very similarly to DirectX for Linux, but utilizes OpenGL for 3D acceleration.

Why Average Frames Per Second Can Be Misleading
POST A COMMENT

23 Comments

View All Comments

  • yelo333 - Friday, September 24, 2004 - link

    #12, as they mentioned in the article, that was with FRAPS...FRAPS does not run on linux, and, IIRC, can give invalid results sometimes. The point of this tool was gpu benchmarking for linux. Only as a side note did they say they might port it to windows. Reply
  • AlphaFox - Friday, September 24, 2004 - link

    I recall seeing these graphs in past reviews, is this really new? Reply
  • yelo333 - Friday, September 24, 2004 - link

    Very nice work...nice to see some linux stuff going on here ;)

    Maybe this will invigorate ATI to make better drivers for linux...Image IQ + speed of drivers are terrible - In my informal tests on a radeon 9000np, the linux drivers have about 1/4 the image IQ/speed of the windows versions...
    Reply
  • Jalf - Friday, September 24, 2004 - link

    Wow, great work... :)
    So, now you just need to port it to Windows as well... ;)
    Reply
  • bdykes - Friday, September 24, 2004 - link

    Nice article! Is there any chance of making the benchmark tool available for download? :) Reply
  • bdykes - Friday, September 24, 2004 - link

    Reply
  • mercador - Friday, September 24, 2004 - link

    Anyone knows when it will be the next PC round up? Reply
  • mercador - Friday, September 24, 2004 - link

    Really good work, hope to see the mext round up, very anxious to choose a gfx card using your method

    Keep the good work
    Reply
  • sprockkets - Friday, September 24, 2004 - link

    Perhaps people need to know where OpenGL came from, hint, perhaps OPEN can help you with that. And then figure out that Linux can run OpenGL games just fine. Reply
  • appu - Friday, September 24, 2004 - link

    You guys are fantastic! Carry on the good work, and yes, I'm really looking forward to the final roundup. Reply

Log in

Don't have an account? Sign up now