Introduction

For those who follow our Linux reviews, we have made a lot of headway in the last two months. Our benchmarking has improved, our graph engine is top notch and we are working closely with all the major manufacturers to bring a definitive resource for Linux hardware to our readers. Today, we want to introduce everyone to our GPU Linux benchmarks and how we will run them in the future. This isn't a comparative analysis yet, but we won't keep you waiting long for that.

The adherent flaw with any benchmark is that you, the reader, only receives a sampling of data - engineers and statisticians alike call this "data compression". When we sample data from a timedemo and format it into an average frames per second, we lose all sort of valuable data, such as what the lowest frames per second was, what the highest was, when the largest dip in FPS had occured, what the image looked like, and the list goes on. There have been a few attempts to convey more than just an average FPS in video benchmarks, most notably with FRAPS. FRAPS does not entirely address the issue of reproducibility and FRAPS runs on Windows only.

Fortunately, we have been graced with some very talented programmers who worked with us to build a benchmarking utility similar to FRAPS (on Linux) that we may eventually port over to Windows as well. Consider this to be our experiment in advancing our benchmarking methods while using Linux as our guinea pig. Eventually, we anticipate releasing the benchmark complete with source to the public.

Here is how our utility works, as explained by the lead developer, Wiktor Kopec.
"The program computes frames per second for an application that uses OpenGL or SDL. It also takes screenshots periodically, and creates an overlay to display the current FPS/time.

"This is accomplished by defining a custom SwapBuffers function. For executables that are linked to GL at compile time, the LD_PRELOAD environment variable is used to invoke the custom SwapBuffers function. For executables that use run-time linking - which seems to be the case for most games - a copy of the binary is made, and all references to libGL and the original glXSwapBuffers function are replaced by references to our library and the custom SwapBuffers function. A similar procedure is done for SDL. We can then do all calculations on the frame buffer or simply dump the frame at will."
You can read more about SDL and OpenGL. SDL is a "newer" library bundled with most recent Linux games (Medal of Honor: AA, Unreal Tournament 2004). In many ways, SDL behaves very similarly to DirectX for Linux, but utilizes OpenGL for 3D acceleration.

Why Average Frames Per Second Can Be Misleading
POST A COMMENT

23 Comments

View All Comments

  • skiboysteve - Friday, September 24, 2004 - link

    this is absolutly excellent work Reply
  • MNKyDeth - Friday, September 24, 2004 - link

    I cannot wait for the Linux gpu roundup. I have already been waiting in anticipation for weeks since I first heard there was gonna be one.
    I know most people do not feel linux is a gaming OS but it is the only OS I use and I use it mainly for gaming so this should be very informative for someone like me.
    I must applaud Anandtech for there early adoption in doing reviews on linux and putting out consistent reviews over time.
    Thx, Anandtech, your the #1 tech site on the web imo.
    Reply
  • Avalon - Friday, September 24, 2004 - link

    Although not a Linux user, I find this a much more informative approach to benching video cards. If it's worth anything, I approve wholeheartedly. Reply

Log in

Don't have an account? Sign up now