Constructing a Database Benchmark (average load)

Our first benchmark was custom-written in .NET, using ADO.NET to connect to the database. The AnandTech Forums database, which is over 14GB in size at the time of the benchmark, was used as the source database. We'll dub this benchmark tool "SQL Loader" for the purposes of discussing what it does.

SQL Loader allows us to specify the following: an XML based workload file for the test, how long the test should run, and how many threads it should use with which to load the database. The XML workload file contains queries that we want executed against the database, and some random ID generator queries that populate a memory resident array with ID's to be used in conjunction with our workload queries. The purpose of using random ID's is to keep the test as real-world as possible by selecting random data. This test should give us a lot of room for growth, as the workload can be whatever we want in future tests.

Example workload:

< workload>

< !--- A SAMPLE WORKLOAD QUERY THAT RETURNS ALL THE FIELDS FROM THE PRIVATEMESSAGES TABLE RANDOMLY --->

<query>

<code>select * from privatemessages where imessageid = @pmessageid</code>

<type>read</type>

<randkey>pmessageid</randkey>

</query>

<!--- RANDOM ID GENERATOR FOR SELECTING RANDOM PRIVATE MESSAGES --->

<randomid>

<rcode>select imessageid,newid() as pmsgid from privatemessages order by pmsgid</rcode>

<name>pmessageid</name>

</randomid>

< /workload>


A screenshot of the SQL Loader


Test Information

The workload used for the test was based on every day use of the Forums, which are running FuseTalk. We took the most popular queries and put them in the workload. Functions, such as reading threads and messages, getting user information, inserting threads and messages, and reading private messages, were in the spotlight. Each iteration of the test was run for 10 minutes, with the first being from a cold boot. SQL was restarted in between each test that was run consecutively.

The importance of this test is that it is as real world as you can get; for us, the performance in this test directly influences what upgrade decisions we make for our own IT infrastructure.

Hyper Threading and The Tests AnandTech Forums Database Test Results
Comments Locked

44 Comments

View All Comments

  • knitecrow - Monday, September 13, 2004 - link

    Something caught my eye:

    "The difference in performance between the Opteron 250 and the Nocona 3.6 is approximately 2%, which is also our tolerance for deviation between test runs."

    I know its an unfair charge to level against this site, as everyone does it, but why aren't there any error bars?

    In my freshman physics class they really made the point that graphs with out error bars are useless.... and the work was rubbish.

    Just by looking at the graph I can't tell the variance or the margin of error. Wouldn't it be helpful if we had the information?

    Anyway, just a thought.
  • saechaka - Monday, September 13, 2004 - link

    the message is clear the beat must go on
  • MAME - Monday, September 13, 2004 - link

    AMD > Intel once again
  • Shadowmage - Monday, September 13, 2004 - link

    The Opteron applies extreme beats yet again! :D

Log in

Don't have an account? Sign up now