Constructing a database benchmark (average load)

Our first new benchmark was custom written in .NET, using ADO.NET to connect to the database. The AnandTech Forums database, which is over 14GB in size at the time of the benchmark, was used as the source database. We'll dub this benchmark tool "SQL Loader" for the purposes of discussing what it does.

SQL Loader allows us to specify the following: an XML based workload file for the test, how long the test should run, and how many threads it should use with which to load the database. The XML workload file contains queries that we want executed against the database, and some random ID generator queries that populate a memory resident array with ID's to be used in conjunction with our workload queries. The purpose of using random ID's is to keep the test as real-world as possible by selecting random data. This test should give us a lot of room for growth, as the workload can be whatever we want in future tests.

Example workload:

< workload>

< !--- A SAMPLE WORKLOAD QUERY THAT RETURNS ALL THE FIELDS FROM THE PRIVATEMESSAGES TABLE RANDOMLY --->

<query>

<code>select * from privatemessages where imessageid = @pmessageid</code>

<type>read</type>

<randkey>pmessageid</randkey>

</query>

<!--- RANDOM ID GENERATOR FOR SELECTING RANDOM PRIVATE MESSAGES --->

<randomid>

<rcode>select imessageid,newid() as pmsgid from privatemessages order by pmsgid</rcode>

<name>pmessageid</name>

</randomid>

< /workload>


A screenshot of the SQL Loader

Test Information

The workload used for the test was based on every day use of the Forums, which are running FuseTalk. We took the most popular queries and put them in the workload. Functions, such as reading threads and messages, getting user information, inserting threads and messages, and reading private messages, were in the spotlight. Each iteration of the test was run for 10 minutes, with the first being from a cold boot. SQL was restarted in between each test that was run consecutively.

The importance of this test is that it is as real world as you can get; for us, the performance in this test directly influences what upgrade decisions we make for our own IT infrastructure.

Hyper Threading and The Tests AnandTech Forums Database Test Results
Comments Locked

58 Comments

View All Comments

  • Rand - Friday, May 20, 2005 - link

  • perlgreen - Tuesday, June 1, 2004 - link

    Is there any chance that you guys could do more tests and benchmarking on Linux for IT Computing/Servers? I really like your site, but it'd be really nice if there would be more stuff for fans of the Penguin!

    cheers,

    Campbell
  • ragusauce - Friday, March 5, 2004 - link

    #54
    We have been building from source and trying different options / debug versions...
  • DBBoy - Friday, March 5, 2004 - link

    #47 - In OLAP, or poorly indexed environments where the amount of data exceeds the 4 MB L3 cache of the Xeons the Opteron is going to shine even more with it's increased memory bandwidth.

    Assuming you do not bottleneck on the disk IO the SQL cache/RAM will be utilised much more thus putting more of a burden on the FSB of the Xeons in addition to allowing the Opteron's memory bandwidth to display it's abilities.
  • Jason Clark - Friday, March 5, 2004 - link

    ragusauce, using binaries or building from source?

    Cheers
  • ragusauce - Friday, March 5, 2004 - link

    We have been doing extensive testing of MySQL64 on Opteron and have had problems with seg faults as well.
  • zarjad - Thursday, March 4, 2004 - link

    Great, thanks.
    My thoughts:
    In this type of application you are likely to use more than 4GB memory.
    Memory bandwidth should matter because you will be doing a lot of full table scans (as opposed to using indexes).
  • Jason Clark - Thursday, March 4, 2004 - link

    zarjad, I'll get back to you on that question I have some thoughts and amd discussing them with one of the guys that worked with us on the tests (Ross).
  • zarjad - Thursday, March 4, 2004 - link

    Jason, any comments on #47?
  • Jason Clark - Wednesday, March 3, 2004 - link

    The os used was windows 2003 enterprise which does indeed support NUMA. So NUMA was enabled.. this was covered in an earlier response.

Log in

Don't have an account? Sign up now