Constructing a Database Benchmark (average load)

Our first benchmark was custom-written in .NET, using ADO.NET to connect to the database. The AnandTech Forums database, which is over 14GB in size at the time of the benchmark, was used as the source database. We'll dub this benchmark tool "SQL Loader" for the purposes of discussing what it does.

SQL Loader allows us to specify the following: an XML based workload file for the test, how long the test should run, and how many threads it should use with which to load the database. The XML workload file contains queries that we want executed against the database, and some random ID generator queries that populate a memory resident array with ID's to be used in conjunction with our workload queries. The purpose of using random ID's is to keep the test as real-world as possible by selecting random data. This test should give us a lot of room for growth, as the workload can be whatever we want in future tests.

Example workload:

< workload>

< !--- A SAMPLE WORKLOAD QUERY THAT RETURNS ALL THE FIELDS FROM THE PRIVATEMESSAGES TABLE RANDOMLY --->

<query>

<code>select * from privatemessages where imessageid = @pmessageid</code>

<type>read</type>

<randkey>pmessageid</randkey>

</query>

<!--- RANDOM ID GENERATOR FOR SELECTING RANDOM PRIVATE MESSAGES --->

<randomid>

<rcode>select imessageid,newid() as pmsgid from privatemessages order by pmsgid</rcode>

<name>pmessageid</name>

</randomid>

< /workload>


A screenshot of the SQL Loader


Test Information

The workload used for the test was based on every day use of the Forums, which are running FuseTalk. We took the most popular queries and put them in the workload. Functions, such as reading threads and messages, getting user information, inserting threads and messages, and reading private messages, were in the spotlight. Each iteration of the test was run for 10 minutes, with the first being from a cold boot. SQL was restarted in between each test that was run consecutively.

The importance of this test is that it is as real world as you can get; for us, the performance in this test directly influences what upgrade decisions we make for our own IT infrastructure.

Hyper Threading and The Tests AnandTech Forums Database Test Results
Comments Locked

44 Comments

View All Comments

  • karlreading - Monday, September 13, 2004 - link

    #31
    whilst i agree with what you are saying in principle, perhaps a polite email to the author(s) of any article(s) pointing out the mistake(s). Would this not be more appropriate? I personally find that when one person points out such mistakes on a public forum, another 10 people throw in 2 pence worth and before you know it everybody starts to pick holes and loses site of the actual topic - like i have now so i'll shut up!!! :)
  • Jason Clark - Monday, September 13, 2004 - link

    In regards to the motherboard questions, we used a Thunder K8W board, I'll ensure the article reflects that. We did some testing on Numa, and as of yet see any difference at all in numbers.

    Cheers
  • Jason Clark - Monday, September 13, 2004 - link

    Since the broke wind pun seems to bring about so much attention.. I've changed the wording :).

  • johnsonx - Monday, September 13, 2004 - link

    @karlreading,

    I may be wrong, but I'd like to think those who write these articles actually appreciate polite corrections to grammar & wording missteps. I mostly don't bother unless the mistake either leads one to a conclusion other than what was intended, or if the mistake is potentially embarassing as is the case for 'broke wind'. Most websites contain such constant and gross errors of this kind that they are unreadable, but AT maintains a pretty high standard as all serious publications, electronic or printed, should. In most cases where folks have pointed out a mistake in an article, the AT staff has seen fit to correct it.

    No one has suggested that the article is less valid because "slightly suspect use of wording".

    All that said, for all I know the author(s) may have intended the wording used, or may have decided they like the unintended wording even if it is a bit 'suspect'.
  • CrystalBay - Monday, September 13, 2004 - link

    Nice read JC, nice replies #6 and #10...
  • karlreading - Monday, September 13, 2004 - link

    Can i just say, i wish all you moaning gits who are ripping the guys grammer and use of words apart could just pipe down. The guys have done a sterling job with this review and if i'd done some hardcore benchmarking session then id prolly be so tierd that i would be a bit fuzzy with my write up. Your just like the stupid kids in class that have to put there hands up and shout " miss, you spelt that wrong!! ". what does it achieve, not a lot, thats what!!!
    Lets try to appreciate the article for its content and commend Jason and Ross for delivering us with a good comparitive benchmark set rather than sit there nit picking about a slightly suspect use of wording!!!
  • johnsonx - Monday, September 13, 2004 - link

    Also, if in fact the board used on the Opteron was the Thunder K8W, I'd like to ask a further question:

    Did you enable cross-processor memory interleaving? (sorry, I forget what it's called in the BIOS) In this mode, memory access is interleaved across both memory banks, effectively yielding a shared 256-bit memory bus which increases performance for UMA operating systems.

    I think Windows 2003 also supports a form of NUMA, so if you *disable* the cross-processor interleaving (and get a couple of other settings right as well), then Windows will try to keep threads and data local to the processor executing them, which should increase performance in some situations. There's a bunch more terminology to this, about how the BIOS passes a configuration table to the OS which tells it which memory is local to which processor, and what other conditions have to be met in the BIOS settings (something about background ECC scrubbing if I recall correctly).

    I have to admit that I've only setup 1 Dual-Opteron Windows 2003 Server (on a Thunder K8S Pro), and I didn't completely understand all the ramifications of the above, so I just enabled the cross-processor memory interleaving (which thus disabled any NUMA support). I figured that was probably the safe bet.

    If AT could investigate this and shed some light on it, it would be most interesting.

    Dave
  • johnsonx - Monday, September 13, 2004 - link

    @Mino and Jason Clark,

    The Tyan K8W board was specified in the article; but there are two very different K8W's. The Tiger K8W is a workstation board with a single bank of memory slots; the second CPU does ALL memory access via HT. The Thunder K8W is high-end workstation & server board with memory slots for both CPUs; the only difference between it and a 'server only' board is the Thunder K8W has an AGP 3.0 tunnel and slot where a 'server only' board like the Thunder K8S Pro does not.

    I'm guessing that the Thunder K8W was used for the test, but Mino thought only of the Tiger K8W.

    Perhaps clarification should be made?
  • johnsonx - Monday, September 13, 2004 - link

    I opened the comments page just to see if anyone had already commented on this, and I see Viditor beat me to it, but still..

    "When AMD first broke wind with the K8 announcement..."

    Yikes. I'm really sure that isn't the phrasing you were looking for. No, this is some sort of Beavis & Butthead "heheheheh he said 'wind' hehehehe" thing; AFAIK "break wind" means one thing and one thing only, and it just can't be used the way you tried to.
  • daveshel - Monday, September 13, 2004 - link

    "So, we've seen AMD compete on both the desktop and server market, but does this transgress into a victory in corporate America?"

    Trans- something, I guess.

Log in

Don't have an account? Sign up now