"Order Entry" Stress Test: Measuring Enterprise Class Performance

One complaint we've historically received about our Forums database test was that it isn't strenuous enough for some of the Enterprise customers to make a good decision based on the results.

In our infinite desire to please everyone, we worked very closely with a company that could provide us with a truly Enterprise Class SQL stress application. We cannot reveal the identity of the Corporation that provided us with the application because of non-disclosure agreements in place. As a result, we will not go into specifics of the application, but rather provide an overview of its database interaction so that you can grasp the profile of this application, and understand the results of the tests better (and how they relate to your database environment).

We will use an Order Entry system as an analogy for how this test interacts with the database. All interaction with the database is via stored procedures. The main stored procedures used during the test are:
sp_AddOrder - inserts an Order
sp_AddLineItem - inserts a Line Item for an Order
sp_UpdateOrderShippingStatus - updates an status to "Shipped"
sp_AssignOrderToLoadingDock - inserts a record to indicate from which Loading Dock the Order should be shipped
sp_AddLoadingDock - inserts a new record to define an available Loading Dock
sp_GetOrderAndLineItems - selects all information related to an Order and its Line Items
The above is only intended as an overview of the stored procedure functionality; obviously, the stored procedures perform other validation, and audit operations.

Each Order had a random number of Line Items, ranging from one to three. Also randomized was the Line Items chosen for an order, from a pool of approximately 1500 line items.

Each test was run for 10 minutes and was repeated three times. The average between the three tests was used. The number of Reads to Writes was maintained at 10 reads for every write. We debated for a long while about which ratio of reads to writes that would best serve the benchmark, and we decided that there was no correct answer. So, we went with 10.

The application was developed using C#, and all database connectivity was accomplished using ADO.NET and 20 threads - 10 for reading and 10 for inserting.

So, to ensure that IO was not the bottleneck, each test was started with an empty database and expanded to ensure that autogrow activity did not occur during the test. Additionally, a gigabit switch was used between the client and the server. During the execution of the tests, there were no applications running on the server or monitoring software. Task Manager, Profiler, and Performance Monitor were used when establishing the baseline for the test, but never during execution of the tests.

At the beginning of each platform, both the server and client workstation was rebooted to ensure a clean and consistent environment. The database was always copied to the 8-disk RAID 0 array with no other files present to ensure that file placement and fragmentation was consistent between runs. In between each of the three tests, the database was deleted, and the empty one was copied again to the clean array. SQL Server was not restarted.

AnandTech Forums Database Test Results Order Entry Stress Test Results
Comments Locked

44 Comments

View All Comments

  • RyanVM - Monday, September 13, 2004 - link

    #9, why would they be in any rush whatsoever to do that when DDR2 is slower and more expensive clock for clock? Besides, with their on-die memory controller, memory performance is far from a limiting factor for K8 systems (look how little difference there is in performance between single channel and dual channel memory for K8 systems).
  • karlreading - Monday, September 13, 2004 - link

    Hmm, seems that the nocona has certainly helped xeons save face against K8. with iAMD64 and 800Mhz FSB they do look much healthier compared to prestona Xeon, raising the perfromance bar and giving the intel fans ( and Dell!! LOL ) the 64 bit option that they always deserved, rather than the expensive, unpopular niche itanic . Considering the implication of having the two CPU's fighting each other for memory bandwidth, something on netburst parts cant get enougth of, the xeons keep pace superbly and it makes one wonder what they would be capable of if each nocona had its own dedicated memory banks and controller!!! Heres one AMD fan hoping opterons @ 2.6Ghz ( opteron 260??) come out quick to keep the chipzilla sweating!!!
  • Viditor - Monday, September 13, 2004 - link

    Excellent first (32bit) look, Jason and Ross! Well written, Many thanks!
    I look forward to the next edition as well.
  • Jason Clark - Monday, September 13, 2004 - link

    Mino, actually AMD sent the tyan and 250's together :) So, they must have some faith in the board since they knew exactly the purpose of it. Aside from that the 250 managed to scale very very well, so I doubt that the board is any issue what so ever... The 64 bit article will be based around 64 bit windows.. not linux. Linux stuff is handled by kris in the linux section.

    Cheers
  • mino - Monday, September 13, 2004 - link

    by "BEST compiler around" I off course meant by performance, read ability to use the hardware given most efficiently (in case of x86-64 it doesn'nt have to be stable, stability will come by time, these chips will be there for long time).

    Just a little expl. ;0
  • mino - Monday, September 13, 2004 - link

    hmm, Jason:

    Please could You explain why with Nocona You do consider it "obviously used the E7520 Lindenhurst server chipset" and with Opteron it is allright to use purely Workstation chipset/board ???

    I know there should just minor difference betwen K8W and K8S Pro , but this "minor" rise in K8 performance might have changet this very narrow race !!

    Other than That I'm waiting for 64-bit comparison, PLEASE do include 32bit results in there then. I would preffer 32bit Nocona optimized code vs 64bit Nocona optimized vs 32bit K7 optimized(runs better than K8 optim.) vs AMD-64 optimized(compiled with BEST compiler around-even if it will be only beta compiler used) code on Opteron. This should be done on linux with systems compiled from scratch for this test. I know This is much of work, but there are people around that can make such a system up and running in 3 to 4 hours.(one is sitting near me:). So, IF You need help, call some friends of yours. Since such an comparison would be ONE and the BEST ONE around this pitty planet.

    good luck,
    mino
  • jrphoenix - Monday, September 13, 2004 - link

    #12... Very funny.... I had to go back and read it again. I guess they did fart out the specs, LMAO!!!
  • Jason Clark - Monday, September 13, 2004 - link

    A 3.2 prestonia vs. 3.2 nocona could be interesting though, I'll see what we can do there.
  • Jason Clark - Monday, September 13, 2004 - link

    We are in planning a 64bit db test, and web test. Not to worry. These types of articles that use real world tests, take time :)
  • Cusqueno - Monday, September 13, 2004 - link

    I believe the authors meant "piqued their interest" instead of "peaked their interest". I don't think this is a reference to climbing mountains.

Log in

Don't have an account? Sign up now