"Order Entry" Stress Test: Measuring Enterprise Class Performance

One complaint that we've historically received regarding our Forums database test was that it isn't strenuous enough for some of the Enterprise customers to make a good decision based on the results.

In our infinite desire to please everyone, we worked very closely with a company that could provide us with a truly Enterprise Class SQL stress application. We cannot reveal the identity of the Corporation that provided us with the application because of non-disclosure agreements in place. As a result, we will not go into specifics of the application, but rather provide an overview of its database interaction so that you can grasp the profile of this application, and understand the results of the tests better (and how they relate to your database environment).

We will use an Order Entry system as an analogy for how this test interacts with the database. All interaction with the database is via stored procedures. The main stored procedures used during the test are:

sp_AddOrder - inserts an Order
sp_AddLineItem - inserts a Line Item for an Order
sp_UpdateOrderShippingStatus - updates a status to "Shipped"
sp_AssignOrderToLoadingDock - inserts a record to indicate from which Loading Dock the Order should be shipped
sp_AddLoadingDock - inserts a new record to define an available Loading Dock
sp_GetOrderAndLineItems - selects all information related to an Order and its Line Items

The above is only intended as an overview of the stored procedure functionality; obviously, the stored procedures perform other validation, and audit operations.

Each Order had a random number of Line Items, ranging from one to three. Also randomized was the Line Items chosen for an order, from a pool of approximately 1500 line items.

Each test was run for 10 minutes and was repeated three times. The average between the three tests was used. The number of Reads to Writes was maintained at 10 reads for every write. We debated for a long while about which ratio of reads to writes would best serve the benchmark, and we decided that there was no correct answer. So, we went with 10.

The application was developed using C#, and all database connectivity was accomplished using ADO.NET and 20 threads - 10 for reading and 10 for inserting.

So, to ensure that IO was not the bottleneck, each test was started with an empty database and expanded to ensure that auto-grow activity did not occur during the test. Additionally, a gigabit switch was used between the client and the server. During the execution of the tests, there were no applications running on the server or monitoring software. Task Manager, Profiler, and Performance Monitor were used when establishing the baseline for the test, but never during execution of the tests.

At the beginning of each platform, both the server and client workstation were rebooted to ensure a clean and consistent environment. The database was always copied to the 8-disk RAID 0 array with no other files present to ensure that file placement and fragmentation was consistent between runs. In between each of the three tests, the database was deleted, and the empty one was copied again to the clean array. SQL Server was not restarted.

SQL Stress Results Order Entry Results
Comments Locked

144 Comments

View All Comments

  • Phlargo - Thursday, April 21, 2005 - link

    Sounds like computing could change a lot in the next 6 months.

    I can't wait to get my hands on a dual core chip - that type of multitasking gaming performance is what I've been waiting for... 92% of unfettered performance during Doom3? That's unbelievable.
  • manno - Thursday, April 21, 2005 - link

    Another thing to consider is that the 939, and 940 Dual Cores work with old mobos. so that saves upgraders ~$100 for the mobo (I have no clue what PD mobo's are going to cost) plus the cost of DDR2 ~$100 for 512MB, not true if you're going with the PD. So if you own a 939 a 2.8 PD will realy cost you ~$441, 3.0 -$516, 3.2 ~$730.
    Throw that out the window if you're starting from scratch however.
  • manno - Thursday, April 21, 2005 - link

    Great review very detailed, I have one caveat, and I should preface it with the fact that I own 4 systems, and all of them have Athlon 64's or XP's.

    That being said your "Multitasking Scenario 2: File Compression" Seems to be misleading, and identifies the A64 as the processor of choice in the 2nd portion of the analysis.

    Maybe I'm wrong but from what I understand with how the test was performed you archived the file, got the amount of time it took to archive the file "x" for instance, and then figured out how many emails got imported in time "x" now because some processors took longer to create the archive than others they had more time to import emails. so in order to make this data more reflective of the performance of each processor you need to divide the number of emails imported by "x", and get e/s. The modified graph should be

    PD 3.2 =19124/5.08 or 3764.57 e/s
    A64 X2 =21687/6.25 or 3469.92 e/s
    PEE 3.2 =16875/6.65 or 2537.59 e/s
    AFX 55 = 3800/5.88 or 646.26 e/s

    These corrected numbers show that at least in this test the PD 3.2 is the winner by ~8%

    -manno
  • Calin - Thursday, April 21, 2005 - link

    At that prices, most of the users really won't need their current mainboards for new dual core processors. I would prefer to have lower priced single core than that prices on dual core.
  • blckgrffn - Thursday, April 21, 2005 - link

    We are talking about by this fall, folks. And a 90nm Dual core still uses LESS die than 2 2800+ put together because it is on a manufacturing smaller process, especially if they stick to a 512k cache which is obviously the sweet spot. By this fall I expect that we will see sub $200 dual cores from Intel to squeeze AMD on the desktop market. What is a real shame is that AMD doesn't have suffcient fabs to handle potential demand.

    For those of you who got all hot in your pants and said I can't buy a sub $200 dual core Intel right now, it is just as true that I can't buy one it all! ;P Chill!
  • PeteRoy - Thursday, April 21, 2005 - link

    Impressive.
  • josedawg - Thursday, April 21, 2005 - link

    Am I mistaken in thinking that AMD was throwing out their desktop dualcore line (A64 X2) in 2006? What prompted their earlier release of them? Was it the demand for desktop dualcore, pressure from Intel stealing desktop dualcore market, or excellent manufacturing at the AMD plants?
  • KeithDust2000 - Thursday, April 21, 2005 - link

    blckgrffn, "so if they can't bring anything out under $200 I will probably have to go with Intel. Boo for that ;) "

    INTEL doesn´t have one under $200 either.
  • Jeff7181 - Thursday, April 21, 2005 - link

    #14... all socket 939 motherboards will support the X2 with a BIOS upgdate.
  • Zebo - Thursday, April 21, 2005 - link

    #9 They'd have to release a 1.6 @ $240 to accurately compete with Intels slowness in Pentium D's starting at $241. Sorry not gonna happen. AMD not for budget shoppers anymore but those interested in performance. Want the best? Pay the price. Or substandard CPU at a discount?

    I can see a 1.8 for about $250 though however no way you're going to get into DC for less than $200.

Log in

Don't have an account? Sign up now