"Order Entry" Stress Test: Measuring Enterprise Class Performance

One complaint that we've historically received regarding our Forums database test was that it isn't strenuous enough for some of the Enterprise customers to make a good decision based on the results.

In our infinite desire to please everyone, we worked very closely with a company that could provide us with a truly Enterprise Class SQL stress application. We cannot reveal the identity of the Corporation that provided us with the application because of non-disclosure agreements in place. As a result, we will not go into specifics of the application, but rather provide an overview of its database interaction so that you can grasp the profile of this application, and understand the results of the tests better (and how they relate to your database environment).

We will use an Order Entry system as an analogy for how this test interacts with the database. All interaction with the database is via stored procedures. The main stored procedures used during the test are:

sp_AddOrder - inserts an Order
sp_AddLineItem - inserts a Line Item for an Order
sp_UpdateOrderShippingStatus - updates a status to "Shipped"
sp_AssignOrderToLoadingDock - inserts a record to indicate from which Loading Dock the Order should be shipped
sp_AddLoadingDock - inserts a new record to define an available Loading Dock
sp_GetOrderAndLineItems - selects all information related to an Order and its Line Items

The above is only intended as an overview of the stored procedure functionality; obviously, the stored procedures perform other validation, and audit operations.

Each Order had a random number of Line Items, ranging from one to three. Also randomized was the Line Items chosen for an order, from a pool of approximately 1500 line items.

Each test was run for 10 minutes and was repeated three times. The average between the three tests was used. The number of Reads to Writes was maintained at 10 reads for every write. We debated for a long while about which ratio of reads to writes would best serve the benchmark, and we decided that there was no correct answer. So, we went with 10.

The application was developed using C#, and all database connectivity was accomplished using ADO.NET and 20 threads - 10 for reading and 10 for inserting.

So, to ensure that IO was not the bottleneck, each test was started with an empty database and expanded to ensure that auto-grow activity did not occur during the test. Additionally, a gigabit switch was used between the client and the server. During the execution of the tests, there were no applications running on the server or monitoring software. Task Manager, Profiler, and Performance Monitor were used when establishing the baseline for the test, but never during execution of the tests.

At the beginning of each platform, both the server and client workstation were rebooted to ensure a clean and consistent environment. The database was always copied to the 8-disk RAID 0 array with no other files present to ensure that file placement and fragmentation was consistent between runs. In between each of the three tests, the database was deleted, and the empty one was copied again to the clean array. SQL Server was not restarted.

SQL Stress Results Order Entry Results
Comments Locked

144 Comments

View All Comments

  • cHodAXUK - Thursday, April 21, 2005 - link

    Anand, Jason and Ross.. hell of a job guys, you have out done yourselves. As for the X2 4400+ preview results, holy shit is all I can say, better than I expected and those scores are WITHOUT the aid of an NCQ enabled drive. The cost is high, very high infact but the X2 just scales so much better than the equivelent Intel. All I want to see now is an X2 4400+ with the FSB overclocked to DDR500 speeds, I am really interested to see how much that extra 1gb/s+ of bandwidth helps a dual core setup. Perhaps that is something you can look into for us please Anand and Co? T.I.A. ;)
  • Darth Farter - Thursday, April 21, 2005 - link

    http://www.hardwaregeeks.com/comments.php?catid=1&...
    [quote]:
    Current Intel Price List(3)

    Price in
    1,000 unit
    Performance Processors quantities

    64-bit Intel Xeon processor MP 3.33 GHz with 8 MB L3 cache $3692
    64-bit Intel Xeon processor MP 3.00 GHz with 8 MB L3 cache $1980
    64-bit Intel Xeon processor MP 2.83 GHz with 4 MB L3 cache $1177

    Value Processors

    64-bit Intel Xeon processor MP 3.66 GHz with 1 MB L2 cache $963
    64-bit Intel Xeon processor MP 3.16 GHz with 1 MB L2 cache $722
    [/quote]

    Intel's not too shabby with pricing either... ;)

    btw Dual OPTERON vs 4way(?) XEON @ techreport
    http://techreport.com/reviews/2005q2/opteron-x75/i...
  • Groovester - Thursday, April 21, 2005 - link

    65- Recommend you reread "A Look at AMD's Dual Core Architecture" page. The fact that AMD's Athlon64 and X2 memory controllers are on-die gives it a leg up on Intel's Pentium D's. On the X2, the communication between the two cores doesn't have to traverse the external FSB.
  • bob661 - Thursday, April 21, 2005 - link

    WHOODOGGIE!!!
  • Quanticles - Thursday, April 21, 2005 - link

    68 - he did the best he could, but the point is the same... lol. we're going to see some pretty amazing preformance from the real thing
  • Son of a N00b - Thursday, April 21, 2005 - link

    hit me up with one of these, four vid cards, some headphones and a 24' screen for hours of gaming bliss....w00t!

    anyway i can actually see also a game suddenly coming out written for dual core, with the developers pulling something outta their collective a$$'s....


    I'll wait for these to get a bit more refined though and the pwnage is clear that a dual core offers total uberness...

    good article anand...almost to complete lol...i actually have to save some time in my day to read em....gj!
  • fishbits - Thursday, April 21, 2005 - link

    From the article: "Although the use of ECC memory and a workstation motherboard would inevitably mean that performance will be slower than what will be when the real Athlon 64 X2s launch, its close enough to get a good idea of the competitiveness of the Athlon 64 X2."

    Anand didn't "cripple" or "misrepresent" anything. He got as close as he could with the materials available to him, and made it clear that some liberties/extrapolation would be required.

    However, it does look promising that the X2 will perform even better than projected today. Just as Anand said up front.
  • KillerBob - Thursday, April 21, 2005 - link

    You are right Griswold, and it was in these tests the Intel won the race;)
  • Zebo - Thursday, April 21, 2005 - link

    Expect at least 15% more performance when real X2 is released.

    Anand crippled/misrepresented it by running a 175 in his tests... Which has ECC memory, 2T, and my guess is 3-3-3 (most all ECC ram is 3-3-3 since he does'nt say I must go with the odds).

    Talk about hamstringing a A64. Anands own tests show just how crippleing 2T is for A64 upwards of 10% alone less performance. I've shown 3-3-3 vs 2-2-2 to be signifigant in my mem matrix tread about 5% since A64's love low latency. ECC knocks out about 3-5% more performance due to extra wait state. Would the "real" X2 debuting at 18% faster be unfair?? I don't think so when paired with desktop memory.

    It's going to get REAL ugly on the desktop for Team Blue no matter how you slice the numbers when a real live X2 comes with un-buffered mem, LL and 1T since Intel already loses to a unadventurous server chip right now.
  • Fricardo - Thursday, April 21, 2005 - link

    64 - I'd like to know the same. I definitely won't buy a processor for more than 250, no matter what the performance is. I'm sure they'll drop eventually, but I wonder if that'll happen before 939 is completely obsolete and I have to buy an M2 mobo anyways...

    Also, something I've been wondering: if dual core does have such an impressive effect on desktop performance and future programs will be multithreaded to take advantage of dual core, how come nobody ever talks about making multi-socket desktop boards? A dual-939 setup with a couple of $120 OC'd Winnie's would be just as fast as the X-2 and a heck of a lot cheaper. Or you could slap a couple of X-2's in there when they actually come out and have sick performance.

Log in

Don't have an account? Sign up now