"Order Entry" Stress Test: Measuring Enterprise Class Performance

One complaint that we've historically received regarding our Forums database test was that it isn't strenuous enough for some of the Enterprise customers to make a good decision based on the results.

In our infinite desire to please everyone, we worked very closely with a company that could provide us with a truly Enterprise Class SQL stress application. We cannot reveal the identity of the Corporation that provided us with the application because of non-disclosure agreements in place. As a result, we will not go into specifics of the application, but rather provide an overview of its database interaction so that you can grasp the profile of this application, and understand the results of the tests better (and how they relate to your database environment).

We will use an Order Entry system as an analogy for how this test interacts with the database. All interaction with the database is via stored procedures. The main stored procedures used during the test are:

sp_AddOrder - inserts an Order
sp_AddLineItem - inserts a Line Item for an Order
sp_UpdateOrderShippingStatus - updates a status to "Shipped"
sp_AssignOrderToLoadingDock - inserts a record to indicate from which Loading Dock the Order should be shipped
sp_AddLoadingDock - inserts a new record to define an available Loading Dock
sp_GetOrderAndLineItems - selects all information related to an Order and its Line Items

The above is only intended as an overview of the stored procedure functionality; obviously, the stored procedures perform other validation, and audit operations.

Each Order had a random number of Line Items, ranging from one to three. Also randomized was the Line Items chosen for an order, from a pool of approximately 1500 line items.

Each test was run for 10 minutes and was repeated three times. The average between the three tests was used. The number of Reads to Writes was maintained at 10 reads for every write. We debated for a long while about which ratio of reads to writes would best serve the benchmark, and we decided that there was no correct answer. So, we went with 10.

The application was developed using C#, and all database connectivity was accomplished using ADO.NET and 20 threads - 10 for reading and 10 for inserting.

So, to ensure that IO was not the bottleneck, each test was started with an empty database and expanded to ensure that auto-grow activity did not occur during the test. Additionally, a gigabit switch was used between the client and the server. During the execution of the tests, there were no applications running on the server or monitoring software. Task Manager, Profiler, and Performance Monitor were used when establishing the baseline for the test, but never during execution of the tests.

At the beginning of each platform, both the server and client workstation were rebooted to ensure a clean and consistent environment. The database was always copied to the 8-disk RAID 0 array with no other files present to ensure that file placement and fragmentation was consistent between runs. In between each of the three tests, the database was deleted, and the empty one was copied again to the clean array. SQL Server was not restarted.

SQL Stress Results Order Entry Results
Comments Locked

144 Comments

View All Comments

  • MDme - Tuesday, April 26, 2005 - link

    #133

    i think what #130 was saying was that: from top to bottom, AMD's offerings are really good...if you want the best "bang for the buck" the 3400+ or whatever, or a 3000+ winnie OC'd will provide you with the best performance per dollar you spend...EVEN against the X2's.

    On the other hand if cost is not an issue, an X2 4400+ provides extremely good performance for people willing to pay the $500 premium.

    Zebo's point is in direct response to your point, which is AMD "STILL" has the best bang for the buck, not intel.

    or maybe YOU missed the logic? LOL
  • MPE - Tuesday, April 26, 2005 - link

    "Intel is just lucky a 3400+ new castle wasn't in that test suite. It's would win the majority of tests over an 830!! and it's still cheaper. Or did you miss this chart? LOL"

    Why not just admit it. AMD's DC is about 10-20% faster while costing 80-100% more.

    Even if the 3400+ is added, that comparison is moot since if you compare the score of that to the price of AMD's own DC - the price performance ratio is stagerrring? Or did you miss that logic?

    Anyways did you miss the part that even AMD DC was being beaten by their own single core.

    Next.
  • nserra - Tuesday, April 26, 2005 - link

    "The Athlon 64 4000+ was the last single core member of the Athlon 64 line.
    The Athlon 64 FX will continue as a single core CPU line, with the FX-57 (2.8GHz) due out later this year."

    Where did you get this info anand, i am not sure if an Athlon64 X2 4400+ could not coexist with a Athlon64 4400+. If this is the last 4000+ than i must say gee thats too bad....
  • Zebo - Tuesday, April 26, 2005 - link

    #125

    Techreports review is better for you. 64-bit OS, 64-bit apps when possible, no mystery unreproducable benchmarks like Anand's database stuff.
  • Zebo - Tuesday, April 26, 2005 - link

    MPE BS, Intel is just lucky a 3400+ new castle wasn't in that test suite. It's would win the majority of tests over an 830!! and it's still cheaper. Or did you miss this chart? LOL
    http://images.anandtech.com/reviews/cpu/amd/athlon...

    Intels DC chips can hardy compete with AMDs single core offerings. Side by side both DC it's a joke.

    So ya, AMD still has the "best bang for the buck" top end to bottom end. And they a far on top of the mountain.
  • MPE - Monday, April 25, 2005 - link

    Isn't the shoe on the other foot?

    For several years now, so many touted AMD's cheaper price and competative pricing.

    Now with Pentium4 D, especially with the 3GHz model, you get half the price of the cheapest X2 while probably at best 20% lower performance?

    What happened here?

    Now P4D 3GHz model is the best bang for the buck and not the AMD offering. This is a complete reversal of what a lot of AMD supporters have been touting?
  • ceefka - Monday, April 25, 2005 - link

    #125 Yeah, good point.

    Compare:
    A. singletreaded 32-bit app on a singlecore
    B. multi-threaded 64-bit app on a dualcore
    Considering that multithreaded apps already see such large gains on dualcores, going 64-bit too could well mean a more than 100% improvement from A to B.

    But of course, NO ONE needs dual core, 64-bit and +4GB memory in the next 5-10 years :P

    The ball now lies with MS and (Linux) app developpers to write more stuff in multithreaded 64-bit code. From what I hear and read it is not so much the 64-bit part as it is the threading that is a real challenge, even for veterans.
  • Ross Whitehead - Sunday, April 24, 2005 - link

    Visual, On P.12 I was referring to the closest Xeon competitor to the 252s which is the Quad Xeon 3.6 GHz 667 MHz FSB.

    Does that make any more sense?
  • Ross Whitehead - Sunday, April 24, 2005 - link

    jvarszegi, the actual stored procs are not prefixed with "sp_", we just used that as part of the "analogy" to the real system.

    One could also argue that we did not prefix the analogy example with the object owner either which also incurs a cache miss.

    Honestly, I have never quantified the expense of the sp_ prefix or the object owner.
  • Binji7 - Sunday, April 24, 2005 - link

    Where are the dual-core Windows x64 and Linux x64 benchmarks?? That's what I really want to see.

Log in

Don't have an account? Sign up now