"Order Entry" Stress Test: Measuring Enterprise Class Performance

One complaint that we've historically received regarding our Forums database test was that it isn't strenuous enough for some of the Enterprise customers to make a good decision based on the results.

In our infinite desire to please everyone, we worked very closely with a company that could provide us with a truly Enterprise Class SQL stress application. We cannot reveal the identity of the Corporation that provided us with the application because of non-disclosure agreements in place. As a result, we will not go into specifics of the application, but rather provide an overview of its database interaction so that you can grasp the profile of this application, and understand the results of the tests better (and how they relate to your database environment).

We will use an Order Entry system as an analogy for how this test interacts with the database. All interaction with the database is via stored procedures. The main stored procedures used during the test are:

sp_AddOrder - inserts an Order
sp_AddLineItem - inserts a Line Item for an Order
sp_UpdateOrderShippingStatus - updates a status to "Shipped"
sp_AssignOrderToLoadingDock - inserts a record to indicate from which Loading Dock the Order should be shipped
sp_AddLoadingDock - inserts a new record to define an available Loading Dock
sp_GetOrderAndLineItems - selects all information related to an Order and its Line Items

The above is only intended as an overview of the stored procedure functionality; obviously, the stored procedures perform other validation, and audit operations.

Each Order had a random number of Line Items, ranging from one to three. Also randomized was the Line Items chosen for an order, from a pool of approximately 1500 line items.

Each test was run for 10 minutes and was repeated three times. The average between the three tests was used. The number of Reads to Writes was maintained at 10 reads for every write. We debated for a long while about which ratio of reads to writes would best serve the benchmark, and we decided that there was no correct answer. So, we went with 10.

The application was developed using C#, and all database connectivity was accomplished using ADO.NET and 20 threads - 10 for reading and 10 for inserting.

So, to ensure that IO was not the bottleneck, each test was started with an empty database and expanded to ensure that auto-grow activity did not occur during the test. Additionally, a gigabit switch was used between the client and the server. During the execution of the tests, there were no applications running on the server or monitoring software. Task Manager, Profiler, and Performance Monitor were used when establishing the baseline for the test, but never during execution of the tests.

At the beginning of each platform, both the server and client workstation were rebooted to ensure a clean and consistent environment. The database was always copied to the 8-disk RAID 0 array with no other files present to ensure that file placement and fragmentation was consistent between runs. In between each of the three tests, the database was deleted, and the empty one was copied again to the clean array. SQL Server was not restarted.

SQL Stress Results Order Entry Results
Comments Locked

144 Comments

View All Comments

  • KillerBob - Friday, April 22, 2005 - link

    Griswold,

    MT Test 1: PEE 1 - X2 0 Very likely scenario
    MT Test 2: PEE 2 - X2 0 Likely scenario
    MT Test 3: PEE 2 - X2 1 So-so scenario
    MT Test 4: PEE 3 - X2 1 Likely scenario
    MT Test 5: PEE 3 - X2 2 Likely scenario
    MT Test 6: PEE 3 - X2 3 Unlikely scenario

    I play a lot of games, but I never have things in the background, as a matter of fact I don't want to have anyting in the background, except for perhaps a big NewsPro download.
  • MrEMan - Friday, April 22, 2005 - link

    102,

    Artificial stupidity run rampant?

    or

    Natural deselection (survival of the twitest)?
  • Quanticles - Friday, April 22, 2005 - link

    I vote that 90% of the people on here have no idea what they're talking about... lol
  • erwos - Friday, April 22, 2005 - link

    "It's odd that some picture game developers immediately supporting the PhysX chip as soon as it's available, but think they'll drag their feet to take advantage of another whole CPU core at their disposal."

    It's basically about the implementation differences of the two. You can be relatively certain that PhysX is going to be shipping their chips/cards with libraries that allow game devs to just speed up certain processing with special function calls (ie, calculate_particle_spread()). Multi-threading requires that you design your application from the very start to take advantage of it (mostly - I would wager splitting off the background music to its own thread is reasonably straightforward).

    Game logic doesn't always lend itself to multi-threading, either. If I shoot my gun, I want to hear the sound next. I don't want it to be thrown at the sound thread, where it may or may not execute next. Threading introduces latency, in other words, unless you so tightly bind your threads together that you may as well not use multi-threading.

    -Erwos
  • Griswold - Friday, April 22, 2005 - link

    KillerBob, so that makes you a brilliant illiterate, since it's not what the benchmarks say. :)
  • cHodAXUK - Friday, April 22, 2005 - link

    #83 Get a clue, a single core 3500+ is faster than the quivelant Opteron at the same speed. Why? Unregistered memory and tigher memory timinings. ECC memory comes with a 2-4% performance penalty but the big difference comes with the command speed, 2T for the Opteron and 1T 3500+, the AMD64 thrives on lower lower latancies that can make as big as an 10% performance difference and that is BEFORE we start to even think about raising the FSB speed which makes a significant difference to overall system perfomance. 15% is in no way unrealistic with a mild overclock and lower latancies, if you don't believe me then email Anand and ask him.
  • Zebo - Friday, April 22, 2005 - link

    Jep4444 (#89) What do you mean X2's "arent nearly as good as the dual core Opterons"??

    Comming from XS I suspect don't OC very well?

    But they are the same cores as the Opterons are. and with ram should run signifigantly faster.

    Or do you mean buggy? That's easily attibuted to BIOS, IE none released yet so no working BIOS.

    How about a link please.
  • Umbra55 - Friday, April 22, 2005 - link

    The benchmark overviews show "dual opteron 252 (2.6 GHz)" all over the review. I suppose this is single 252 instead of dual?

    Please correct accordingly
  • emboss - Friday, April 22, 2005 - link

    #40 (Doormat):
    You're forgetting that the size of a dual-core is (roughly) double that of a single-core. So, assuming 1000 cores/wafer, 70% defect rate per core, then a single-core wafer (with an ASP of $500) will net AMD 700*500 = $350K.

    The same wafer with dual-cores will produce (approximately) 1000/2 * (0.7)^2 = 245 CPUs. So, to get the same amount of cash per wafer, AMD needs an ASP of $1429, or the second core costing 85% more than the first core.

    Of course, it's not quite this simple ("bad" chips running OK at lower speeds, etc) but it's not entirely unreasonable to see dual-cores with prices ~3 times that of a single core at the same speed grade. Intel is almost dumping (in the economic sense of the word) dual-core chips.
  • saratoga - Friday, April 22, 2005 - link

    "saratoga, waah? There are similarities between C# and C++. While agree it's java'ish as well, it definitely has similarties to c++. One could say c# shaes similarities with c/c/c++.

    read away:

    http://www.mastercsharp.com/article.aspx?ArticleID...

    http://www.csharphelp.com/archives/archive138.html

    "

    I'm guessing you're not a c++ programmer ;)

    Anyway, yes they both use c syntax, however thats pretty much irrelevent given that Java also uses c syntax (as does Managed c++ which incidently IS the .net language directly based on c++) and I've never heard anyone call it related to c++. Beyond (some) syntax heritage and the fact that they're both OO langauges, they're very different beasts.

    ""C# is directly related to C and C++. This is not just an idea, this is real. As you recall C is a root for C++ and C++ is a superset of C. C and C++ shares several syntax, library and functionality." Quoted from above.

    L8r."

    Err yeah c++ is mostly a superset of c++. Thats neither here nor there. Just try and use the c/c++ preprocessor in c# and you'll see very quickly what the difference is. Or try using c++ multiple inherritance. You'll find that just because you took java and added operator overloading and made binding static by default, its not c++.

Log in

Don't have an account? Sign up now