"Order Entry" Stress Test: Measuring Enterprise Class Performance

One complaint we've historically received about our Forums database test was that it isn't strenuous enough for some of the Enterprise customers to make a good decision based on the results.

In our infinite desire to please everyone, we worked very closely with a company that could provide us with a truly Enterprise Class SQL stress application. We cannot reveal the identity of the Corporation that provided us with the application because of non-disclosure agreements in place. As a result, we will not go into specifics of the application, but rather provide an overview of its database interaction so that you can grasp the profile of this application, and understand the results of the tests better (and how they relate to your database environment).

We will use an Order Entry system as an analogy for how this test interacts with the database. All interaction with the database is via stored procedures. The main stored procedures used during the test are:
sp_AddOrder - inserts an Order
sp_AddLineItem - inserts a Line Item for an Order
sp_UpdateOrderShippingStatus - updates an status to "Shipped"
sp_AssignOrderToLoadingDock - inserts a record to indicate from which Loading Dock the Order should be shipped
sp_AddLoadingDock - inserts a new record to define an available Loading Dock
sp_GetOrderAndLineItems - selects all information related to an Order and its Line Items
The above is only intended as an overview of the stored procedure functionality; obviously, the stored procedures perform other validation, and audit operations.

Each Order had a random number of Line Items, ranging from one to three. Also randomized was the Line Items chosen for an order, from a pool of approximately 1500 line items.

Each test was run for 10 minutes and was repeated three times. The average between the three tests was used. The number of Reads to Writes was maintained at 10 reads for every write. We debated for a long while about which ratio of reads to writes that would best serve the benchmark, and we decided that there was no correct answer. So, we went with 10.

The application was developed using C#, and all database connectivity was accomplished using ADO.NET and 20 threads - 10 for reading and 10 for inserting.

So, to ensure that IO was not the bottleneck, each test was started with an empty database and expanded to ensure that autogrow activity did not occur during the test. Additionally, a gigabit switch was used between the client and the server. During the execution of the tests, there were no applications running on the server or monitoring software. Task Manager, Profiler, and Performance Monitor were used when establishing the baseline for the test, but never during execution of the tests.

At the beginning of each platform, both the server and client workstation was rebooted to ensure a clean and consistent environment. The database was always copied to the 8-disk RAID 0 array with no other files present to ensure that file placement and fragmentation was consistent between runs. In between each of the three tests, the database was deleted, and the empty one was copied again to the clean array. SQL Server was not restarted.

AnandTech Forums Database Test Results Order Entry Stress Test Results
Comments Locked

44 Comments

View All Comments

  • TrogdorJW - Wednesday, September 15, 2004 - link

    41 - Oh, I know that. I've used Netware networks and I've used Windows Server networks. (I've administered both.) The Netware servers were less powerful machines, and yet they still tended to be faster, and NDS was so much less of a pain in the ass.... I used to live next door to Novell HQ, and I have lots of friends that work there, so it was always rather disappointing to see all the big corporations being sucked (*suck*ed) in by the MS marketing. These days, I work for a large corporation that went with Windows server for everything. It works, but I'm amazed at how slow a lot of operations are. Bah.
  • Cygni - Wednesday, September 15, 2004 - link

    Heres the link to the article:
    http://www.anandtech.com/linux/showdoc.aspx?i=2163
  • Cygni - Wednesday, September 15, 2004 - link

    Mikeymasta's post is pure comedy. He’s apparently unaware that effectively 100% of Win DB work is done in a 32bit environment, unaware that Nocona is targeted directly at this Wintel server market, unaware that much DB work is done on NT/2k/2k3 based systems, and even unaware that anandtech HAS ALREADY DONE A Nocona Vs. Hammer for DB in 64bit Linux article. Heck, the kid doesnt even know that misleading is one word.

    I dont care how many trillions of linux based computers he claims to be the admin of, or how many billions of jillions of hits he gets per nanosecond, hes still an idiot to the nth power. Hes the "gayest", if you will.
  • johnsonx - Wednesday, September 15, 2004 - link

    TrogdorJW,

    NDS runs great on Windows servers; indeed most of Novell's product line runs on both NetWare and Windows Server, and alot of it runs on Linux too (much more now of course that Novell bought SUSE, plus NetWare 7 will be a customized Linux kernel so it'll ALL eventually be Linux). Even if you've GOT to run Windows, a network of Windows Servers running NDS, GroupWise, & ZENWorks will be a much happier network (with happier users and admins) than one running AD, Exchange and SMS.
  • TrogdorJW - Tuesday, September 14, 2004 - link

    #39 - Unless you are doing something much more complex than running a file server and domain controller (and firewall), I still say NetWare is tough to beat. I would also take Novell's Directory Services over Microsoft's Active Directory in a heartbeat. Ah, well - too late for that. Yet another instance of the superior Microsoft Marketing Machince winning out. :p
  • johnsonx - Tuesday, September 14, 2004 - link

    @Mikeymasta,

    Just because you are all geeked up about BSD doesn't mean the real world is. In the real world, a large majority of servers run Windows, like or not. Myself, I'm still geeked up about Novell NetWare, but I don't delude myself into thinking that everyone else is. Before you chuckle keep in mind that there are most likely more NetWare servers running out there in the real world of corporate IT than Linux servers.



  • Jason Clark - Tuesday, September 14, 2004 - link

    Mikey, the tests are absolutely not misleading, contrary to your belief ALOT of the IT organizations use windows and are not interested in BSD, or BSD based benchmarks. We're talking about an opterating system that has at least 80% of the market here, people are interested in how it currently peforms with the latest hardware.

    Since there is no RELEASED version of 64 bit windows yet, we have to wait for permission to run some benchmarks. We are working on it. As for linux benchmarks, visit the linux section.

    Relax.
  • mikeymasta - Tuesday, September 14, 2004 - link

    This has to be one of the gayest mis leading benchmarks I have ever seen, comparing server grade 64 bit CPUs on a 32bit operating system.
    As far as I am concerned Anandtech is still for average tech folks and you are seriously miss leading people and giving them opinions about these 64bit CPUs in a flawed test enviroment.
    Get FreeBSD or Linux 64 bit based OS's and do something real. convert your db system over to MySQL if you have to.

    Maybe I am being a bit harsh but you can't just do these kind of misleading kind of tests.
    As an admin/developer running many *nix based machines that get millions of hits a day this is just annoying article to read
    Your not helping anyone!
  • gchcker - Tuesday, September 14, 2004 - link

    just wondering
    do server boards use the 1000 ht bus that is available to desktops for amd?
    if they do are they being used on these benchmarks?
  • Pumpkinierre - Monday, September 13, 2004 - link

    For once I agree with Viditor- good article Jason and Ross. And I agree with karlreading on the phrasing and spelling. Only the petty get caught up with the detail. You guys are flat out with the new technology and we appreciate your sacrifice. Put the "broke wind" metaphor back in. Either in the vulgar sense or as I think you intended in the esoteric sense, it is good.

Log in

Don't have an account? Sign up now