AMD Socket-F Opteron vs. Intel Woodcrest
by Jason Clark & Ross Whitehead on December 18, 2006 12:05 AM EST- Posted in
- IT Computing
Benchmark Methodology
For AnandTech Database Benchmarks, we have always focused on "real world" Benchmarks. To achieve this, we have used real applications with loads such that CPU utilization was 80-90%. Recently we discussed how most Enterprise Database Servers do not average 80-90% CPU utilization, but rather something closer to the 30-60% range. We thought it would make more sense to show performance where it is most likely going to be used, as well as the saturation numbers for the situations where the CPU is maxed.
We feel this is consistent with how GPUs are reviewed, and how you might test drive a car. With GPUs, the cards are tested with varying resolutions, and anti-aliasing levels. With a car, you don't just hit the highway and see what the top end is.
We settled on six load points 20%, 40%, 60%, 80%, 100%, and 120+% for testing the varying ranges of load. These load points are consistent across all platforms and are throttled from the client, independent of the platform being measured. We chose these load points as they split the load range into 6, roughly, equal parts and allow us to extrapolate data between the points. The 120+% load point was included to verify that our 100% load point really was 100%.
The 100% load point was determined by starting an execution of the client and adding threads until the CPU utilization was between 95% and 100%. The other load points were determined by altering the number of threads from the client, thus adjusting the rate of client request per second, until the appropriate ratio of Orders/Minute for Dell and Transactions/Minute for Forums was obtained relative to the 100% load point. These thread counts were recorded and maintained consistent across all platforms.
For any given load point, there is a defined number of threads. Each test is 20 minutes in duration, which includes an 8 minute warm up period followed by a 12 minute measured period. For a given load point, the client submits requests to the DB server as fast as the DB server will respond. The rate which the client is able to submit requests is measured during the final 12 minutes of the test and averaged to determine the Orders/Minute for Dell, and Transactions/Minute for Forums.
After much blood, sweat, and almost tears we were able to produce repeatable loads with an average deviation of 1.6%.
For each platform we ran the test 5 times for each load point and than averaged the 5 results, this was repeated for all loads, all tests, on all platforms... that is 300 test executions!.
Dell & Forum SQL Trace Analysis
The Dell and Forum benchmarks are quite different workloads, which you will see in the benchmark results. Dell executes approximately 10 times more queries during the test, and the durations are approximately 4 times less than that of the Forum benchmark durations. To summarize, Dell is a workload with a high transaction volume, and each query executes in a very short amount of time. The Forum workload has a medium transaction volume, and the queries execute in a reasonable amount of time but are much more read intensive (larger datasets are returned).
Dell DVD Store Information
For AnandTech Database Benchmarks, we have always focused on "real world" Benchmarks. To achieve this, we have used real applications with loads such that CPU utilization was 80-90%. Recently we discussed how most Enterprise Database Servers do not average 80-90% CPU utilization, but rather something closer to the 30-60% range. We thought it would make more sense to show performance where it is most likely going to be used, as well as the saturation numbers for the situations where the CPU is maxed.
We feel this is consistent with how GPUs are reviewed, and how you might test drive a car. With GPUs, the cards are tested with varying resolutions, and anti-aliasing levels. With a car, you don't just hit the highway and see what the top end is.
We settled on six load points 20%, 40%, 60%, 80%, 100%, and 120+% for testing the varying ranges of load. These load points are consistent across all platforms and are throttled from the client, independent of the platform being measured. We chose these load points as they split the load range into 6, roughly, equal parts and allow us to extrapolate data between the points. The 120+% load point was included to verify that our 100% load point really was 100%.
The 100% load point was determined by starting an execution of the client and adding threads until the CPU utilization was between 95% and 100%. The other load points were determined by altering the number of threads from the client, thus adjusting the rate of client request per second, until the appropriate ratio of Orders/Minute for Dell and Transactions/Minute for Forums was obtained relative to the 100% load point. These thread counts were recorded and maintained consistent across all platforms.
For any given load point, there is a defined number of threads. Each test is 20 minutes in duration, which includes an 8 minute warm up period followed by a 12 minute measured period. For a given load point, the client submits requests to the DB server as fast as the DB server will respond. The rate which the client is able to submit requests is measured during the final 12 minutes of the test and averaged to determine the Orders/Minute for Dell, and Transactions/Minute for Forums.
After much blood, sweat, and almost tears we were able to produce repeatable loads with an average deviation of 1.6%.
For each platform we ran the test 5 times for each load point and than averaged the 5 results, this was repeated for all loads, all tests, on all platforms... that is 300 test executions!.
Dell & Forum SQL Trace Analysis
The Dell and Forum benchmarks are quite different workloads, which you will see in the benchmark results. Dell executes approximately 10 times more queries during the test, and the durations are approximately 4 times less than that of the Forum benchmark durations. To summarize, Dell is a workload with a high transaction volume, and each query executes in a very short amount of time. The Forum workload has a medium transaction volume, and the queries execute in a reasonable amount of time but are much more read intensive (larger datasets are returned).
Dell DVD Store Information
Facebook
Twitter
RSS
38 Comments
View All Comments
yyrkoon - Monday, December 18, 2006 - link
After reading about FB-DIMMS, and the direct comparison to DDR2, I can not help but wonder, IF it were possible to use these Xeons with standard off the shelf DDR2, how well the Xeons would compare. Maybe this correlates with the quoted text above from your article, I do not know, as I don't know a lot about server grade equipment. Well, at least not "cutting edge" server equipment.
JarredWalton - Monday, December 18, 2006 - link
FB-DIMMs definitely use more power, and there's technically nothing to prevent someone from making a dual socket Xeon board that uses DDR2 or even DDR (or DDR3, etc.) instead of FB-DIMMs. However, for now Intel has decided that FB-DIMM DDR2 is the way they're going for workstation/server platforms, so all we can do is wonder "what if...?"Furen - Monday, December 18, 2006 - link
Huh? As I understand it there IS a difference. The FB memory controller is different and the pin configuration is significantly different, as half the pins connect to the memory controller, the other half connect to the next DIMM on the same channel. Then there's also the fact that having quad-channel DDR2 would require an insane amount of traces while quad-channel FB requires roughly the same amount of traces.yyrkoon - Tuesday, December 19, 2006 - link
Speed / power difference, silly . . .mino - Monday, December 18, 2006 - link
What has the memory controller to do with the possibility of sticking 2 Woodcrests on DDR2 chipset ???The only thing to play is the FSB compatibility.
BTW, NVIDIA is stepping in so such a platform is pretty much possible in 2007. SLI Quadro anyone...
joex444 - Monday, December 18, 2006 - link
Interesting, not what you mom said last night. Oh, who got pwned?glennpratt - Monday, December 18, 2006 - link
He didn't say there isn't a difference, did he?ltcommanderdata - Monday, December 18, 2006 - link
I guess AMD is encouraging many sites to do this heads on comparison since Tech Report has one too with similar systems. They swapped in a pair of 2.67GHz X5355 Clovertons too which was interesting. It's good that you put in a pair of E5150s though since that's probably more comparable to the 2nd from the top 2.6GHz 2218s that AMD provided.What I would love to see is data points for the top of the line 2.8GHz 2220SE, to see if the power numbers are actually that much worse, and a 2.4GHz 2216HE to see if the power numbers are that much better. I'd also be interested in seeing a pair of 2.33GHz 5148 Woodcrests reviewed since I haven't seen anyone look to see how much better the LV chips are compared to regular 65W and 80W Woodcrests.
It may not be that fair a comparison, but inclusion of the 2.67GHz X5355 Cloverton like Tech Report did would also be informative. Although, data for the 2.33Ghz E5345 Cloverton is probably more important since it still offers a 1333MHz FSB while keeping a 80W TDP of the lower parts, theoretically putting it in the sweet spot for performance/watt.
It should probably be pointed out too that Tech Report also tried a 4x2GB configuration for the FB-DIMMs and they found they saved 22W or something, compared to a 8x1GB configuration. That's something to note for system configurators and leaves more room for future expansion too.