AnandTech uses SPEC CPU2000 for the first time

Unique to this review as well as a handful of reviews to come is the presence of an extremely useful benchmark, SPEC CPU2000.

One of the biggest problems that exists when dealing with benchmarks is the problem of platform specific optimizations that exist in these benchmarks.  For example, in our Desktop CPU Comparison that was published in September 1999 we showed exactly what SSE or 3DNow! specific optimizations can do to otherwise normal performance standings.  It is quite simple to make an Athlon seem like the slowest CPU in a test, just optimize heavily for SSE instructions.  The reverse works as well, if you eliminate any sort of SSE optimizations and simply optimize for Athlon architectural advantages you can just as easily make the Pentium III seem like the slowest CPU. 

The only way around this is to essentially make your own benchmarks, however that is not always a realistic thing to do as developing such benchmarks usually require more time than can be devoted to a single review.  Instead, we provide you all with a suite of benchmarks that present an overall performance picture, and based on that you can generally come to the conclusion that one processor is faster or slower than another. 

The Standard Performance Evaluation Corporation (SPEC) came up with another solution to this problem, make the person running the tests handle the job of compiling them as well.  By forcing the tester to compile the tests on his/her own, the tester can choose to heavily optimize for one platform or another, but even better yet, the tester can heavily optimize for more than one platform to get the best possible performance figures on any given system.  And being a non-profit organization, you don’t have to worry about one company or another contributing to paying SPEC’s bills at the end of each month.

We have already been using a SPEC supplied benchmark for quite some time now, SPECviewperf which measures graphics performance.  SPEC CPU2000 is, as you can probably guess, a benchmark that more closely focuses on CPU performance.  It does so by executing a total of 25 CPU intensive applications that can be used to stress different performance areas.  The SPEC CPU2000 benchmark is split into two separate benchmarks, SPEC CINT2000 and SPEC CFP2000, the Integer and Floating Point benchmark sets. 

The ‘C’ in the name CINT2000 and CFP2000 is supposed to stand for component-level benchmarks, indicating that SPEC CPU2000 is designed not to measure the overall performance of a system, as a real world application test would (i.e. SYSMark, Winstone, etc…) but measures the performance of one or more components in a system. 

SPEC CPU2000 is completely disk and graphics independent, rather its performance depends on your CPU, FSB, memory bus, and of course, your compiler; the latter being a performance constant as long as you use the same compiler in all of your tests on a given platform. 

For this particular review we’re going to be looking at SPEC CFP2000 performance since it seems to stress memory performance much more than the Integer benchmarks.  In order to be fair to both AMD and Intel, we used the same config files that they used to submit their benchmarks to SPEC. 

We also limited the compilers that we used to things that were currently available in final form meaning that Intel’s 5.0 Compiler did not make it into this review.  This hurts Intel’s performance a bit as the 5.0 Compiler optimizations improved performance around 10%, but we will take that into account in our analysis and in our next major use of SPEC CPU2000 we will attempt to obtain the beta 5.0 Compiler from Intel to include benchmarks with as well. 

The only other change we made was that we did not use the SmartHeap libraries that Intel and AMD both used in their SPEC tests as we did not have a copy during the time of benchmarking.  With each full run of SPEC CFP2000 taking approximately 12 – 14 hours from compile time to result production, we were limited as to the amount of time that could be devoted to the SPEC benchmarks alone.  In future tests we will use the SmartHeap libraries which seem to boost performance a few percent across the board. 

The last thing to take into account when dealing with SPEC CPU2000 scores is that there are two overall scores that are outputted for every official run: SPECfp2000 and SPECfp_base2000. 

The difference between the two is simple, base only allows a certain number of platform specific optimizations to be made and only one compiler to be used whereas the SPECfp2000 (otherwise known as ‘peak’) is a bit more lenient with the compiling stipulations and allows for more aggressive optimization, including the ability to use more than one compiler (i.e. using one compiler for some tests, and another compiler for others). 

So basically the base numbers show you how the platform performs without any aggressive optimizations, while the peak numbers show you what kind of performance you can get if a particular application developer heavily optimizes for a particular platform.  Since we’re heavily optimizing for both Intel and AMD CPUs in our tests, we should see the highest performance available today using these CPUs. 

Now that you’ve made it through all of that, let’s get to see our tests in action and see how DDR SDRAM really stacks up on the Athlon…

New Test Bed & New Benchmarks The Test
Comments Locked

0 Comments

View All Comments

Log in

Don't have an account? Sign up now