Synthetic Benchmarks

Synthetic benchmarks can still give us a good idea of what our processors should be doing. However, since they are theoretical and not good real world demonstrations of the technology, we generally rely on them only to prove or disprove if our testbeds are operating correctly. Below is the Scalar Product Opstone 04q2 as described by the author:
"The 'SP' benchmark calculates the scalar product (dot product) of 2 vectors ranging in size from 16 elements to 1048576 elements for both single and double-precision floats. Although the Gflops/sec. for every vector length is recorded (in the resulting output log file), the average of all these values is reported. This benchmark is indicative of the performance of many raw floating-point data processing apps (movie format conversion, MP3 extraction, etc.)"
We used the Athlon 64 binaries under SuSE x86_64, and the Pentium4 binaries under SuSE x86.



Hold your mouse over for the 64-bit graph.


Below is the SSP benchmark, as explained by the author:
"The 'ssp' benchmark also calculates the scalar product of 2 vectors, except that these vectors are sparsely populated (only the non-zero value elements are stored) ranging from a 'loading factor' (non-zero/zero elements) of 0.000001 to 0.01 for both single and double-precision floats. Since the data is not contiguous in memory, the performance is much lower than regular 'sp' and is measured in Mflops/sec. There is not much difference in performance between different loading factors as this benchmark really challenges the ability of the processor to perform short bursts of calculations coupled with lots of conditional testing. It is this reason that the P4 with its longer pipeline does not generally perform as well as the Athlon64. This benchmark is indicative of the performance of many 3D games as the processing is similar (short bursts of calculations with numerous conditional testing)"



Hold your mouse over for the 64-bit graph.


The Intel processors score very high marks on our Opstone benchmarks. Unfortunately, we find that this is not entirely indicative of good performance, and in fact, the Opstone benchmark does not scale well with the rest of our test suite.

Compiling Benchmarks Content Creation
Comments Locked

33 Comments

View All Comments

  • Aquila76 - Monday, September 20, 2004 - link

    It's happened again: the cheapest AMD64 939 processor is the better choice of the processors reviewed here for price/performance. Granted the Linux market isn't a huge chunk of the market for either company, but among those who do use Linux there is no reason to use anything but AMD.
  • fitten - Monday, September 20, 2004 - link

    It would be interesting to see these tests run on several S754 boxes as well. I know this one was entitled "cutting edge performance" but with the cost difference between the S939 solutions and the S754 solutions, many will opt (as I have) to go with the S754 parts. I run SuSE 9.1 Professional AMD64 on an Athlon 64 3000+ and have been pleased with it.
  • Aquila76 - Monday, September 20, 2004 - link

    This may be a little off-topic, but shows yet another reason California hates MS. This is what happens when you move to Windows...

    http://software.silicon.com/applications/0,3902465...

    Love the caption: Someone forgot to reboot
  • fitten - Monday, September 20, 2004 - link

    "Our processor test bed is completely caseless, and if we have issuse with our 3.6GHz processor out of a normal case, we can't imagine what issues might exist in a full enclosure. "

    Todays processors ("today" meaning back to the days of the original Pentium) typically run hotter with the case open than with it closed. With the case closed, cooler air is forced through the case and across the right places to help lower component temperatures, assuming that the fans are placed appropriately, are venting in the right direction, and cables/components are arranged to allow the air to flow properly.

    Newer form factors, such as Intel's BTX, are designed in a large part to cope with cooling the thing down.

    You should be running your test bed in a case with appropriate cooling.
  • Araemo - Monday, September 20, 2004 - link

    A note about the compilation benchmark:

    Make and gcc themselves are not multi-threaded, but make does understand a -j option to specify the # of concurrent make jobs to run. The general rule-of-thumb I have heard, is to specify a -j equal to n+1, where n is the number of processors in the system(One compile job per processor, and one control job.) So, to test if hyperthreading lowers the compile time, specifiy `time make -j 3`.

    I would personally like to see this, in addition to the single-threaded results. I read some quick-and-dirty benchmark results suggesting hyperthreading does help the compile time, but those results were published by curious people when hyperthreading was new.. I haven't seen any results on recent processors.
  • garfield - Monday, September 20, 2004 - link

    Isn't it normal procedure for a P4 that's getting too hot that it throttles the clock speed down? Maybe that would explain the extremely bad results, though it seems a bit unrealistic that the results has been reproducible, unless each iteration of the benchmark has been run for a long time.
  • ceefka - Monday, September 20, 2004 - link

    First of all : good article. It really shows the early benefits of well written 64-bit software. The 3500+ is definately on my wishlist ;-)

    "This does not bode well for the processor. Our processor test bed is completely caseless, and if we have issues with our 3.6GHz processor out of a normal case, we can't imagine what issues might exist in a full enclosure."

    Rather confusing this bit, Kristopher. Anyway, I read that a good case will offer convection which apparently a caseless testbed does not. How were the tempratures of the other CPU's? Were they also a tad above average or typical peak?
  • balzi - Monday, September 20, 2004 - link

    Some thoughts --

    can we please have Graphs where the order of the legend is the same as the order of the bars in the bar-graph. Surely that's possible.

    also, the strange "Thermal issue" error. It seems that you thought it was weird but immediately assumed that it was correct; that the Intel CPU(s) were getting too hot.

    Did you verify this somehow?? It seems strange to call it "An unusual problem" and then trust that it's correct without question or explanation.

    Thanks
  • Shinei - Monday, September 20, 2004 - link

    "This does not bode well for the processor. Our processor test bed is completely caseless, and if we have issuse with our 3.6GHz processor out of a normal case, we can't imagine what issues might exist in a full enclosure."
    That quote made me laugh, and I'm not entirely sure why. :D

    Anyway, I see that going to 64-bit is definitely worth the price of admission, considering the huge gains the processors get in the jump. One thing I had a question on, though: Why does the result from the SSL benchmark halve between 32-bit and 64-bit? Is it that the keys are longer in 64-bit?
  • gherald - Monday, September 20, 2004 - link

    I too would prefer longer images that include both 32 and 64-bit results. Mouseover comparisons are cumbersome.

    Is sample.wav 800mb or 700mb? I'm guessing the 7 was probably just a typo.

    Nice analysis of DDR1 vs 2.

    My only gripe is I wish a full complement of "lower end" processors were included in all these benchmarks (754s, slower prescotts, and heck even northwoods)... but I guess that'd be too much work.

Log in

Don't have an account? Sign up now