Synthetic Benchmarks

Our Nocona server was setup in a remote location with little access, so we had limited time to run as many real world benchmarks as we are typically accustomed to. Fortunately, there are multitudes of synthetic benchmarks that we can use to deduce information quickly and constructively.

Sieve of Atkin (primegen)

Primegen is an older, but still useful library for generating prime numbers in order using the Sieve of Atkin. We compiled the Bernstein implementation by simply running "make". We ran the program as so:

# time ./primes 1 100000000000 > /dev/null

primegen 0.97

We found the benchmark to be extremely reliable and we replicated our figures continually with less than 1% difference.

Super Pi

We ran the Linux compilation of Super Pi 2.0, which is a closed source application. We are not aware of which optimizations are compiled with the program and we are prohibited from redistributing the binaries. Please download the latest binaries from ftp://pi.super-computing.org/Linux. We ran the command:

# ./super_pi 20

Below is the program's output of calculation time in number of seconds.

Super Pi 2.0

After re-running the program several times, our benchmarks never deviated outside of 1%. In a mathematical operation-only situation, the Intel processor has outpaced the AMD offering twice now.

Content Creation Synthetic Benchmarks (continued)
Comments Locked

275 Comments

View All Comments

  • Viditor - Tuesday, August 10, 2004 - link

    johnsonx - "What other conclusion could be drawn from the benchmark results?"

    That's just it...the benchmark results have been shown to be rather intensely flawed (for those of you with conspiracy theories on the brain, don't be stupid...it wasn't deliberate!).
    While Kris HAS been correcting them, he hasn't re-written the conclusion to reflect those changes...
  • WiZzACkR - Tuesday, August 10, 2004 - link

    hey steve - thanks for the response. next time i'll calm down first and will try to not look like an idiot. promise ;)

    concerning the processor choice i still think it's a bit odd though: if intels desktop 3.60F equals its server offering in form of the here tested XEON that doesn't mean AMDs desktop offering in form of the 3500+ is as well it's top-of-the-line server chip. if you compare the 850$ version you should do so against AMDs 850$ counterpart - if you compare the desktop offerings it may well be that the 3.60F will do better when comparing it against the 3500+ in the desktop arena. do i make any sense here? man, too tired...

    anyways - thanks for the response, and again sorry for acting like a fuckwitt.
  • ss284 - Tuesday, August 10, 2004 - link

    WiZzACkR, I dont disagree with the statements about how the selected benchmarks wasnt that great a choice(although he was limited considering the 64-bit nature of the tests
    ), in addition to poor execution.

    I just disagree with all the people complaining about the processor choice and using idiotic analogies about how this is comparing a duron and a xeon. ====The core in the xeon 3.6 and the p4 3.6e em64t are the same.==== I want that drilled into everyone's mind who is complaining about unfair comparisons. Given the setup used in the article, its actually likely that the p4 3.6e em64t on a desktop board with standard non registered memory at cl2 will perform better than the tumwater setup. As for pricing/performance, pricing ends up being the more important factor. In the case of the 3500+, its currently running ~350 dollars, while the p4 3.6 em64t is slated to be released at 416 dollars in a week or so. I dont think pricewise the comparison is much off. A 3600+ from amd will no doubt cost at least 50 dollars more than the 3500+, putting them at a price parallel.

    And regarding the hardforums, yes there are some decently constructive posts. However, the general populace there seems a lot less curteous and tend to get into craptastic posts from immature members. Forums like Anandtech, Ace's hardware, and arstechnica, have much more contructive posting by the general populace. Semi decent punctuation also seems to be a skill that most Anandtech forum goers possess that a great deal of hardocpers do not. The cluelessness of many of the posts are pretty apparent by browsing almost any of their forums (I've actually been a member since 2000, so I would know).


    I overreacted as well, and I'm sorry for that. Its just I believed some of the quotes you used from the hardforums werent valid, and that readily available correct information should have been used.

    -Steve
  • WiZzACkR - Tuesday, August 10, 2004 - link

    replying to 178 - Posted on Aug 10, 2004 at 1:18 PM by johnsonx

    ok, i put that weired. what i meant was: if you run a benchmark suit of mostly server apps and one server CPU you should compare it to the other company's server chip as well, not a desktop one. comparing a DESKTOP intel product (3.60F) vs a DESKTOP 3500+ is fine - but then do so with a desktop benchmarking suit, typical desktop applications and as few as possible synthetic benchmarks.
    hmmm, still weired...
  • dougSF30 - Tuesday, August 10, 2004 - link

    John the Ripper results are completely invalid.

    The source code contains Hand-tuned ASSEMBLY routines, some of which only get called if the CPU has the word "Intel" in the name!!!!!

    (And others are optimized for the K6. LOL.)

    Totally bogus.

    See x86.S detect.c and best.sh in the source package.

    These results are garbage-- this "test" should be removed from the article.

    The architecture-detection code is also probably broken, even in the generic case.

  • chaosengine - Tuesday, August 10, 2004 - link

    Atleast someone is taking a note on what we are irate abt

    http://www.theinquirer.net/?article=17754
  • flip0mode - Tuesday, August 10, 2004 - link

    Oh man, I got here late. Wow, what a response. I think that the response thread is much more interesting than the benchies. OK, pass me a cold beer too...thanx...

    I guess I'll throw 2 more cents on top of this mountain of pennies and see how tall it will get; though a stone in a field is generally more interesting than a stone in a mountain range.

    I think it's just fine to compare these two processors - inaccuracies aside. But the review should include amd's top of the line, seriously. Comparing high-end to low-end is extremely useful - we all want to know why we should spend the extra dough. But to be equitable you need a true sampling of high end and low end, which was not provided here.

    I don't care who wins the crown, I just want to see the price/performance. Besides, I'll probably never buy a server chip anyway - in the desktop arena, there is currently only one choice IMO.

    Now is a terrible time to buy anything anyway. At least 4-6 months need to go by to allow PCI-E and DDR-2 to find their way to market.

    All in all, I think two-thumbs down is not inappropriate, but this is a whole new level of kicking a dead horse. In the end I still have to thank you for this review just because of the feedback it generated!

    Priceless!!!
  • johnsonx - Tuesday, August 10, 2004 - link

    To #175, WiZzACkR,

    "but giving the unavailable 3.6F as an excuse i find inappropriate"

    Why? Theres no difference between the XEON 3.6 and the Pentium 4 3.6F, and further, aside from 64-bit, there is no difference between the 3.6F and the regular 3.6Ghz P4 (what is that, a 560?). So even if this were a CPU review, how would comparing a A64 3500+ and a 3.6Ghz P4 be so horribly wrong? A 3800+ is only 200Mhz faster and thus would affect the benchmarks by 12% at most (more likely only a few percent.)
  • MikeEFix - Tuesday, August 10, 2004 - link

    To clarify PR rating:
    The reference CPU used in AMDs' PR scheme is an AMD cpu not an Intel part.I always yhought this was common knowledge whether it be right or wrong, smart or stupid.

    Opteron a higher end smp capable chip had always been targeted at Xeon and lower cost solution to Itanium.
  • Noubourne - Tuesday, August 10, 2004 - link

    I see what you were trying to do. I am not super technical but the comparison did seem a bit odd to me when I first read it. If anything it shows the technical knowledge of the AT community, which is good, even if the manners are a bit lacking here and there.

    It's not easy to admit you are wrong, so I applaud the corrections that have been made. It would take at least 10 more articles like this to get me to stop reading AT, but I look forward to a more thorough comparison in the future. My Linux experiments have been mostly failures (again, not too technical), but I do try and it will be cool to see some more "real world" benchmarks on a true 64 bit OS. It might inspire me to d/l the latest fedora and give it another shot.

Log in

Don't have an account? Sign up now