ERP benchmark 1: SAP SD

The SAP SD (sales and distribution, 2-tier internet configuration) benchmark is an interesting benchmark as it is a real world client-server application. We decided to take a look at SAP's benchmark database. The results below are 2-tier benchmarks, so the database and the underlying OS can make a difference. It is best to keep those parameters the same, although the type of database (Oracle, MS SQL server, MaxDb or DB2) only makes a small difference. The results below all run on Windows 2003 Enterprise Edition and MS SQL Server 2005 database (both 64-bit). Every "2-tier Sales & Distribution" benchmark was performed on SAP's "ERP release 2005".

In our previous server oriented article, we summed up a rough profile of SAP S&D:

  • Very parallel resulting in excellent scaling
  • Low to medium IPC, mostly due to "branchy" code
  • Somewhat limited by memory bandwidth
  • Likes large caches (memory latency!)
  • Very sensitive to sync ("cache coherency") latency
SAP Sales & Distribution 2 Tier benchmark
(*) Estimate based on Intel's internal testing

If you focus on the cores only, the differences between the Xeon 55xx "Nehalem" and the previous generation Xeon 54xx "Harpertown" and Xeon 53xx "Clovertown" is relatively small. The enormous differences in SAP scores are solely a result of Hyper-Threading, the "uncore", and the NUMA platform. According to SAP benchmark specialist Tuan Bui (Intel), enabling Hyper-Threading accounts for a 31% performance boost. Using somewhat higher clocked DDR3 (1066 instead of 800 or 1333 instead of 1066) is good for another 2-3%. Enabling the prefetcher provides another 3% and the Turbo mode increased performance by almost 5%. As this SAP benchmark scales almost perfectly with clock speed, that means that the Xeon X5570 2.93GHz was in fact running at 3.07GHz on average.

Consider the following facts:

  • The quad-core AMD Opteron 8384 at 2.7GHz has no problem beating the higher clocked 5470 at 3.3GHz.
  • It is well known that the Xeon 54xx raw integer power is a lot higher than any of the Opterons (just take a look at SPECint2006).
  • Faster memory and thus bandwidth plays only a minor role in the SAP benchmark.
  • SAP threads share a lot of data (as is typical for these kind of database driven applications).

It is clear that synchronization (between L2 caches) that happens in the L3 cache, the fast inter-CPU synchronization that happens via dedicated interconnects, is what made the "native quad-cores" of AMD winners in this benchmark. Slow cache synchronization is probably the main reason why the integer crunching power hidden deep inside the "Harpertown" cores did not result in better performance.

Take the same (slightly improved) core and give it the right (L3 as quick syncing point for the L2s) cache architecture and NUMA platform with fast CPU interconnects and all that integer power is unleashed. The result is the Nehalem X5570 Xeon is clock for clock about 66% faster than its predecessor (19000 vs. 11420). Add SMT (Simultaneous Multi-Threading) and you allow the integer core to process a second thread when it is bogged down by one of those pesky branches. The last hurdle for supreme SAP performance is taken: The eight core "Nehalem" server is just as fast as a 24 core "Dunnington" and 80% faster than the competition.

AMD has just launched the Opteron 2389 at 2.9GHz. We estimate that this will bring AMD's best SAP score to about 14800, so Nehalem's advantage will be lowered to ~70%. Unfortunately for AMD, that is still a very large advantage!

Benchmark Configuration OLTP - Dell DVD Store
Comments Locked

44 Comments

View All Comments

  • Veteran - Wednesday, April 1, 2009 - link

    I didn't mean to offend you, because i can imagine how much time it takes to test hardware properly. And i personally think that OLTP/OLAP testing is very innovative and needed. Because otherwise people would have no idea what to buy for servers. You cannot let you server purchase be influenced with meaningless (for servers) simple benchmarks like 3D 2006/Vantage/FPS test etc.
    You guys always are doing a great a job at testing any piece of hardware, but it is just feeling to much biased towards Intel. For example, at the last page of this review you get a link to Intel resource Center (in the same place as the next button). If you have things like that, you are not (trying to be) objective IMO.
  • JohanAnandtech - Wednesday, April 1, 2009 - link

    Thank you for clarifying in a very constructive way.

    "the last page of this review you get a link to Intel resource Center"

    I can't say I am happy with that link as it creates the wrong impression. But the deal is: editors don't involve in ad management, ad sales people don't get involved when it comes to content.

    So all I can say is to judge our content, not our ads. And like I said, it didn't stop us from claiming that Shanghai was by far the best server CPU a few months ago. And that conclusion was not on many sites.
  • Veteran - Wednesday, April 1, 2009 - link

    Thanks for clarrifying this matter.

    But ad sales people should know this creates the wrong impression. A review site (for me at least) is all about objectivity and credibility. When you place a link to Intel's Resource Center at the end of every review, it feels weird. People on forums already call Anandtech, Inteltech. And i don't think this is what you guys want.

    I always liked Anandtech since when I was a kid, and I still do. You guys always have one of the most in-depth reviews (especially on the very technical side) and I like that. But you guys are gaining some very negative publicity on the net.
  • BaronMatrix - Tuesday, March 31, 2009 - link

    Unfortunately, I don't buy from or recommend criminals.
  • carniver - Wednesday, April 1, 2009 - link

    AMDZone is the biggest joke on the internet. I just went there to see how the zealots like abinstein are still doing their damage control; just like before he went on rambling how the Penryn is still weak against Shanghai, and the old and tired excuses like how if people all bought AMD they can drop in upgrades etc etc. ZootyGray...he's the biggest joke on AMDZone. None of them had the mental capacity to accept AMD has been DEFEATED, which is disappointing but funny to say the least
  • duploxxx - Wednesday, April 1, 2009 - link

    It's not just AMDZone, you are just the opposite. Its like in Woodcrest and conroe times, it's not because the high-end cpu is the best of all that the rest of the available cpu's in the line is by default better. It's all about price performance ratio. Like many who were buying the low-end and think they had bought the better system, well wrong bet.

    As mentioned before, why not test the mid range that is where the sales will be. Time to test 5520-5530 against 2380-82 after all those have the same price.
  • carniver - Wednesday, April 1, 2009 - link

    Your argument is valid, however, it just so happens that for low end 1S systems the Penryns are doing just fine against the Shanghais, for higher end 2S systems they used to be limited by memory bandwidth and AMD pulls ahead. No more is this the case, Intel now beats AMD in their own territory.
  • CHADBOGA - Tuesday, March 31, 2009 - link

    You probably also can't afford to buy a computer, so I doubt that Intel will be too concerned with your AMDZone insanity. LOL!!!!
  • smilingcrow - Tuesday, March 31, 2009 - link

    Those grapes you are chewing on sure sound sour to me. Try listening to a few tracks by The Fun Loving Criminals to help take away the bad taste.
  • cjcoats - Tuesday, March 31, 2009 - link

    There's more to HPC applications than you indicate: environmental modeling apps, particularly, tend to be dominated by memory access patterns rather than by I/O or pure computation. Give me a ring if you'd like some help with that -- I'm local for you, in fact...

Log in

Don't have an account? Sign up now