Power

At this point in time, performance/watt comparisons are impossible. The current AMD Opteron 6100 systems available to reviewers are very basic reference systems. They consist of a motherboard, server CPUs, and low RPM desktop fans. The Xeon X7560 systems available are fully featured quad-socket systems with remote management cards, SAS/SATA backplanes, an extra daughterboard for PCIe expansion slots and so on. A decent power comparison can only happen when we receive a similar OEM Opteron system. We still expect that the Xeon X7560 systems will need more power. For example, a medium range server will come with eight SMBs, good for about 60W extra power consumption. High-end quad systems will have 16 SMBs, probably good for 120W. Add to that most Xeon 7500 are in the 130W TDP range (Opterons: 115W TDP) and that the chipset consume slightly more too, and you can see why the Xeon X7560 will likely require more power.

Conclusion

At this moment, we have started with our tests on the quad Xeon X7560. That is the natural "habitat" for this Xeon if you want to measure performance. The dual Xeon Nehalem EX is not meant to be a top performer, but instead enables servers with high amounts of memory protected by a battery of reliability features. We have to admit that we do not have the tests at this time to check Intel's RAS claims. But Google's own research shows that we should not underestimate soft errors, and VMware's enthusiasm for MCA tells us that we should take this quite serious. Microsoft has pledged to support this in Windows 2008 R2; Red Hat and Novell are supporting MCA too in their next releases. Software support will be here pretty soon.

From that point of view, the Dell R810 makes a lot of sense: running close to hundred VMs on a 256GB system with only ECC for protection seems risky: one bad VM can take down a complete host system. High availability at the software level as found in VMware's vSphere is fine, but 100 failing VMs can wreak havoc on the performance of your virtualized cluster. So in the "workloads needing large amounts of memory and availability" market, we don't see any real competition for the newest Intel Xeon when running ERP, OLTP or virtualization loads. Dell's R810 has made these kinds of high reliability servers more accessible with the R810, and for that we have to congratulate them. It's too bad that Intel does not play along.

We feel Intel has missed an opportunity with the pricing of the Xeon 6500 series, which is high for a dual-CPU capable server processor. Intel could make it easier for Dell to bring "mainframe RAS" to IT departments with a smaller budget. Right now, two server CPUs (X6550) can easily be up to 35% of the cost of a R810 server, which is luckily still an affordable server. Those with Oracle or IBM software licenses don't care, but the rest of us do.

The situation is very different if we look at the quad Xeon servers. The Dell R910 in the midrange and the IBM X3950 servers in the high-end really bring the x86 server to new heights. For $50,000 it is possible to spec a 32-core quad Xeon X7560 system with 512GB of RAM. For comparison, for that kind of money, you'll get 16 Power 7 cores at 3GHz and only 64GB of RAM in an IBM Power 750 system. The Power 7 might still be the fastest server around, but the Xeon 7500 servers are a real bargain in this market.

The Xeon 7500 is not for the HPC processing and bandwidth craving people; AMD has a better and most of all cheaper alternative. Likewise, the 7500 does not offer the price/performance or performance/watt that the popular dual-CPU servers offer. And there is a key market where we prefer the AMD Opteron 6100: the data mining market. AMD's twelve-core Opteron performs great here, and a very rare memory glitch should not be a disaster for the data mining people: you just start your reporting query again. Many work on a copy of the production database anyway.

But for the rest, the Xeon 7500 series does what it's supposed to do. It out-scales the other Xeons in key applications such as ERP, OLTP, and heavy virtualization scenarios while offer RISC—or should we say "Itanium"—like reliability. And looking at the published benchmarks, it is a winner in the Java benchmarks too. The Xeon 7500 is the most attractive expandable/MP Xeon so far, and the first one that can really threaten the best RISC CPUs in their home market.

I like to thank Tijl Deneut of the Sizing Servers Lab (Dutch) for his help with the benchmarking.

Virtualization and Consolidation
Comments Locked

23 Comments

View All Comments

  • JohanAnandtech - Tuesday, April 13, 2010 - link

    "Damn, Dell cut half the memory channels from the R810!"

    You read too fast again :-). Only in Quad CPU config. In dual CPU config, you get 4 memory controllers, which connect each two SMBs. So in a dual Config, you get the same bandwidth as you would in another server.

    The R810 targets those that are not after the highest CPU processing power, but want the RAS features and 32 DIMM slots. AFAIK,
  • whatever1951 - Tuesday, April 13, 2010 - link

    2 channels of DDR3-1066 per socket in a fully populated R810 and if you populate 2 sockets, you get the flex memory routing penalty...damn..............!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! R810 sucks.
  • Sindarin - Tuesday, April 13, 2010 - link

    whatever1951 you lost me @ Hello.........................and I thought Sauron was tough!! lol
  • JohanAnandtech - Tuesday, April 13, 2010 - link

    "It is hard to imagine 4 channels of DDR3-1066 to be 1/3 slower than even the westmere-eps."

    On one side you have a parallel half duplex DDR-3 DIMM. On the other side of the SMB you have a serial full duplex SMI. The buffers might not perform this transition fast enough, and there has to be some overhead. I also am still searching for the clockspeed of the IMC. The SMIs are on a different (I/O) clockdomain than the L3-cache.

    We will test with Intel's / QSSC quad CPU to see whether the flexmem bridge has any influence. But I don't think it will do much. You might add a bit of latency, but essentially the R810 is working like a dual CPU with four IMCs just like another (Dual CPU) Nehalem EX server system would.
  • whatever1951 - Tuesday, April 13, 2010 - link

    Thanks for the useful info. R810 then doesn't meet my standard.

    Johan, is there anyway you can get your hands on a R910 4 Processor system from Dell and bench the memory bandwidth to see how much that flex mem chip costs in terms of bandwidth?
  • IntelUser2000 - Tuesday, April 13, 2010 - link

    The Uncore of the X7560 runs at 2.4GHz.
  • JohanAnandtech - Wednesday, April 14, 2010 - link

    Do you have a source for that? Must have missed it.
  • Etern205 - Thursday, April 15, 2010 - link

    I think AT needs to fix this "RE:RE:RE...:" problem?
  • amalinov - Wednesday, April 14, 2010 - link

    Great article! I like the way in witch you describe the memory subsystem - I have readed the Intel datasheets and many news articles about Xeon 7500, but your description is the best so far.

    You say "So each CPU has two memory interfaces that connect to two SMBs that can each drive two channels with two DIMMS. Thus, each CPU supports eight registered DDR3 DIMMs ...", but if I do the math it seems: 2 SMIs x 2 SMBs x 2 channels x 2 DIMMs = 16 DDR3 DIMMs, not 8 as written in the second sentence. Later in the article I think you mention 16 at different places, so it seems it is realy 16 and not 8.

    What about Itanium 9300 review (including general background on the plans of OEMs/Intel for IA-64 platform)? Comparision of scalability(HT/QPI)/memory/RAS features of Xeon 7500, Itanium 9300 and Opteron 6000 would be welcome. Also I would like to see a performance comparision with appropriate applications for the RISC mainframe market (HPC?) with 4- and 8-socket AMD, Intel Xeon, Intel Itanium, POWER7, newest SPARC.
  • jeha - Thursday, April 15, 2010 - link

    You really should review the IBM 3850 X5 I think?

    They have some interesting solutions when it comes to handling memory expansions etc.

Log in

Don't have an account? Sign up now