Power

At this point in time, performance/watt comparisons are impossible. The current AMD Opteron 6100 systems available to reviewers are very basic reference systems. They consist of a motherboard, server CPUs, and low RPM desktop fans. The Xeon X7560 systems available are fully featured quad-socket systems with remote management cards, SAS/SATA backplanes, an extra daughterboard for PCIe expansion slots and so on. A decent power comparison can only happen when we receive a similar OEM Opteron system. We still expect that the Xeon X7560 systems will need more power. For example, a medium range server will come with eight SMBs, good for about 60W extra power consumption. High-end quad systems will have 16 SMBs, probably good for 120W. Add to that most Xeon 7500 are in the 130W TDP range (Opterons: 115W TDP) and that the chipset consume slightly more too, and you can see why the Xeon X7560 will likely require more power.

Conclusion

At this moment, we have started with our tests on the quad Xeon X7560. That is the natural "habitat" for this Xeon if you want to measure performance. The dual Xeon Nehalem EX is not meant to be a top performer, but instead enables servers with high amounts of memory protected by a battery of reliability features. We have to admit that we do not have the tests at this time to check Intel's RAS claims. But Google's own research shows that we should not underestimate soft errors, and VMware's enthusiasm for MCA tells us that we should take this quite serious. Microsoft has pledged to support this in Windows 2008 R2; Red Hat and Novell are supporting MCA too in their next releases. Software support will be here pretty soon.

From that point of view, the Dell R810 makes a lot of sense: running close to hundred VMs on a 256GB system with only ECC for protection seems risky: one bad VM can take down a complete host system. High availability at the software level as found in VMware's vSphere is fine, but 100 failing VMs can wreak havoc on the performance of your virtualized cluster. So in the "workloads needing large amounts of memory and availability" market, we don't see any real competition for the newest Intel Xeon when running ERP, OLTP or virtualization loads. Dell's R810 has made these kinds of high reliability servers more accessible with the R810, and for that we have to congratulate them. It's too bad that Intel does not play along.

We feel Intel has missed an opportunity with the pricing of the Xeon 6500 series, which is high for a dual-CPU capable server processor. Intel could make it easier for Dell to bring "mainframe RAS" to IT departments with a smaller budget. Right now, two server CPUs (X6550) can easily be up to 35% of the cost of a R810 server, which is luckily still an affordable server. Those with Oracle or IBM software licenses don't care, but the rest of us do.

The situation is very different if we look at the quad Xeon servers. The Dell R910 in the midrange and the IBM X3950 servers in the high-end really bring the x86 server to new heights. For $50,000 it is possible to spec a 32-core quad Xeon X7560 system with 512GB of RAM. For comparison, for that kind of money, you'll get 16 Power 7 cores at 3GHz and only 64GB of RAM in an IBM Power 750 system. The Power 7 might still be the fastest server around, but the Xeon 7500 servers are a real bargain in this market.

The Xeon 7500 is not for the HPC processing and bandwidth craving people; AMD has a better and most of all cheaper alternative. Likewise, the 7500 does not offer the price/performance or performance/watt that the popular dual-CPU servers offer. And there is a key market where we prefer the AMD Opteron 6100: the data mining market. AMD's twelve-core Opteron performs great here, and a very rare memory glitch should not be a disaster for the data mining people: you just start your reporting query again. Many work on a copy of the production database anyway.

But for the rest, the Xeon 7500 series does what it's supposed to do. It out-scales the other Xeons in key applications such as ERP, OLTP, and heavy virtualization scenarios while offer RISC—or should we say "Itanium"—like reliability. And looking at the published benchmarks, it is a winner in the Java benchmarks too. The Xeon 7500 is the most attractive expandable/MP Xeon so far, and the first one that can really threaten the best RISC CPUs in their home market.

I like to thank Tijl Deneut of the Sizing Servers Lab (Dutch) for his help with the benchmarking.

Virtualization and Consolidation
Comments Locked

23 Comments

View All Comments

  • dastruch - Monday, April 12, 2010 - link

    Thanks AnandTech! I've been waiting for an year for this very moment and if only those 25nm Lyndonville SSDs were here too.. :)
  • thunng8 - Monday, April 12, 2010 - link

    For reference, IBM just released their octal chip Power7 3.8Ghz result for the SAP 2 tier benchmark. The result is 202180 saps for approx 2.32x faster than the Octal chipNehalem-EX
  • Jammrock - Monday, April 12, 2010 - link

    The article cover on the front page mentions 1 TB maximum on the R810 and then 512 GB on page one. The R910 is the 1TB version, the R810 is "only" 512GB. You can also do a single processor in the R810. Though why you would drop the cash on an R810 and a single proc I don't know.
  • vol7ron - Tuesday, April 13, 2010 - link

    I wish I could afford something like this!

    I'm also curious how good it would be at gaming :) I know in many cases these server setups under-perform high end gaming machines, but I'd settle :) Still, something like this would be nice for my side business.
  • whatever1951 - Tuesday, April 13, 2010 - link

    None of the Nehalem-EX numbers are accurate, because Nehalem-EX kernel optimization isn't in Windows 2008 Enterprise. There are only 3 commercial OSes right now that have Nehalem-EX optimization: Windows Server R2 with SQL Server 2008 R2, RHEL 5.5, SLES 11, and soon to be released CentOS 5.5 based on RHEL 5.5. Windows 2008 R1 has trouble scaling to 64 threads, and SQL Server 2008 R1 absolutely hates Nehalem-EX. You are cutting Nehalem-EX benchmarks short by 20% or so by using Windows 2008 R1.

    The problem isn't as severe for Magny cours, because the OS sees 4 or 8 sockets of 6 cores each via the enumerator, thus treats it with the same optimization as an 8 socket 8400 series CPU.

    So, please rerun all the benchmarks.
  • JohanAnandtech - Tuesday, April 13, 2010 - link

    It is a small mistake in our table. We have been using R2 for months now. We do use Windows 2008 R2 Enterprise.
  • whatever1951 - Tuesday, April 13, 2010 - link

    Ok. Change the table to reflect Windows Server 2008 R2 and SQL Server 2008 R2 information please.

    Any explanation for such poor memory bandwidth? Damn, those SMBs must really slow things down or there must be a software error.
  • whatever1951 - Tuesday, April 13, 2010 - link

    It is hard to imagine 4 channels of DDR3-1066 to be 1/3 slower than even the westmere-eps. Can you remove half of the memory dimms to make sure that it isn't Dell's flex memory technology that's slowing things down intentionally to push sales toward R910?
  • whatever1951 - Tuesday, April 13, 2010 - link

    As far as I know, when you only populate two sockets on the R810, the Dell R810 flex memory technology routes the 16 dimms that used to be connected to the 2 empty sockets over to the 2 center CPUs, there could be significant memory bandwidth penalties induced by that.
  • whatever1951 - Tuesday, April 13, 2010 - link

    "This should add a little bit of latency, but more importantly it means that in a four-CPU configuration, the R810 uses only one memory controller per CPU. The same is true for the M910, the blade server version. The result is that the quad-CPU configuration has only half the bandwidth of a server like the Dell R910 which gives each CPU two memory controllers."

    Sorry, should have read a little slower. Damn, Dell cut half the memory channels from the R810!!!! That's a retarded design, no wonder the memory bandwidth is so low!!!!!

Log in

Don't have an account? Sign up now