Conclusion

Performance and RAS features took a giant leap forwared when Intel replaced the Xeon 7400 with the Xeon 7500. The memory subsystem went from a high latency, totally bandwidth choked loser (hardly 10GB/s for 24 cores) to a low latency and very high bandwidth champion (up to 70GB/s). The Xeon E7 builds further on that excellent platform, and adds up to 35% higher performance.

We now have a proven platform with excellent RAS features that needs slightly less power now while it provides a decent performance boost. That's excellent, but the Xeon E7 still has a few weakness. One weakness is the relatively high power consumption at idle load. Compared to the high-end Power 7 servers, this kind of power consumption is probably very reasonable. The Power 7 CPUs are in the 100 to 170W TDP range, while the Xeon E7s are in the 95 to 130W TDP range. A quad 3.3GHz Power 755 with (256GB RAM) server consumes 1650W according to IBM (slide 24), while our first measurements show that our 2.4GHz E7-4870 server will consume about 1300W in those circumstances.

Considering that the 3.3GHz Power 7 and 2.4GHz E7-4870 perform at the same level, we'll go out on a limb and assume that the new Xeon wins in the performance/watt race. AMD might take advantage of this "weakness", but availablility of quad 16-core "Bulldozer" servers is still months away and we don't know what the power use will be yet.

The 10-core Xeons are pretty expensive ($3000-4600 per CPU), but many of these systems are bought to run software that will cost 10 times more. In a nutshell, Intel's Xeon E7 moves up the server CPU food chain. The Xeon E7 closes the performance gap with the best RISC CPUs (see the SAP benchmarks), offers lower power and cost, and the rest of the x86 competition is relegated to the low-end of the quad x86 market.

For those looking for a virtualization platform, there is no x86 server that is able to offer such low response times at such high consolidation ratios. However, in order to get a good performance/watt ratio, you need to make sure that your quad Xeon E7 servers are working under high CPU loads. The quad Xeon E7 server is a good platform for consolidating CPU intensive applications. For less intensive VMs, it makes a lot more sense to check out the dual Xeon and quad Opteron offerings.

I would also like to thank to Tijl Deneut for his invaluable assistance.

Real-World Power
Comments Locked

62 Comments

View All Comments

  • Fallen Kell - Thursday, May 19, 2011 - link

    As the subject says. Would love to see how these deal with something like Linpack or similar.
  • erple2 - Thursday, May 19, 2011 - link

    I'd be more interested at seeing how they perform in slightly more "generic" and non-GPU optimizeable workloads. If I'm running Linpack or other FPU operations, particularly those that parallelize exceptionally well, I'd rather invest time and money into developing algorithms that run on a GPU than a fast CPU. The returns for that work are generally astounding.

    Now, that's not to say that General Purpose problems work well on a GPU (and I understand that). However, I'm not sure that measuring the "speed" of a single processor (or even a massively parallelized load) would tell you much, other than "it's pretty fast, but if you can massively parallelize a computational workload, figure out how to do it on a commodity GPU, and blow through it at orders of magnitude faster than any CPU can do it".

    However, I can't see running any virtualization work on a GPU anytime soon!
  • stephenbrooks - Thursday, May 19, 2011 - link

    Yeah, well, in an ideal world...

    But sometimes (actually, every single time in my experience) the "expensive software" that's been bought to run on these servers lacks a GPU option. I'm thinking of electromagnetic or finite element analysis code.

    Finite element engines are the sort of thing that companies make a lot of money selling. They are complicated. The commercial ones probably have >10 programmer-years of work in them, and even if they weren't fiercely-protected closed source, porting and re-optimising for a GPU would be additional years work requiring programmers again at a high level and with a lot of mathematical expertise.

    (There might be some decent open-source alternatives around, but they lack the front ends and GUI that most engineers are comfortable using.)

    If you think fixing the above issues are "easy", go ahead. You'll make millions.
  • L. - Thursday, May 19, 2011 - link

    lol

    if you code .. i don't want to read your code
  • carnachion - Friday, May 20, 2011 - link

    I agree with you. In my experience GPU computing for scientific applications are still in it's infancy, and in some cases the performance gains are not so high.

    There's still a big performance penalty by using double precision for the calculations. In my lab we are porting some programs to GPU, we started using a matrix multiplication library that uses GPU in a GTX590. Using one of the 590's GPU it was 2x faster than a Phenon X6 1100T, and using both GPUs it was 3.5x faster. So not that huge gain, using a Magny-Cours processor we could reach the performance of a single GPU, but of course at a higher price.

    Usually scientific applications can use hundreds of cores, and they are tunned to get a good scaling. But I don't know how GPU calculations scales with the number of GPUs, from 1 to 2 GPUs we got this 75% boost, but how it will perform using inter-node communication, even with a Infiniband connection I don't know if there'll be a bottleneck for real world applications. So that's why people still invest in thousands of cores computers, GPU still need a lot of work to be a real competitor.
  • DanNeely - Saturday, May 21, 2011 - link

    single vs double precision isn't the only limiting factor for GPU computing. The amount of data you can have in cache per thread is far smaller than on a traditional CPU. If your working set is too big to fit into the tiny amount of cache available performance is going to nose dive. This is farther aggravated by the fact that GPU memory systems are heavily optimized for streaming access and that random IO (like cache misses) suffers in performance.

    The result is that some applications which can be written to fit the GPU model very well will see enormous performance increases vs CPU equivalents. Others will get essentially nothing.

    Einstein @ Home's gravitational wave search app is an example of the latter. The calculations are inherently very random in memory access (to the extent that it benefits by about 10% from triple channel memory on intel quads; Intel's said that for quads there shouldn't be any real world app benefit from the 3rd channel). A few years ago when they launched cuda, nVidia worked with several large projects on the BOINC platform to try and port their apps to CUDA. The E@H cuda app ended up no faster than the CPU app and didn't scale at all with more cuda cores since all they did was to increase the number of threads stalled on memory IO.
  • Marburg U - Thursday, May 19, 2011 - link

    Finally something juicy,
  • JarredWalton - Thursday, May 19, 2011 - link

    So, just curious: is this spam (but no links to a separate site), or some commentary that didn't really say anything? All I've got is this, "On the nature of things":

    http://en.wikipedia.org/wiki/De_rerum_natura

    Maybe I missed what he's getting at, or maybe he's just saying "Westmere-EX rocks!"
  • bobbozzo - Monday, May 23, 2011 - link

    Jarred, my guess is that it is spam, and that there was a link or some HTML posted which was filtered out by the comments system.

    Bob
  • lol123 - Thursday, May 19, 2011 - link

    Why is there a 2 socket only line of E7 (E7-28xx), but at least as far as I can tell, not any 2-socket motherboards or servers? Are those simply not available yet?

Log in

Don't have an account? Sign up now