Conclusion: the SoCs

The proponents of high core and thread counts are quick to discount the brawny cores for scale-out applications. On paper, wide issue cores are indeed a bad match for such low ILP applications. However, in reality, the high clock speed and multi-threading of the Xeon E3s proved to be very powerful in a wide range of server applications.

The X-Gene 1 is the most potent ARM Server SoC we have ever seen. However, it is unfortunate that AppliedMicro's presentations have created inflated expectations.

AppliedMicro insisted that the X-Gene 1 is a competitor for the powerful Haswell and Ivy Bridge cores. So how do we explain the large difference between our benchmarks and theirs? The benchmark they used is the "wrk" benchmark, which is very similar to Apache Bench. The benchmark will hit the same page over and over again, and unless you do some serious OS and Network tweaking, your server will quickly run out of ports/connections and other OS resources. So the most likely explanation is that the Xeon measurements are achieved at lower CPU load and are bottlenecked by running out of network and OS resources.

This is in sharp contrast with our Drupal test, where we test with several different user patterns and thus requests. Each request is a lot heavier, and as a result available connections/ports are not the bottleneck. Also, all CPUs were in the 90-98% CPU load range when we compared the maximum throughput numbers.

The 40nm X-Gene can compete with the 22nm Atom C2000 performance wise, and that is definitely an accomplishment on its own. But the 40nm process technology and the current "untuned" state of ARMv8 software does not allow it to compete in performance/watt. The biggest advantage of the first 64-bit ARM SoCs is the ability for an ARM processor to use eight DIMM slots and address more RAM. Better software support (compilers, etc.) and the 28nm X-Gene 2 SoC will be necessary for AppliedMicro to compete with the Intel Xeon performance/watt wise.

The Atom C2750's raw performance fails to impress us in most of our server applications. Then again we were pleasantly surprised that its power consumption is below the official TDP. Still, in most server applications, a low voltage Xeon E3 outperforms it by a large margin.

And then there's the real star, the Xeon E3-1230L v3. It does not live up to the promise of staying below 25W, but the performance surpassed our expectations. The result is – even if you take into account the extra power for the chipset – an amazing performance/watt ratio. The end conclusion is that the introduction of the Xeon-D, which is basically an improved Xeon E3 with integrated PCH, will make it very hard for any competitor to beat the higher-end (20-40W TDP) Intel SoCs in performance/watt in a wide range of "scale-out" applications.

Conclusion: the Servers

As our tests with motherboards have shown, building an excellent micro or scale-out server requires much more thought than placing a low power SoC in a rack server. It is very easy to negate the power savings of such an SoC completely if the rest of server (motherboard, fans, etc.) is not built for efficiency.

The Supermicro MicroCloud server is about low acquisition costs and simplicity. In our experience, it is less efficient with low power Xeons as the cooling tends to consume proportionally more (between 7-12W per node with eight nodes installed). The cooling system and power supplies are built to work with high performance Xeon E3 processors.

HP limits the most power efficient SoCs (such as the Atom C2730) to cartridges that are very energy efficient but also come with hardware limitations (16GB max. RAM, etc.). HP made the right choice as it is the only way to turn the advantages of low power SoCs into real-world energy efficiency, but that means the low power SoC cartridges may not be ideal for many situations. You will have to monitor your application carefully and think hard about what you need and what not to create an efficient datacenter.

Of the products tested so far, the HP Moonshot tends to impress the most. Its cleverly designed cartridges use very little power and the chassis allows you to choose the right server nodes to host your application. There were a few application tests missing in this review, namely the web caching (memcached) and web front-end tests, but based on our experiences we are willing to believe that the m300/m350 cartridge are perfect for those use cases.

Still, we would like to see a Xeon E3 low voltage cartridge for a "full web infrastructure" (front- and back-end) solution. That is probably going to be solved once HP introduces a Xeon-D based cartridge. Once that is a reality, you can really "right-size" the Moonshot nodes to your needs. But even now, the HP Moonshot chassis offers great flexibility and efficiency. The flexibility does tend to cost more than other potential solutions – we have yet to find out the exact pricing details – but never before was it so easy to adapt your tier one OEM server hardware so well to your software.

The War of the SoCs: Performance/Watt


View All Comments

  • gdansk - Monday, March 9, 2015 - link

    xgene is not looking so great. Even if it is 50% more efficient as they promise they'll still be behind Atom. Reply
  • Samus - Monday, March 9, 2015 - link

    HP Moonshot chassis are still *drool* Reply
  • Krysto - Monday, March 9, 2015 - link

    The main problem with the non-Intel systems is not only that they use older processes compared to Intel, but that they use older processes even compared to the rest of the non-Intel chip industry. AMD is typically always behind 1 process node among non-Intel chip makers. If they'd at least use the cutting edge processes as they become available from non-Intel processes, maybe they'd stand a chance, especially now that the gap in process technologies is shrinking. Reply
  • Samus - Monday, March 9, 2015 - link

    AMD simply isn't as bad as people continually make them out to be. Yes, they're "behind" Intel but it's all in the approach. We are talking about two engineering houses that share nothing in common but a cross licensing agreement. AMD has very competitive CPU's to Intel's i5's for nearly half the price, but yes, they use more power (at times 1/3 more.)

    But facts are facts: AMD is the second high-tech CPU manufacture in the world. Not Qualcomm, not Samsung. It's pretty obvious AMD engineering talent spreads more diversity than anyone other than Intel, and potentially superior to Intel on GPU design (although this has obviously been shifting over the years as Intel hires more "GPU talent.")

    AMD in servers is a hard pill to swallow though. If purchasing based on price alone, it can be a compelling alternative, but for rack space or low-energy computing?
  • Taneli - Tuesday, March 10, 2015 - link

    AMD doesn't even make it in top 10 semiconductor companies in sales. Qualcomm is three, Samsung semicondutors six and Intel almost ten times the size of AMD.

    Outside of the gaming consoles they are being completely overrun by competition.
  • owan - Tuesday, March 10, 2015 - link

    I'm sorry, at one point I was an AMD fanboy, back when they actually deserved it based on their products, but you just sound like an apologist. Facts are the facts, FX processors aren't competitive with i5's in performance or power or performance/$ because they get smacked so hard they can't be cheap enough to make up for it. Their CPU designs are woefully out of date, their APU's are bandwidth starved and use way too much power to be useful in the one place they'd be great (mobile), and their lagging process tech means theres not much better coming on the horizon. I don't want to see them go, but at the rate ARM is eating up general computing share, it won't be long before AMD becomes completely irrelevant. It will be Intel vs. ARM and AMD will be an afterthought. Reply
  • xenol - Wednesday, March 11, 2015 - link

    Qualcomm is used in pretty much used in most cell phones in the US to the point you'd think Qualcomm is the only SoC manufacturer. I'm pretty sure that's also how it looks in most of the other markets as Korea. Plus even if their SoCs aren't being used, they're modems are heavily used.

    If anything, Qualcomm is bigger than AMD. Or rather, Qualcomm is the Intel of the SoC market.
  • xenol - Wednesday, March 11, 2015 - link

    [Response to myself since I can't edit]
    Qualcomm's next major competitor is Apple. But that's about it.

    Also I meant to say other markets except Korea.
  • CajunArson - Monday, March 9, 2015 - link

    Bear in mind that the Atom parts were commercially available in 2013, so they are by no means brand-new technology and the 14nm Atom upgrades will definitely help power efficiency even if raw performance doesn't jump a whole lot.

    Anandtech is also a bit behind the curve because Intel is about to release Xeon-D (8 Broadwell cores and integrated I/O in a 45 watt TDP, or lower), which is designed for exactly this type of workload and is going to massively improve performance in the low-power envelope sphere:
  • SarahKerrigan - Monday, March 9, 2015 - link

    14nm server Atom isn't coming.

    "Atom will become a consumer only SoC."

Log in

Don't have an account? Sign up now