Virtualization Performance: Linux VMs on ESXi

We introduced our new vApus FOS (For Open Source) server workloads in our review of the Facebook "Open Compute" servers. In a nutshell, it is a mix of four VMs with open source workloads: two PhpBB websites (Apache2, MySQL), one OLAP MySQL "Community server 5.1.37" database, and one VM with VMware's open source groupware Zimbra 7.1.0. Zimbra is quite a complex application as it contains the following components:

  • Jetty, the web application server
  • Postfix, an open source mail transfer agent
  • OpenLDAP software, user authentication
  • MySQL is the database
  • Lucene full-featured text and search engine
  • ClamAV, an anti-virus scanner
  • SpamAssassin, a mail filter
  • James/Sieve filtering (mail)

All VMs are based on a minimal CentOS 6 setup with VMware Tools installed. All our current virtualization testing is on top of the hypervisor which we know best: ESXi (5.0). We have changed two things in our vApusMark FOS setup: we upgradeded the guestOS from 5.6 to 6.0 and increased the number of vCPUs of the OLAP VM from 2 to 4. This small upgrade means that our latest results should not be compared to the results in our older articles. We test with four tiles (one tile = four VMs). Each tile needs nine vCPUs, so the test requires 36 vCPUs.

vApusMark FOS

For being just a minor update, the new Piledriver core does pretty well. Clock for clock performance goes up by 11%. The total performance gain (IPC+clock) is 20%, which is significant. The Opteron 6376 performs only 4% better than its direct competitor the E5-2630 (as the latter will perform very similar to our E5-2660 with 6 cores), but that is not bad at all: you get slightly better performance for a lower (server) price.

The top of the line 6380 cannot keep up with the best Xeons. Offering 86% of the more expensive Xeon E5-2660 is hardly a disaster, however. Note "maximum amount of affordable memory" is on top of many virtualization hosts shopping lists followed by price/performance. For those buyers, considering that a server based upon the Opteron cost less, the Opteron is once again a potent virtualization host if the power usage is similar.

With the lack of c-states, the Opteron 6174 did pretty poorly. The Opteron 6276 consumed a lot less at idle than its predecessor, but consumed a lot more when pressured to perform at high load. So we were very keen to learn whether AMD has improved power consumption too. Did AMD finally get that part right?

Benchmarking Configuration Measuring Real-World Power Consumption
Comments Locked

55 Comments

View All Comments

  • coder543 - Wednesday, February 20, 2013 - link

    99%? I love your highly scientific numbers. and yes, of course only Intel can design a perfect processor. I'm glad you were here to let everyone know.

    To quote Abraham Lincoln, (no, not really) "All of our servers run Intel. Everything AMD makes is no better than British tea."
  • Tams80 - Wednesday, February 20, 2013 - link

    How much are Intel paying you? XD

    Seriously though; you've gone through the entire comments* posting walls of texts that add little to the discussion. Not only that, but your posts are a little offensive.

    *I realise I'm being hypocritical here.
  • JKflipflop98 - Wednesday, February 20, 2013 - link

    Well, Intel does pay me and I'll be the first to say these chips are lookin pretty good in comparison with their previous generation counterparts. Good value for the money for sure.

    As Anand says, however, HPC users are usually after the "extreme" ends of the scale. They're either after max performance or max performance to fit into a certain power/heat envelope. In either case, we win.
  • Tams80 - Wednesday, February 20, 2013 - link

    I'm sure you know what I mean. It wasn't exactly high brow humour.

    They certainly do look good, especially for a company that has already invested in AMD chips. Intel may well be better in both use cases, but at least AMD are providing competent competition.
  • tech6 - Wednesday, February 20, 2013 - link

    The AMD 6x000 series has always looked nearly competitive on paper but is nowhere near Intel performance and efficiency. We have 3 data centers and one is running a mix of 6100 and 6200 Opterons while the others a re older Xeon 7300s and new E5 Xeons. In terms of single threaded and total performance of the 6x00 series cannot keep up with even old 7300 Xeons and can't touch the E5s. What AMD needs is a 30-40% boost in real world performance before they could be considered competitive. AMD also needs better relations with VMWare to optimize memory management on that platform.

    The price difference won't help them as the cost for a data center host is mostly software and can be $15 vCloud and $10K hardware. That reduces the cost advantage to 5% but delivers worse performance and uses more power.

    Most data centers are looking to get the most from their VMWare investments while reducing power consumption and these AMDs do neither.
  • duploxxx - Wednesday, February 20, 2013 - link

    interesting information, but hard to catch if you don't add some figures and real data.

    Firsts of all the 7300 series had huge disadvantages with there FSB, so mentioning that these are way faster then the 6100-6200 opteron series is debatable. I 100% tend to disagree and we had severe Vmware performance issues on these machines on our highend applications.

    i'll just used anandtech as a refference:
    http://www.anandtech.com/show/2851/8
    http://www.it.anandtech.com/show/2978/amd-s-12-cor...

    even the 7400 series are a dog against opteron 8000 series and they are way older and slower against the 6000 series.

    for the E5 you have a point there, often the E5 series show a higher responsive platform, but once you load real life applications within hypervisor and they are starting to hit those HT cores we have seen several degraded performance within our datacenters, this is not really resulted into the anandtech VAPU's scores due to some sw within the benchmark that provides some code optimised results for the intels (the web servers) hence the higher score.

    The 6200 series did showe some response disadvantages but many things have to do with configuration of bios and power profiles in both server and hypervisor. might want to blame the setup rather then the servers. so for 6200 series we actually bought a 10% higher clock speed version to cover that, but reduced that again now with 6300 series.
  • silverblue - Wednesday, February 20, 2013 - link

    I'm going to go trawl the internet (note I said trawl, not troll - very important to bear in mind) for articles on FX CPUs resulting in PCs dying... nope, no matches. Funny, huh?

    I've also run a search concerning AMD CPUs producing incorrect results and crashing; any such occurrences would be the results of design bugs which, I must point out, are not limited to AMD. Nehalem had a bug causing spurious interrupts that locks up the hypervisor on Windows Server 2008 R2, for example. Core 2 had a huge list of bugs.
  • Shadowmaster625 - Wednesday, February 20, 2013 - link

    It is hard to disagree with the statement, knowing how overpaid US IT professionals are. But I just want to point out that this mentality is one of the reasons IT is being outsourced at a furious rate. Keep that in mind before you go blaming someone else for US jobs being lost.

    This meager cost savings may not matter here, but what about some company in Asia? They might actually bite on a few hundred dollar savings, especially if they are ordering quantities in the hundreds. In that case, $300 becomes $30,000. Which might be more than they spend on the people who deploy those servers.
  • ExarKun333 - Wednesday, February 20, 2013 - link

    Outsourced work isn't much cheaper these days and the workers are of much less quality, on a whole.
  • sherlockwing - Wednesday, February 20, 2013 - link

    Except in Asia( especially developing countries) the cost of electricity is a lot higher due to rapidly expanding industry,population & lacking power plants.

Log in

Don't have an account? Sign up now