Virtualization Performance: Linux VMs on ESXi

We introduced our new vApus FOS (For Open Source) server workloads in our review of the Facebook "Open Compute" servers. In a nutshell, it is a mix of four VMs with open source workloads: two PhpBB websites (Apache2, MySQL), one OLAP MySQL "Community server 5.1.37" database, and one VM with VMware's open source groupware Zimbra 7.1.0. Zimbra is quite a complex application as it contains the following components:

  • Jetty, the web application server
  • Postfix, an open source mail transfer agent
  • OpenLDAP software, user authentication
  • MySQL is the database
  • Lucene full-featured text and search engine
  • ClamAV, an anti-virus scanner
  • SpamAssassin, a mail filter
  • James/Sieve filtering (mail)

All VMs are based on a minimal CentOS 6 setup with VMware Tools installed. All our current virtualization testing is on top of the hypervisor which we know best: ESXi (5.0). We have changed two things in our vApusMark FOS setup: we upgradeded the guestOS from 5.6 to 6.0 and increased the number of vCPUs of the OLAP VM from 2 to 4. This small upgrade means that our latest results should not be compared to the results in our older articles. We test with four tiles (one tile = four VMs). Each tile needs nine vCPUs, so the test requires 36 vCPUs.

vApusMark FOS

For being just a minor update, the new Piledriver core does pretty well. Clock for clock performance goes up by 11%. The total performance gain (IPC+clock) is 20%, which is significant. The Opteron 6376 performs only 4% better than its direct competitor the E5-2630 (as the latter will perform very similar to our E5-2660 with 6 cores), but that is not bad at all: you get slightly better performance for a lower (server) price.

The top of the line 6380 cannot keep up with the best Xeons. Offering 86% of the more expensive Xeon E5-2660 is hardly a disaster, however. Note "maximum amount of affordable memory" is on top of many virtualization hosts shopping lists followed by price/performance. For those buyers, considering that a server based upon the Opteron cost less, the Opteron is once again a potent virtualization host if the power usage is similar.

With the lack of c-states, the Opteron 6174 did pretty poorly. The Opteron 6276 consumed a lot less at idle than its predecessor, but consumed a lot more when pressured to perform at high load. So we were very keen to learn whether AMD has improved power consumption too. Did AMD finally get that part right?

Benchmarking Configuration Measuring Real-World Power Consumption
Comments Locked

55 Comments

View All Comments

  • arnd - Wednesday, February 20, 2013 - link

    I have dual Opteron 6344 workstation system, which tends to be either near complete idle or near complete busy, so C states are extremely important to me. The CPU has power sensors that are exposed in Linux using the 'sensors' tool. With C6 enabled, I get the power consumption per socket down to 42 Watts, which still seems like a lot, but disabling C6 made it jump to 104W per socket, when under 100% load it is constantly within 1W of the 115W TDP limit.
    I did not see a significant impact of C1E, neither with C6 enabled nor disabled, presumably because I rarely have cores that are idle for a short period.
    More annoying to me is the lack of S3 suspend mode, the system still consumes around 100W on S1.
  • nevertell - Wednesday, February 20, 2013 - link

    The difference I believe is that you cannot use AES-NI instructions when using Twofish and serpent. I guess that AMD's AES-NI implementation is just slower.
  • JohanAnandtech - Wednesday, February 20, 2013 - link

    Sounds reasonable. The question is then why Twofish and serpent are so fast on the Opteron. They probably scale very well with cores.
  • Yorgos - Wednesday, February 20, 2013 - link

    I've been abandoning tech sites due to stupid posters and internet trolls.
    There is so much addition info and questions in the comments and I don't know why are you letting people ruin that feature from your site?
    You should make a ranking system(similar to /. ) for users, in order to automatically hide someone's comments, so we don't have to double check every time the poster and/or the comment.

    I feel stupid for making that type of comment, also reading specific stupid opinions, below that article.
  • silverblue - Wednesday, February 20, 2013 - link

    I like your ideas, however most of the laugh (or should I say cringe?) worthy comments would be hidden and the entertainment value would be tainted by having to click the Show button all the time. ;)
  • lwatcdr - Wednesday, February 20, 2013 - link

    Or requiring real names.
  • JohanAnandtech - Wednesday, February 20, 2013 - link

    I had meetings and people visiting me, so I could not "baby sit" the reactions. But if you don't react to the offensive message we can delete them. So the best way to deal with th trolls is to ignore. Sooner or later, they will be banned.
  • coder111 - Wednesday, February 20, 2013 - link

    Because some of the people posting here are obviously trolling for Intel and do not bring anything constructive to the discussion.
  • MonkeyPaw - Wednesday, February 20, 2013 - link

    Yes, it is quite pathetic. An ignore button would take care of this situation nicely.
  • iamezza - Wednesday, February 20, 2013 - link

    An ignore button and a report button would be great!

Log in

Don't have an account? Sign up now