Virtualization Performance: Linux VMs on ESXi

We introduced our new vApus FOS (For Open Source) server workloads in our review of the Facebook "Open Compute" servers. In a nutshell, it is a mix of four VMs with open source workloads: two PhpBB websites (Apache2, MySQL), one OLAP MySQL "Community server 5.1.37" database, and one VM with VMware's open source groupware Zimbra 7.1.0. Zimbra is quite a complex application as it contains the following components:

  • Jetty, the web application server
  • Postfix, an open source mail transfer agent
  • OpenLDAP software, user authentication
  • MySQL is the database
  • Lucene full-featured text and search engine
  • ClamAV, an anti-virus scanner
  • SpamAssassin, a mail filter
  • James/Sieve filtering (mail)

All VMs are based on a minimal CentOS 6 setup with VMware Tools installed. All our current virtualization testing is on top of the hypervisor which we know best: ESXi (5.0). We have changed two things in our vApusMark FOS setup: we upgradeded the guestOS from 5.6 to 6.0 and increased the number of vCPUs of the OLAP VM from 2 to 4. This small upgrade means that our latest results should not be compared to the results in our older articles. We test with four tiles (one tile = four VMs). Each tile needs nine vCPUs, so the test requires 36 vCPUs.

vApusMark FOS

For being just a minor update, the new Piledriver core does pretty well. Clock for clock performance goes up by 11%. The total performance gain (IPC+clock) is 20%, which is significant. The Opteron 6376 performs only 4% better than its direct competitor the E5-2630 (as the latter will perform very similar to our E5-2660 with 6 cores), but that is not bad at all: you get slightly better performance for a lower (server) price.

The top of the line 6380 cannot keep up with the best Xeons. Offering 86% of the more expensive Xeon E5-2660 is hardly a disaster, however. Note "maximum amount of affordable memory" is on top of many virtualization hosts shopping lists followed by price/performance. For those buyers, considering that a server based upon the Opteron cost less, the Opteron is once again a potent virtualization host if the power usage is similar.

With the lack of c-states, the Opteron 6174 did pretty poorly. The Opteron 6276 consumed a lot less at idle than its predecessor, but consumed a lot more when pressured to perform at high load. So we were very keen to learn whether AMD has improved power consumption too. Did AMD finally get that part right?

Benchmarking Configuration Measuring Real-World Power Consumption
Comments Locked

55 Comments

View All Comments

  • JohanAnandtech - Wednesday, February 20, 2013 - link

    Per Core. So with the -np 32 setting. I have tried less before, but the LS-DYNA really likes the extra load and store units of the second unit. so 32 MPI processes give a 30% boost.
  • alpha754293 - Wednesday, February 20, 2013 - link

    That's a healthy boost! It's amazing how these technologies are maturing to the point where conventional wisdom that they might be starved for FPU resources isn't enough to slow them down.

    It'd be interesting to see whether it makes a difference if you were to let the OS handle the job/process scheduling or whether manual intervention can help reduce some of the thread/process migration overhead, especially across 16 FPUs.
  • dmytty - Friday, February 22, 2013 - link

    Before the benchmarks came in, I looked at an HPC build for the new 6300 series. I saw the real sweet spot for AMD being the 6344 which is a CPU of type 2.6 Ghz @ 12 or 6 core (depending how you define a 'core'). Anandtech never mentioned this CPU in the review. (?)

    In simple $ terms for the CPU, it's the E5 2640 @ $815 vs the Opteron 6344 @ $415. So how does AMD not have a decisive price advantage?

    In 4S land the price advantage widens.

    AMD 4S...I priced a build (not including a case) at ~$3860 for 4 x 6344, 128 GB RAM, mobo and PSU. Note that this mobo could also go out to 256 GB. CPU cost is 4 x 6344 = $1660. Again, total system cost was $3860.

    Intel 4S...I picked the 4607 as being the best 'bang for the buck processor'. It's 6 cores @ 2.2 Ghz. However, at $885 per 4607 processor the CPU cost is more than double than AMD (ie 4x Intel 4607 = $3540). The same build cost using the Intel 4607 would then be $5740.

    AMD @ $3860 vs Intel @ $5740. Why did Anandtech not talk about 4S? Why no mention of the 6344?
  • dmytty - Friday, February 22, 2013 - link

    I forgot to mention that the Intel board I specified for 4S build was $1200 whereas the AMD board was $800.

    A marketing person would call the system cost comparison ~$3k vs $~6k.

    Again, am I missing something?

    The AMD 6344 based 4S system has a ~19% clockspeed advantage and comes in at 62% of the cost.

    Worried about electrical cost? You can buy 700 watts of PV solar panels with the cost savings between AMD and Intel. 700 watts peak = ~3.5 kwh daily output. That would nicely mitigate (and then some) the 20 watts/cpu difference (80 watts total with 4x cpu) between AMD 6344 and Intel 4607. You would net about 1.5 kwh/day electricity going with the AMD + PV over the slightly more efficient Intel.
  • geok1ng - Monday, May 19, 2014 - link

    this is all and good, but looking at spec int 2006 results, Abu Dhabi still does not match performance/watt of Magni Cours opterons. And the best competition AMd can offer against 2 gen old Xeons is still the Opteron 6180.

Log in

Don't have an account? Sign up now