Introducing Our Open Virtualization Benchmark

vApus Mark II has been our own virtualization benchmark suite that tests how well servers cope with virtualizing "heavy duty applications". We explained the benchmark methodology here. The beauty of vApus Mark II is that:

- We test with real-world applications used in enterprises all over the world

- We can measure response times

- It can scale from 8 to 80 thread servers

- It is lightweight on the client side: one humble client is enough to bring the most massive server to its knees. For a virtualizated server or cluster, you only need a few clients.

There is one big disadvantage, however: the OLAP and web applications are the intellectual property of several software vendors, so we can't let third parties verify our tests. To deal with this, the Sizing Servers Lab developed a new benchmark, called vApus For Open Source workloads, in short vApus FOS.

vApus FOS uses a similar methodology as vApus Mark II with "tiles". The exact software configuration may still change a bit as we tested with the 0.9 version. One vApus FOS 0.9 tile uses four different VMs, consisting of:

- A PhpBB (Apache2, MySQL) website with one virtual CPU and 1GB RAM. The website uses about 8GB of disk space. We simulate up to 50 concurrent uses with press keys every 0.6 to 2.4 s.

- The same VM but with two vCPUs.

- An OLAP MySQL database that is used by an online webshop. The VM gets two vCPUs and 1GB RAM. The database is about 1GB, with up to 500 connections active.

- Last but not least: the Zimbra VM. VMware's open source groupware offering is by far the most I/O intensive VM. This VM gets two vCPUs and 2GB RAM, with up to 100 concurrent users active.

All VMs are based on minimal CentOS 5.6 with VMware Tools installed. vApus FOS can also be run on different hypervisors: we already tried using KVM, but encountered a lot of KVM specific problems.


Benchmark Configuration vApus FOS results
Comments Locked


View All Comments

  • harrkev - Thursday, November 3, 2011 - link

    You should look again at the sine-wave plots. Power factor has more to do with the phase of the current and not so much how much like a sine-wave it looks like.

    As an example, a purely capacitive or purely inductive load will have a perfect sine wave current (but completely out of phase with the voltage), but have a power factor very close to zero...

    So, those graphs do not really tell us much unless you actually crank the numbers to calculate the real power factor.
  • ezekiel68 - Thursday, November 3, 2011 - link

    On page 2:

    "The next piece in the Facebook puzzle is that the Open Source tools are Memcached."

    In fact, the tools are not memchached. Instead, software objects from the PHP/c++ stack, programmed by the engineers, are stored in Memcached. Side note - those in the know pronounce it "mem-cache-dee", emphasizing with the last syllable that it is a network daemon. (similar to how the DNS server "bind" is pronounced "bin-dee") So the next piece is Memcached, but the tools are not 'memcached'.
  • JohanAnandtech - Thursday, November 3, 2011 - link

    That is something that went wrong in the final editing by Jarred. Sorry about that and I feel bad about dragging Jarred into this, but unfortunately that is what happened. As you can see further, "Facebook mostly uses memcached to alleviate database load", I was not under the impression that the "Open Source tools are Memcached. " :-)
  • ezekiel68 - Thursday, November 3, 2011 - link

    I was pretty sure it was a mistake and I only mentioned it to have the blemish removed - I've been following and admiring your technical writing since the the early 2000s. Please keep on bringing us great server architecture pieces. Don't worry about Jarred, he's fine too. We all make mistakes.
  • Dug - Thursday, November 3, 2011 - link

    I'm curious what the cost would be on the servers compared to something like the HP.
  • Lucian Armasu - Thursday, November 3, 2011 - link

    According to SemiAccurate, Facebook is considering Calxeda's recently announced ARM servers, too. It could be a lot more efficient to run something like Facebook on those types of servers.
  • JohanAnandtech - Thursday, November 3, 2011 - link

    I personally doubt that very much. The memcached servers are hardly CPU intensive, but a 32 bit ARM processor will not fit the bill. Even when ARM will get 64 bit, it is safe to say that x86 will offer much more DIMM slots. It remains to be seen how the ratio Watt/ RAM cache will be. Until 64 bit ARMs arrive with quite a few memory channels: no go IMHO.

    And the processing intensive parts of the facebook architecture are going to be very slow on the ARMs.

    The funny thing about the ARM presentations is that they assume that virtualization does not exist in the x86 world. A 24 thread x86 CPU with 128 GB can maybe run 30-60 VMs on it, lowering the cost to something like 5-10W per VM. A 5W ARM server is probably not even capable of running one of those machines at a decent speed. You'll be faced with serious management overhead to deal with 30x more servers (or worse!), high response times (single thread performance take a huge dive!) just to save a bit on the power bill.

    As a general rule: if the Atom based servers have not made it to the shortlist yet, they sure are not going to replace it by ARM based ones.
  • tspacie - Thursday, November 3, 2011 - link

    The FaceBook servers take a higher line voltage for increased efficiency. What voltage was supplied to the HP server for these tests?
  • JohanAnandtech - Thursday, November 3, 2011 - link

    Both servers used 230V. I have added this to benchmark page (Thanks, good question). So in reality the Facebook server can consume slightly less.
  • Alex_Haddock - Thursday, November 3, 2011 - link

    TBH we'd position SL class servers for this kind of scenario rather than DL380G7 (which does have a DC power option btw) so not sure it is a relevant comparison. Though I understand using what is available to test.

Log in

Don't have an account? Sign up now