Introducing Our Open Virtualization Benchmark

vApus Mark II has been our own virtualization benchmark suite that tests how well servers cope with virtualizing "heavy duty applications". We explained the benchmark methodology here. The beauty of vApus Mark II is that:

- We test with real-world applications used in enterprises all over the world

- We can measure response times

- It can scale from 8 to 80 thread servers

- It is lightweight on the client side: one humble client is enough to bring the most massive server to its knees. For a virtualizated server or cluster, you only need a few clients.

There is one big disadvantage, however: the OLAP and web applications are the intellectual property of several software vendors, so we can't let third parties verify our tests. To deal with this, the Sizing Servers Lab developed a new benchmark, called vApus For Open Source workloads, in short vApus FOS.

vApus FOS uses a similar methodology as vApus Mark II with "tiles". The exact software configuration may still change a bit as we tested with the 0.9 version. One vApus FOS 0.9 tile uses four different VMs, consisting of:

- A PhpBB (Apache2, MySQL) website with one virtual CPU and 1GB RAM. The website uses about 8GB of disk space. We simulate up to 50 concurrent uses with press keys every 0.6 to 2.4 s.

- The same VM but with two vCPUs.

- An OLAP MySQL database that is used by an online webshop. The VM gets two vCPUs and 1GB RAM. The database is about 1GB, with up to 500 connections active.

- Last but not least: the Zimbra VM. VMware's open source groupware offering is by far the most I/O intensive VM. This VM gets two vCPUs and 2GB RAM, with up to 100 concurrent users active.

All VMs are based on minimal CentOS 5.6 with VMware Tools installed. vApus FOS can also be run on different hypervisors: we already tried using KVM, but encountered a lot of KVM specific problems.


Benchmark Configuration vApus FOS results
Comments Locked


View All Comments

  • fpsvash - Thursday, November 3, 2011 - link

    In the middle of the paragraph below the image caption, the sentence reads "...and offers better slightly better performance..."

    Other than that, nice post!
  • ahmetmy330 - Thursday, December 29, 2016 - link

    this is my university work about open servers. Thanks for post
  • InternetGeek - Thursday, November 3, 2011 - link

    It's interesting that no many players have taken a look at Open Compute.
  • alent1234 - Thursday, November 3, 2011 - link

    it's a solution for a specific workload. there are still a lot of workloads that require the traditional model of big database servers

    unlike your bank, facebook's noSQL is not ACID
  • FunBunny2 - Saturday, November 5, 2011 - link

    Well, yes a voice of reason. OTOH, the Facebook et al folks are convinced that their back to the COBOL era is the future. As if a toy application, albeit pervasive, is "innovation".
  • Sivar - Saturday, November 5, 2011 - link

    It's a little difficult to look at a comment about Facebook being a toy application and take it seriously. Yes, Facebook is not directly processing bank transactions on a Tandem, but their site is used to conduct business -- and is even the basis for many businesses, all over the world.

    Zynga, the company that makes a few annoying games for Facebook, is worth $15 -- more than Electronic Arts.

    Nearly every major online publisher, including Anandtech, uses their API for content distribution and often as the entire forum system for discussion of publications.

    The founder is the youngest billionaire in history.

    Calling theirs a toy application sounds like a Blockbuster customer calling Redbox a toy. It's denial of an obviously successful, large, powerful, innovative company because they don't do things "the old way."

    I suspect what matters more is that the business is executing flawlessly, the actual problems with data loss or other non-ACID compliant traditional issues are minimal, and that they are making enough money that Google and Microsoft are feel seriously threatened.

    One last thing -- if you really look into what ACID compliance means (and I know you did not specifically mention the acronymn, but replied to someone that did) none of the current major DBMS's are truly ACID compliant. It's too slow. Not Oracle. Not MSSQL. Not Greenplum. Not Teradata. None of them. They may be closer than NoSQL or the like, but then it's all about the right tool for the job, right?
  • Ceencee - Wednesday, November 9, 2011 - link

    This is true but ACID can be over-rated for many workloads. How many pieces of data HAVE to be consistent across the entire cluster to be valid? What about NoSQL with configurable consistency like Cassandra?

    NoSQL databases provide the holy grail of system growth which is horizontal scaling and this is no small thing for anyone who has worked with a very large RDBMS like ORACLE and implemented RAC to find it doesn't scale all that linearly for most workloads.
  • ahmetmy330 - Sunday, January 1, 2017 - link

    ctually i assume you might have actually stumble upon these dubious <a href=""&g... King cheats </a>programs and software packages downloads but are convinced us
  • ahmetmy330 - Thursday, January 12, 2017 - link
    I have told you
  • ac2 - Thursday, November 3, 2011 - link

    Wouldn't the presence of the graphics on the HP server account for the 32W idle load savings?

Log in

Don't have an account? Sign up now