Introducing Our Open Virtualization Benchmark
vApus Mark II has been our own virtualization benchmark suite that tests how well servers cope with virtualizing "heavy duty applications". We explained the benchmark methodology here. The beauty of vApus Mark II is that:
- We test with real-world applications used in enterprises all over the world
- We can measure response times
- It can scale from 8 to 80 thread servers
- It is lightweight on the client side: one humble client is enough to bring the most massive server to its knees. For a virtualizated server or cluster, you only need a few clients.
There is one big disadvantage, however: the OLAP and web applications are the intellectual property of several software vendors, so we can't let third parties verify our tests. To deal with this, the Sizing Servers Lab developed a new benchmark, called vApus For Open Source workloads, in short vApus FOS.
vApus FOS uses a similar methodology as vApus Mark II with "tiles". The exact software configuration may still change a bit as we tested with the 0.9 version. One vApus FOS 0.9 tile uses four different VMs, consisting of:
- A PhpBB (Apache2, MySQL) website with one virtual CPU and 1GB RAM. The website uses about 8GB of disk space. We simulate up to 50 concurrent uses with press keys every 0.6 to 2.4 s.
- The same VM but with two vCPUs.
- An OLAP MySQL database that is used by an online webshop. The VM gets two vCPUs and 1GB RAM. The database is about 1GB, with up to 500 connections active.
- Last but not least: the Zimbra VM. VMware's open source groupware offering is by far the most I/O intensive VM. This VM gets two vCPUs and 2GB RAM, with up to 100 concurrent users active.
All VMs are based on minimal CentOS 5.6 with VMware Tools installed. vApus FOS can also be run on different hypervisors: we already tried using KVM, but encountered a lot of KVM specific problems.
Other than that, nice post!