vApus Mark I: Performance-Critical Applications Virtualized

Our vApus Mark I benchmark is not a VMmark replacement. It is meant to be complimentary: while VMmark uses runs 60 to 120 light loads, vApus Mark I runs 8 heavy VMs on 24 virtual CPUs (vCPUs). Our current vApus Stressclient is being improved to scale to much higher amount of vCPUs, but currently we limit the benchmark to 24 virtual CPUs.

A vApus Mark I tile consists of one OLTP, one OLAP and two heavy websites are combined in one tile. These are the kind of demanding applications that still got their own dedicated and natively running machine a year ago. vApus Mark I shows what will happen if you virtualize them. If you want to fully understand our benchmark methodology: vApus Mark I has been described in great detail here. We have changed only one thing compared to our original benchmarking: we used large pages as it is generally considered as a best practice (with RVI, EPT).

The current vApus Mark I uses two tiles. Per tile we have 4 VMs with 4 server applications:

  • A SQL Server 2008 x64 database running on Windows 2008 64-bit, stress tested by our in-house developed vApus test (4 vCPUs).
  • Two heavy duty MCS eFMS portals running PHP, IIS on Windows 2003 R2, stress tested by our in house developed vApus test (each 2 vCPUs).
  • One OLTP database, based on Oracle 10G Calling Circle benchmark of Dominic Giles (4 vCPUs).

The beauty is that vApus (stress testing software developed by the Sizing Servers Lab) uses actions made by real people (as can be seen in logs) to stress test the VMs, not some benchmarking algorithm.

Update: We have noticed that the CPU load of Magny-cours is at 70-85%, while the Six-core "Istanbul" is running at 80-95%". As we have noted before, 24 cores is at the limit of our current benchmark until we launch vApus Mark 2. We have reason to believe that the opteron 6174 has quite a bit of headroom left. The results above are not wrong, but do not show the full potential of the 6174. We are checking the CPU load numbers of the six-core Xeon X5670 as we speak. Expect an update in the coming days.

vAPUS Mark I 2 tile test - 24 vCPUs - ESX 4.0

The AMD Opteron 6174 performs well here, but disappoints a bit at the same time. vApus Mark I does not scale as well as VMmark. The reason is simple: as we used 4 virtual CPUs for both the OLTP as the OLAP virtual machine, scaling depends more on the individual applications. One VM with 4 virtual CPUs will not scale as well as 16 VMs sharing the same 4 virtual CPUs. Also, we use heavy database applications that typically like a decent amount of cache. The difference with the Xeon X5670 is small though. Servers based on both CPUs will make excellent virtualization platforms.

Next, the same test with Hyper-V, the hypervisor beneath Windows 2008 R2. We are testing with Hyper-V R2 6.1.7600.16385 (21st of July 2009).

vAPUS Mark I 2 tile test - 24 vCPUs - Hyper-V

Based on the excellent results of the Dual Opteron 2435 we expected AMD to take the crown in this benchmark, but that did not happen. We only had one week to get all of the Opteron testing done (AMD didn't have any hardware until the last minute), so we could not analyze this in depth. For some reason, the Opteron 6174 does not scale very well in our vApus benchmark. Compared to a 2.2GHz six-core, we only see a 30% increase in performance, about the same as Intel gets out of adding 2 extra cores to their Xeon. Part of the reason might be our benchmark: at the moment we are limited to 24 CPUs. We’ll investigate this in more detail in the coming quarter when vApus v2 is available.

The difference with the Xeon X5670 is small though, and the slightly lower price of the Opteron makes up for the slightly lower performance.  

Virtualization & Consolidation HPC and Encryption Benchmarks
Comments Locked

58 Comments

View All Comments

  • kokotko - Saturday, April 24, 2010 - link

    why you are NOT SHARIG same "shareable" components - like PSU ??????

    NO WONDER THE NUMBERS ARE WORSE ! ! !
  • blurian589 - Tuesday, May 11, 2010 - link

    3ds max crashes because of the mental ray renderer. remove the plugin from loading and max will start up. its due to mental ray cannot see more than 16 threads (physical or virtual via hyper-threading). please do test the max rendering performance. thanks
  • Desired_Username - Tuesday, June 29, 2010 - link

    In the final words it states "We estimate that the new Opteron 6174 is about 20% slower than the Xeon 5670 in virtualized servers with very high VM counts. " But in the virtualization section I can't seem to figure out what brought you to that conclusion. The VMmark scores for the Cisco X5680 system was 35.83@26 tiles. You have the VMmark for the 6176SE at 31 which is dead on to the HP DL385 G7 which got 30.96@22 tiles. I see the X5680 15% better at best. And the Cisco x5680 system had 192GB of memory to the HP 6176SE system had 128GB. What am I missing here?
  • jeffjeff - Wednesday, September 22, 2010 - link

    I appreciate AMD's lower CPU cost but on the other hand, Oracle will license me their RDBMS per core and whether it's an Intel 56xx or AMD 61xx, I am still paying a relation of .5 license per core.

    So in the end, I would pay 6 cores for AMD and 3 cores for Intel. The price per core is much higher than the hardware price difference.

    Any thought or solutions on this issue would be appreciated...

    Joffrey
  • stealthy - Wednesday, November 24, 2010 - link

    Would it be possible to get the xml parameter files you have used in this test ?
    We are currently in a trial phase at my company to see how the current crop of intel boxes (dual Xeon X5460 procs) hold up against a new z10 system.
    Did you run the swingbench on the server itself or did you use a dedicated client to test ?
  • Big_Mr_Mac - Thursday, December 16, 2010 - link

    In 1991 I had an AMD 386-40 that kicked the snot out of Intel pride and joy 486DX2-66. Benchmarks were 25%+ across the board over Intel. Then Intel lied to the market and started passing off cull processors as viable options calling the 25Mhz and 50Mhz processors, when they were actually processors that failed the benchmarks for 33Mhz and 66Mhz respectively.

    In 1998 When Win98 Beta was released I was building Servers and workstations at a Tech-company and Again the AMD was kicking the snot out of Intel. Load times on new system builds, boot time and performance. The Intel chips could not hack it. Then when MS release their actual market version of Win98...all of a sudden you could not even use an AMD processor to run it. You had to wait 2 weeks for MS come up with a "AMD Patch" to run on an AMD system.

    One think I have seen over 20 years in the industry is that Intel will, Lie, Cheat, Steal and Bribe to try and get the upper hand on AMD. Always have....Always Will!!
  • rautamiekka - Saturday, December 25, 2010 - link

    Why the fuck are you testing with WinServer and M$ SQL ? Just reading this makes my blood boil 9 times in a second.
  • polbel - Saturday, May 21, 2011 - link

    i've been an amd fan for as long as i can remember. started fixing computers in 1979. used to fix mai basic four minis in the mid-80s that were built on amd bit-slice bipolar cpus on boards that cost 15,000$.

    just got 2 opteron 6172 cpus from ebay for what i thought was peanuts (450 $ each) only to discover upon delivery that both had hairline cracks at a 45 degree angle on one corner of the contact pad surface. looking at their web site i could figure i was out on limb and they would laugh in my face if asked for warranty support on these not-boxed cpus. i know some dumb ass managed to break those cpu corners, and tried to shove the crap to an ebay sucker, but the problem lies deeper, mostly in the g34 socket physical design itself of these otherwise beautiful electronic products. the edge of the metal cover doesn't reach the edge of the fiber board, leaving some unsupported area to be broken by dumb asses mimicking the old days when they could put a 40-pin dip cpu upside-down in its socket. so i'm freshly reviewing my belief system about amd while i figure a solution for this crap-hits-the-fan situation. wish i could have told amd engineers to cover theses last millimeters at the bleeding edge. they might say this and that about warranty, i still hold them responsible for this preventable disaster.

    paul :-)

Log in

Don't have an account? Sign up now