Infrastructure as a Service: Benchmarking Cloud Computing
by Johan De Gelas on June 2, 2011 8:50 PM EST- Posted in
- IT Computing
- IT Computing general
- Cloud Computing
Benchmarking the Terremark Cloud
We wanted to compare the virtual IaaS servers of the Terremark Enterprise Cloud with a virtualized physical server, because that is the decision you will have to make: will you deploy your application on a server in your own local data center, or will you deploy to a virtual server in an IaaS environment?
The "In House" Reference Machine
Nowadays most applications find a home inside a virtual machine on top of a hypervisor. Since Terremark servers have Intel's Xeon 7500s inside, we decided to use a reference machine based on the same platform. We used the QSSC-4R machine, equipped with four Xeon X7560 CPUs running at 2.26GHz. We ran vSphere 4.1 Update 1, basedpon the 64-bit ESX 4.1.0 b348481 hypervisor on top of this server.
| CPU | 4x Xeon X7560 at 2.26GHz |
| RAM | 16x4GB Samsung Registered DDR3-1333 at 1066MHz |
| Motherboard | QCI QSSC-S4R 31S4RMB00B0 |
| Chipset | Intel 7500 |
| BIOS version | QSSC-S4R.QCI.01.00.S012,031420111618 |
| PSU | 4x Delta DPS-850FB A S3F E62433-004 850W |
Typically, a group of virtual machines share the CPU, memory and storage resources that have been allocated to their "resource pool", so we tested the "in house" machine in two ways. In the first benchmark run, virtual machines were only limited by the amount of virtual CPUs they were given. The one OLAP virtual machine got to eight virtual CPUs, and our three web servers each got two virtual CPUs. With sixteen total CPU cores, that means the OLAP machine is able to use up to eight physical CPUs and each web server is able to use two physical CPUs.
In the second benchmark setup, we limited the virtual machines (14 virtual CPUs in total) to a resource pool of 10GHz of CPU power. This is similar to the Terremark setup (as well as other "cloud" setups), which also use resource pools to make optimal use of the underlying hardware. After all, it is costly to reserve hardware resources if they are not being used.
The Terremark Virtual Server Infrastructure
We reserved 5GHz (10GHz limit) of CPU power, 10GB of RAM, and 215GB of storage space in the Terremark Enterprise Cloud. We tested this IaaS cloud in two ways. First, we disabled the burst function, which means that we are limited to a maximum of 10GHz of CPU power. Second, we enabled the burst function. In that case, the Terremark Infrastructure will offer extra CPU power, but the amount of processing power that will be made available to your server depends on how heavy the Terremark cluster is currently loaded. Terremark guarantees that in all circumstances 20% extra resources are available, and during our tests we saw up to 24GHz was made available to us.
Facebook
Twitter
RSS
29 Comments
View All Comments
benwilber - Friday, June 3, 2011 - link
this is a joke, right?there is not one bit of useful information in this article. if i wanted to read a Terremark brochure, i'd call our sales rep.
speaking as an avid reader for more than 12 years, it's my opinion that all these braindead IT virtualization articles are poorly conceived and not worthy of anandtech.
krazyderek - Friday, June 3, 2011 - link
submit a better one thenDigitalFreak - Friday, June 3, 2011 - link
I guess it's a good thing then that your opinion doesn't matter.HMTK - Monday, June 6, 2011 - link
Yeah, I also prefer yet another vidcard benchmark fest.Not.
Shadowmaster625 - Friday, June 3, 2011 - link
Still waiting for that $100 tablet that can provide me a remote desktop that is so responsive you cant even tell it is a remote desktop. I want it to be able to stream video at 480p. With good compression, this only requires a 1 mbps connection. I dont think this is too much to ask for $100. I dont care that much about HD. Streaming a desktop at 30 fps should only require a small fraction of my bandwidth.tech6 - Friday, June 3, 2011 - link
As you mentioned, Terremark cloud benchmarks vary greatly depending on the underlying hardware. We did some tests on their Miami cloud last year and found the old AMD infrastructure to be a disappointing performer. The software is very clever but, like all clouds, some benchmarking and asking the right questions is essential before making a choice.duploxxx - Sunday, June 5, 2011 - link
as usual this is very debatable information you provide. How did you bench and what storage platform? what is your compare a 2008 vs 2010? What kind of application did you bench? Specint? :) Just like Anandtech has greatly shown in the past is that appplications performance can be influenced by the type of cpu (look at the web results within the vApp that is clearly showing it likes faster cache architecture and to certain extend influences the final vApp result to much) you need to look at the whole environment and applications running in the environment, this requires decent tools to benchmark your total platform. (We have more code written by dev to automaticaly test any functional and performance aspect then the applications by themselves) everything in a virtual layer can influence the final performance.Our company has from 2005 till now always verified the platforms between intel and AMD on virtualization every 2 and 4 socket machine. Currently approx 3000 AMD servers on line all on Vmware private clusters from many generations. They are doing more then fine. The only timeframe that the Intel was faster and a better choice was just at launch time of the Nehalem Xeon for a few months. Offcourse one also need to look at the usecase for example the latest Xeon EX is very interesting with huge amount of small vm's, but requires way more infrastructure to handle for example load and the failure of a server. (Not to mention license cost from some 3th party vendors like Oracle.....)
lenghui - Friday, June 3, 2011 - link
A very well thought-out comparison betwen the in-house and IaaS environments. Even those who have the in-house resources would need to spend a lot of research time to reach a conclusion. In that sense, your review is most invaluable -- saving hundreds of hours or otherwise guess work for your readers. You probably can include a price analysis as the other readers have suggested.Thanks, Johan, for the great article.
brian2p98 - Friday, June 3, 2011 - link
This is, imo, the biggest unknown with cloud computing--and the most critical. Poor performance here could result in degradation of performance on the scale of several orders of magnitude. Website hosting, otoh, is rather straightforward. Who cares if 5Ghz of cloud cpu power is equivalent to only 1Ghz of local, so long as buying 25Ghz still makes economic sense?duploxxx - Sunday, June 5, 2011 - link
depends on how good or bad your app can scale with cpu cores.....if it doesn't and you need more vm's to handle the same load you also need other systems to spread the load between apps.