Measuring Real-World Power Consumption, Part One

The Equal Workload (EWL) version of vApus FOS is very similar to our previous vApus Mark II "Real-world Power" test. To create a real-world “equal workload” scenario, we throttle the number of users in each VM to a point where you typically get somewhere between 20% and 80% CPU load on a modern dual CPU server. The amount of requests is the same for each system, hence "equal workload".

The CPU Load on the Opteron 6276 looked like this:

The CPU load is typically around 30-50%, with peaks up to 65%. At the end of the test, we get to a low 10%, which is ideal for the machine to boost to higher CPU clocks (Turbo) and race to idle. First we check out the response times.

vApus FOS Response times (ms)
CPU PhpBB1 PHPBB2 MySQL OLAP Zimbra
AMD Opteron 6276 134 47 3.6 44
AMD Opteron 6174 118 41 3.8 45
Intel Xeon X5670 76 27 2.2 28

ESXi and our Interlagos "Opteron" probably don't understand each other fully, given the newness of the architecture. Some extensive monitoring with ESXtop shows that the lower CPU load is spread among all the cores, and the result is that the Opteron 6276 never reaches its highest clock speed (3.2GHz). That helps make the response times significantly higher than on the Xeon, although they are acceptable. Again, the Interlagos Opteron fails to really beat the "Magny-cours" Opteron.

Our main focus of this benchmark is of course energy consumption.

vApus FOS EWL Power consumption

The Xeon consumes 25% less power, and the older Opteron about 10% less. The performance/Watt ratio of the newest Opteron looks rather bad when running on top of ESX. We shall delve into this deeper in the next several pages.

Virtualization Performance: Linux VMs on ESXi Virtualization Performance: ESX + Windows
Comments Locked

106 Comments

View All Comments

  • veri745 - Tuesday, November 15, 2011 - link

    Shouldn't there be 8 x 2MB L2 for Interlagos instead of just 4x?
  • ClagMaster - Tuesday, November 15, 2011 - link

    A core this complex in my opinion has not been optimized to its fullest potential.

    Expect better performance when AMD introduces later steppings of this core with regard to power consumption and higher clock frequencies.

    I have seen this in earlier AMD and Intel Cores, this new core will be the same.
  • C300fans - Tuesday, November 15, 2011 - link

    1x i7 3960x or 2x Interlagos 6272? It is up to you. Money cow.
  • tech6 - Tuesday, November 15, 2011 - link

    We have a bunch of 6100 in our data center and the performance has been disappointing. They do no better in single thread performance than old 73xx series Xeons. While this is OK for non-interactive stuff, it really isn't good enough for much else. These results just seem to confirm that the Bulldozer series of processors is over-hyped and that AMD is in danger of becoming irrelevant in the server, mobile and desktop market.
  • mino - Wednesday, November 16, 2011 - link

    Actually, for interactive stuff (read VDI/Citrix/containers) core counts rule the roost.
  • duploxxx - Thursday, November 17, 2011 - link

    this is exactly what should be fixed now with the turbo when set correct, btw the 73xx series were not that bad on single thread performance, it was wide scale virtualization and IO throughput which was awefull one these systems.
  • alpha754293 - Tuesday, November 15, 2011 - link

    "Let us first discuss the virtualization scene, the most important market." Yea, I don't know about that.

    Considering that they've already shipped like some half-a-million cores to the leading supercomputers of the world; where some of them are doing major processor upgrades with this new release; I wouldn't necessarily say that it's the most IMPORTANT market. Important, yes. But MOST important...I dunno.

    Looking forward to more HPC benchmark results.

    Also, you might have to play with thread schedule/process affinity (masks) to make it work right.

    See the Techreport article.
  • JohanAnandtech - Thursday, November 17, 2011 - link

    Are you talking about the Euler3D benchmark?

    And yes, by any metric (revenue, servers sold) the virtualization market is the most important one for servers. Depending on the report 60 to 80% of the servers are bought to be virtualized.
  • alpha754293 - Tuesday, November 15, 2011 - link

    Folks: chip-multithreading (CMT) is nothing new.

    I would explain it this way: it is the physical, hardware manifestation of simultaneous multi-threading (SMT). Intel's HTT is SMT.

    IBM's POWER (since I think as early as POWER4), Sun/Oracle/UltraDense's Niagara (UltraSPARC T-series), maybe even some of the older Crays were all CMT. (Don't quote me on the Crays though. MIPS died before CMT came out. API WOULD have had it probably IF there had been an EV8).

    But the way I see it - remember what a CPU IS: it's a glorified calculator. Nothing else/more.

    So, if it can't calculate, then it doesn't really do much good. (And I've yet to see an entirely integer-only program).

    Doing integer math is fairly easy and straightforward. Doing floating-point math is a LOT harder. If you check the power consumption while solving a linear algebra equation using Gauss elimination (parallelized or using multiple instances of the solver); I can guarantee you that you will consume more power than if you were trying to run VMs.

    So the way I see it, if a CPU is a glorified calculator, then a "core" is where/whatever the FPU is. Everything else is just ancillary and that point.
  • mino - Wednesday, November 16, 2011 - link

    1) Power is NOT CMT, it allways was a VERY(even by RISC standards) wide SMT design.

    2) Niagara is NOT a CMT. It is interleaved multipthreading with SMT on top.

    Bulldozer indeed IS a first of its kind. With all the associated advantages(future scaling) and disadvantages(alfa version).

    There is a nice debate somewhere on cpu.arch groups from the original author(think 1990's) of the CMT concept.

Log in

Don't have an account? Sign up now