Measuring Real-World Power Consumption, Part Two

First, we wanted to check if the Interlagos power management problems were specific to ESXi. Therefore, we measured the power consumption when running Windows 2008 R2 SP1 x64. We set the power management policy to "Balanced" and "High Performance".

Windows Server 2008 R2 Idle Power consumption

Wow, that is a lot better! The core gating of the Bulldozer cores is first rate, as good as the Xeons of today. Idle power draw is a serious problem of the Opteron 6174: it is between 30 to 63% higher! So even if the ESX scheduler does not really understand how to handle the power management features of the "Bulldozer" Opteron, the question remains why the Opteron 6276 cannot even beat the Opteron 6174 when running idle in ESXi.

ESX 5.0

While I was testing the power consumption on Windows, my colleague Tijl Deneut dug up some interesting information about the ESX power manager. The Balanced Power policy (the default power policy for ESXi 5) is rather simple: it uses an algorithm that exploits only the processor’s P-states and C-state C0 and C1. So "Balanced" does not make very good use of the deeper sleep states. So we went for custom, which is the same as "Balanced" until you start to customize of course. We enabled the other C-states and things started to make sense.

ESX 5.0

After some tinkering, the Opteron 6276 does quite a bit better and saves 17W (10%). The Xeon reduces power consumption by 3W, and the Opteron 6174's less advanced power management is not able to save any more power. So enabling the C-states is an important way to improve the power consumption of the Opteron "Interlagos" with ESXi 5.

Virtualization Performance: ESX + Windows Power Management in Windows Server 2008 SP2
POST A COMMENT

106 Comments

View All Comments

  • veri745 - Tuesday, November 15, 2011 - link

    Shouldn't there be 8 x 2MB L2 for Interlagos instead of just 4x? Reply
  • ClagMaster - Tuesday, November 15, 2011 - link

    A core this complex in my opinion has not been optimized to its fullest potential.

    Expect better performance when AMD introduces later steppings of this core with regard to power consumption and higher clock frequencies.

    I have seen this in earlier AMD and Intel Cores, this new core will be the same.
    Reply
  • C300fans - Tuesday, November 15, 2011 - link

    1x i7 3960x or 2x Interlagos 6272? It is up to you. Money cow. Reply
  • tech6 - Tuesday, November 15, 2011 - link

    We have a bunch of 6100 in our data center and the performance has been disappointing. They do no better in single thread performance than old 73xx series Xeons. While this is OK for non-interactive stuff, it really isn't good enough for much else. These results just seem to confirm that the Bulldozer series of processors is over-hyped and that AMD is in danger of becoming irrelevant in the server, mobile and desktop market. Reply
  • mino - Wednesday, November 16, 2011 - link

    Actually, for interactive stuff (read VDI/Citrix/containers) core counts rule the roost. Reply
  • duploxxx - Thursday, November 17, 2011 - link

    this is exactly what should be fixed now with the turbo when set correct, btw the 73xx series were not that bad on single thread performance, it was wide scale virtualization and IO throughput which was awefull one these systems. Reply
  • alpha754293 - Tuesday, November 15, 2011 - link

    "Let us first discuss the virtualization scene, the most important market." Yea, I don't know about that.

    Considering that they've already shipped like some half-a-million cores to the leading supercomputers of the world; where some of them are doing major processor upgrades with this new release; I wouldn't necessarily say that it's the most IMPORTANT market. Important, yes. But MOST important...I dunno.

    Looking forward to more HPC benchmark results.

    Also, you might have to play with thread schedule/process affinity (masks) to make it work right.

    See the Techreport article.
    Reply
  • JohanAnandtech - Thursday, November 17, 2011 - link

    Are you talking about the Euler3D benchmark?

    And yes, by any metric (revenue, servers sold) the virtualization market is the most important one for servers. Depending on the report 60 to 80% of the servers are bought to be virtualized.
    Reply
  • alpha754293 - Tuesday, November 15, 2011 - link

    Folks: chip-multithreading (CMT) is nothing new.

    I would explain it this way: it is the physical, hardware manifestation of simultaneous multi-threading (SMT). Intel's HTT is SMT.

    IBM's POWER (since I think as early as POWER4), Sun/Oracle/UltraDense's Niagara (UltraSPARC T-series), maybe even some of the older Crays were all CMT. (Don't quote me on the Crays though. MIPS died before CMT came out. API WOULD have had it probably IF there had been an EV8).

    But the way I see it - remember what a CPU IS: it's a glorified calculator. Nothing else/more.

    So, if it can't calculate, then it doesn't really do much good. (And I've yet to see an entirely integer-only program).

    Doing integer math is fairly easy and straightforward. Doing floating-point math is a LOT harder. If you check the power consumption while solving a linear algebra equation using Gauss elimination (parallelized or using multiple instances of the solver); I can guarantee you that you will consume more power than if you were trying to run VMs.

    So the way I see it, if a CPU is a glorified calculator, then a "core" is where/whatever the FPU is. Everything else is just ancillary and that point.
    Reply
  • mino - Wednesday, November 16, 2011 - link

    1) Power is NOT CMT, it allways was a VERY(even by RISC standards) wide SMT design.

    2) Niagara is NOT a CMT. It is interleaved multipthreading with SMT on top.

    Bulldozer indeed IS a first of its kind. With all the associated advantages(future scaling) and disadvantages(alfa version).

    There is a nice debate somewhere on cpu.arch groups from the original author(think 1990's) of the CMT concept.
    Reply

Log in

Don't have an account? Sign up now