Power Consumption

The Magny-Cours Opteron arrived one week ago, which is barely enough time to do virtualization benchmarking. So we have to postpone extensive power testing to a later date. The Opteron 6174 came in a desktop reference system which is in no way comparable to our Xeon X5670 1U server. We do have an six-core Opteron based system which is very similar to the Opteron 6174 reference system: the motherboard is also equipped with the new AMD SR5670 chipset and housed in the same desktop system. We can tell you that the idle power of the Opteron 6174 is a few watts lower than the six-core Opteron 2435. Both throttle back to 800 MHz, but the Opteron 6100 series gets a real C1E mode.

C1E mode can only be entered if all CPUs are idle. In a dual socket system, both CPUs enter C1E or they don’t. C1E mode is entered only after longer periods of inactivity. All cores flush their L1 and L2 caches to the L3-cache. Then all cores are clockgated (C1). Once that happens, the Hyper Transport links are put in a lower power state. This allows the chipset to enter a lower power state as well. Only when all these previous steps are done, both sockets are in C1E. DMA events will make the sockets go out of the C1E state. So C1E probably won't happen much on server systems. The C1E state is only entered if absolutely no processing is happening at all.

The C1E mode can reduce power quite a bit:

  • Core clocks are turned off (Clockgate C1 state)
  • L3, North Bridge, and memory controller all divide their clock frequencies (but are not clockgated!)
  • All HyperTranspor links transition to LS2 low power state (LDT_STOP_L)
  • DRAM DLL’s disabled
  • Memory Transitions from precharge power down mode to self refresh mode (low power)

According to AMD, at full load a 1.7GHz 65W ACP Opteron 6164 HEwould consume about 4% more power than a 2.1 GHz 55W ACP 6-core Opteron 2425 HE. AMD measured 225W for  the former, 215W for the latter. We measured 263W on the same system at full load with an Opteron 6174. That's 48W more, or about 24W per CPU. Assuming that the low power CPUs were running at their ACP (65W), we can conclude that the 2.2 GHz Magny-Cours needs about 89W. While the new twelve-core Opteron clearly needs a bit more power than the six-core Opteron, it's not a dramatic increase.

HPC and Encryption Benchmarks Final Words
Comments Locked

58 Comments

View All Comments

  • wolfman3k5 - Monday, March 29, 2010 - link

    Great review! Thanks for the review, when will you guys be reviewing the AMD Phenom II X6 for us mere mortals? I wonder how the Phenom II X6 will stack up against the Core i7 920/930.

    Keep up the good work!
  • ash9 - Tuesday, March 30, 2010 - link

    Since SSE4.1,SSE4.2 are not in AMD's , its Andand's way of getting an easy benchmark win, seeing some of these benchmark test probably use them-

    http://blogs.zdnet.com/Ou/?p=719
    August 31st, 2007
    SSE extension wars heat up between Intel and AMD

    "Microprocessors take approximately five years to go from concept to product and there is no way Intel can add SSE5 to their Nehalem product and AMD can’t add SSE4 to their first-generation 45nm CPU “Shanghai” or their second-generation 45nm “Bulldozer” CPU even if they wanted to. AMD has stated that they will implement SSE4 following the introduction of SSE5 but declined to give a timeline for when this will happen."

    asH
  • mariush - Tuesday, March 30, 2010 - link

    One of the best optimized and multi threaded applications out there is the open source video encoder x264.

    Would it be possible to test how well 2 x 8 and 2x12 amd configurations work at encoding 1080p video at some very high quality settings?

    A workstation with 24 cores from AMD would cost almost as much as a single socket 6 cores system from Intel so it would be interesting to see if the increase in frequency and the additional SSE instructions would be more advantage than the number of cores.
  • Aclough - Tuesday, March 30, 2010 - link

    I wonder if the difference between the Windows and Linux test results is related to the recentish changes in the scheduler? From what I understand the introduction of the CFS in 2.6.23 was supposed to be really good for large numbers of cores, and I'm given to understand that before that the Linux scheduler worked similarly to the recent Windows one. It would be interesting to try running that benchmark with a 2.6.22 kernel or one with the old O(1) patched in.

    Or it could just be that Linux tends to be more tuned for throughput whereas Windows tends to be more tuned for low latency. Or both.
  • Aclough - Tuesday, March 30, 2010 - link

    In any event, the place I work for is a Linux shop and our workload is probably most similar to Blender, so we're probably going to continue to buy AMD.
  • ash9 - Tuesday, March 30, 2010 - link

    http://www.egenera.com/pdf/oracle_benchmarks.pdf


    "Performance testing on the Egenera BladeFrame system has demonstrated that the platform
    is capable of delivering high throughput from multiple servers using Oracle Real Application
    Clusters (RAC) database software. Analysis using Oracle’s Swingbench demonstration tool
    and the Calling Circle schema has shown very high transactions-per-minute performance
    from single-node implementations with dual-core, 4-socket SMP servers based on Intel and
    AMD architectures running a 64-bit-extension Linux operating system. Furthermore, results
    demonstrated 92 percent scalability on either server type up to at least 10 servers.
    The BladeFrame’s architecture naturally provides a host of benefits over other platforms
    in terms of manageability, server consolidation and high availability for Oracle RAC."
  • nexox - Tuesday, March 30, 2010 - link

    It could also be that Linux has a NUMA-aware scheduler, so it'd try to keep data stored in ram which is connected to the core that's running the thread which needs to access the data. I probably didn't explain that too well, but it'd cut down on memory latency because it would minimize going out over the HT links to fetch data. I doubt that Windows does this, given that Intel hasn't had NUMA systems for very long yet.

    I sort of like to see more Linux benchmarks, since that's really all I'd ever consider running on data center-class hardware like this, and since apparently Linux performance has very little to do with Windows performance, based on that one test.
  • yasbane - Wednesday, May 19, 2010 - link

    Agreed. I do find it disappointing that they put so few benchmarks for Linux for servers, and so many for windows.

    -C
  • jbsturgeon - Tuesday, March 30, 2010 - link

    I like the review and enjoyed reading it. I can't help but feel the benchmarks are less a comparison of CPU's and more a study on how well the apps can be threaded as well as the implementation of that threading -- higher clocked cpus will be better for serial code and more cores will win for apps that are well threaded. In scientific number crunching (the code I write ), more cores always wins (AMD). We do use Fluent too, so thanks for including those benchamarks!!
  • jbsturgeon - Tuesday, March 30, 2010 - link

    Obviously that rule can be altered by a killer memory bus :-).

Log in

Don't have an account? Sign up now