Hardware Configuration

We have reviewed the Intel Xeon X5670 before: it is the best performing Intel Six-core in the 95W TDP power envelope. For comparison, we add the Intel Xeon L5640. The 32 nm “Westmere”L5640 reduces TDP to 60W, although it still has 6 cores. This chip runs at 2.26 GHz, but at lighter load it should boost itself to 2.8 GHz.

Test server
Asus RS700-E6/RS4 1U Server
Asus Z8PS-D12-1U Motherboard
Six-core Xeon L5640 2.26 GHz or Six-core Xeon X5670 2.93 GHz
6x Samsung M393B5170DZ1 - CH9 1333MHz CL9 ECC (24GB)
2x Western Digital WD1000FYPS 1TB (VM images and OS installation)
2 x Intel X25-E SLC SSD 32GB (Data Oracle OLTP & Log Oracle OLTP)

BIOS
Most Important BIOS Settings: (BIOS version 0701 (20/01/2010))

C1E Support: Enabled
Hardware Prefetcher: Enable
Adjacent Cache Line Prefetch: Enabled
Intel VT: Enabled
Active Intel SpeedStep Tech: Enabled
Intel TurboMode: Enable
Intel C-State Tech:Enabled
C3 State: ACPI C3

Software configuration:

Windows 2008 R2 Enterprise, Hyper-V enabled
vApus Mark I softwareas described here.

Current measurements

We used the racktivity PM0816-ZB datacenter PDU to measure power.

 

Using a PDU for accurate power measurements might same pretty insane, but this is not your average PDU. Measurement circuits of most PDUs assume that the incoming AC is a perfect sine wave, but it never is. The Rackitivity PDU measures however true RMS current and voltage at a very high sample rate: up to 20.000 measurements per seconds for the complete PDU. We read out the current and voltage out each second, which already gives us more than 4000 data points along our 70 minutes long virtualization power test. As the PDU has 8 ports, this allows us to test several servers at once, which will be very handy for future reviews. 

Where is AMD’s Opteron?
We did not manage to get a decent server based on the latest AMD’s Opterons in the lab. The current “Magny-Cours” Servers in our lab are reference motherboards running in a desktop tower. So to avoid any unfair comparison with our Xeon rack servers we delay our measurements on the AMD platform until we find a way to get  a real server in the lab.

The new methodology Average power and energy
Comments Locked

49 Comments

View All Comments

  • duploxxx - Friday, July 16, 2010 - link

    It is a very nice test, but always those tests are done with extreme low or extreme high bins while the mass isn't buying these parts, In the field you rather see huge piles of for example E5620-30 series and X5650, result differences should be interesting to see or at least for once provide an idea about the scaling in portfolio. (I know this is a difficult one)

    Although It is very interesting to see the differences within vendors or between vendors there is one major thing that is missing in all these tests. This review will provide any interested IT a good overview what he could do with power consumption and the performance but what is left out here is the big influence from OEM.

    OEM have there own powersavings/regulators/dynamics that influence the server a lot both in OS and BIOS, even often in a very bad way. So while it is an interesting article most IT will never get the result they wanted due to the OEM implementation.
  • indiamap - Friday, July 16, 2010 - link

    I do agree 100% with the author. Johan is very much clear on what he is more concerned about, the electricity bill. It’s very much true that electricity bills eat up large amount of income of <a href="http://www.indiamaphosting.com/">Internet Marketing India</a> companies and it’s a huge burden from business point of view. This can be tackled by opting for better hardware's which is compatible with green energy and helps in cutting the electricity bill extensively and saves huge sum of money for the organization in the long run.
  • indiamap - Friday, July 16, 2010 - link

    This is what the internet companies were looking for. Cost Cutting. It is very essential from business POV. Lesser the energy consumption, more the investment on product research. Great numbers exposed. Thanks a lot Johan. I wonder this is why Google started electricity generating stations to power its massive data centers.

    For more information, please visit: http://www.indiamaphosting.com/
  • mino - Friday, July 16, 2010 - link

    Johan - great article.

    Keep it up !
  • Whizzard9992 - Monday, July 19, 2010 - link

    Can we please clean up the spam here? Where's the "REPORT" button?
  • Toadster - Friday, July 16, 2010 - link

    Given the results you've found it would be great to see how power capping can influence the workloads as well. The latest DELL Poweredge-C platforms support Intel Intelligent Power Node Manager technology - would be fantastic to have a look!

    Keep up the great work - excellent article!
  • Casper42 - Saturday, July 17, 2010 - link

    So first off I have to say that using a home built Machine with an Asus mobo and trying to talk intelligently about Datacenter Power is not really a fair comparison. Stick to the big 3 (HP/Dell/IBM) when doing these kinds of comparisons.

    Now the real reason I posted was because you mentioned the speed of the memory you used, but made NO mention of the speed the memory was actually running at.

    With the Nehalem and Westmere Xeons, only the X series can run the memory at 1333 while the others (E and L series) start at 1066. When you run more than one bank of memory you can also see your memory frequency decline depending on the server vendor you are using. I think HP has a bit you can flip in their machines that will allow you to run 2 banks @ 1333 (again, assuming X Series proc) but if you don't turn that on, you step down from 1 bank @ 1333 to 2 @ 1066 and even 3 @ 800.

    The reason I bring this up is because you said yourself your machine was NOT CPU bound, and you weren't entirely sure why the tests completed with such different times. Well memory performance could be part of that equation.

    Lastly, you have to remember that not every server in a DC is running VMWare/HyperV and there are still tons of servers with basic Windows or Linux App/Web workloads running right on the hardware. These kind of servers on average in the industry run less than 10% of the CPU max in a given day (might be spikes for backups and other jobs, but the average is <10%)
    So if you had a rack with 20 2U Servers and you didn't need VMWare/SQL/Oracle level performance in those racks, why not run them with L series processors and across an entire rack you are saving a decent amount of power.

    PS: Where are you guys at AT located? Your "About Us" button up top has been useless for quite some time now. Not sure it could be pulled off, but you should really look into asking the big 3 for demo gear. Getting a Nehalem EX right now is damn near impossible but a Westmere EP would be doable. The problem here is they do loaners to get sales, not to get reviews, so what you really need to do is find some friends who work in IT at very large companies in your area who would be willing to let you get some wrench time on their demo equipment. 60-90 day loans are quite common.

    -Casper42
  • tjohn46 - Tuesday, July 20, 2010 - link

    I'm surprised I haven't seen anyone else make a similar comment yet: I've been curious about this for a long time, but I would rather see a comparison between 2 CPUs that are intended to be competitive.

    It looks Intel changed things a bit with 5600 series xeons, but previously (including with the 5500's) intel would match up model numbers, cores, and clock speeds. The model with the 'L' prefix would just have a lower TDP. I was always curious if those performed just as well or not?

    For example:

    E5520 vs L5520
    E5530 vs L5530

    I see AMD also has some 80W and 65W comparable models if you do end up testing opterons.

    That would be the real "is it worth the processor price premium?" question in my opinion. Of course the high end part which doesn't have a comparable "low power" model is going to perform better.. but like someone else said, a typical data center tends to have many more of the midrange parts (like an E5530) installed which also have lower power conterparts at a $200ish premium.
  • eva2000 - Saturday, July 31, 2010 - link

    interesting to see how they compare when it comes to linux OS i.e. centos 5.5 or redhat 5.5 :)

Log in

Don't have an account? Sign up now