Low Power Server CPUs: the energy saving choice?
by Johan De Gelas on July 15, 2010 4:54 AM EST- Posted in
- IT Computing
How useful are low power server CPUs?
We were quite a bit surprised that the lower power CPU did not return any significant energy savings compared to the X5670. Intuition tells you that the best server CPUs like the X5670 only would pay off in a high performance environment (for example an HPC server). But human intuition is a bad guide when dealing with physics. Cold hard measurements are a much better way to make up your mind. And all our measurements point in the same direction: the fastest Xeon offers a superior performance/watt ratio in a typical virtualization consolidation scenario.
You could argue that the X5670 is quite a bit more expensive; a server equipped with a dual X5670 will indeed cost you about $900 more. We admit: our comparison is not completely accurate price wise… as always we work with the CPUs that we got in the lab. But a typical server with these CPUs, 64 GB and some accessories will set you back $9000 easily. The 2.8 GHz Xeon X5560 is hardly 4% slower than the X5670, and will probably show the same favorable performance/watt ratio. And if you place a X5560 2.8 GHz instead of a L5640 2.26 GHz, you only add $200 dollar to a $8k-$9k server. That is peanuts, lost in the noise of the TCO calculation. So the question is real: are the "Low power Xeons" (L-series) useless and should you immediately go for the X-series?
Defeated as it may be, the L5640 can still play one trump card: lower maximum currents. Over the period of our 70 minutes of testing, we decided to take a look at maximum power. To avoid that any extreme peaks would muddle up the picture, we used the 95th percentile.
Let us focus on the “balanced” power plan. The L5640 makes sure power never goes beyond 231 W, while the peak of the X5670 is almost 20% higher. As a result, a rack of low power Xeon will be able to keep the maximum current consumed lower. You could consider the low power Xeon L5640 a “power capped” version of Xeon X5670. In many datacenters you pay a hefty fine if you briefly need more than the amp limit you have agreed upon. So the low power Xeon might save you money by guaranteeing that you never need more than a certain amperage.
Facebook
Twitter
RSS
49 Comments
View All Comments
WillR - Thursday, July 15, 2010 - link
10p per kWh may be low in the UK but it's not for residential in the US. $.12/kWh is average. Just pulled out a bill from earlier this year and we paid 7.3 cents per kWh at my house. What it comes down to is the data center potentially overcharges people in the, using your numbers, 19 to 38 cents per kWh range, but rates can be higher than $.20/kWh in high density areas like NYC or SF. The extra costs should go to paying for upgrades and expansion of their infrastructure so it's not unreasonable.Worth mentioning to put in perspective is 4 250 watt servers uses 720kWh/month and the average house in the US uses 920kWh/month, so it's not really as simple a setup as one might initially think.
http://www.eia.doe.gov/cneaf/electricity/esr/table... provides a nice table of average rates and usages.
knedle - Thursday, July 15, 2010 - link
I'm not sure if you're aware, but in most countries residential has much (at least twice) lower price per 1kWh, than commercial. Also commercial pays extra for using electricity during the day, and gets electricity cheaper during night.This is why there are some factories in Europe that work only during night.
WillR - Thursday, July 15, 2010 - link
That is not the case in the US. Residential pays higher rates than either Commercial or Industrial users.http://www.eia.doe.gov/cneaf/electricity/epm/table...
Average Retail Price (Cents/kWh)
Items Mar-10 Mar-09
Residential 11.2 11.33
Commercial 10.03 10.07
Industrial 6.5 6.79
Industrial settings tend to use very large amounts of energy in a very small area or number of clients so they get cheaper bulk rates for purchasing a lot with little administrative overhead. It's also often the case they can get a high voltage line installed directly to the plant which is expensive to install but increases efficiency dramatically.
These averages may reflect heavy use of off-peak consumption, but most plants I've experienced operate 24/7. Much of it is politics and bargaining for a better rate on the contract.
DaveSylvia - Thursday, July 15, 2010 - link
Hey Johan, great article as usual! Always enjoyed and appreciated your articles including those from back in the day at Ace's Hardware!JohanAnandtech - Thursday, July 15, 2010 - link
Good memory :-). I have been part of Anand's team for 6 years now, that is the same amount of time that I spend at Ace's.DaveSylvia - Thursday, July 15, 2010 - link
Yeah! One of the first tech articles I recall reading was back in 1999. It was about how pipeline length influenced CPU clock speeds. You used Pentium II, K6, DEC Alpha's as examples :). All good stuff!MrSpadge - Thursday, July 15, 2010 - link
Isn't there a power plan which lets the CPUs turbo up (as max performance does) and also lets them clock down if not needed (as balanced does)? It seems outright stupid to take turbo away from a Nehalem-like chip.
MrS
has407 - Thursday, July 15, 2010 - link
No reason you shouldn't be able to specify a power plan that does both, but for whatever reason it isn't provided out-of-the-box.I'd guess that given the relatively small difference in idle power between "performance" and "balanced" (which seems to be more of "power capped" plan), maybe they (presumably the OEM?) decided it wasn't worth it.
There may also be stability issues with some system configurations or support concerns, as there's yet another set of variables to deal with.
has407 - Thursday, July 15, 2010 - link
Johan -- That brings up an interesting question: How much of the underlying CPU's power management are you testing vs. a particular vendor or OS configuration? I'd expect them to closely reflect each other assuming everyone has their job.As you're using Win 2008, it would be interesting to see what powercfg.exe shows for the various parameters for different modes and systems; e.g., "Busy Adjust Threshold", "Increase Policy", "Time Check", "Increase Percent", "Domain Accounting Policy" etc. Are there significant differences across systems/CPUs for the same profile?
Whizzard9992 - Thursday, July 15, 2010 - link
Heat dissipation is also a concern, no? It's expensive to cool a datacenter. Low power should bring cooling costs down.There's also a question of density. You can fit more low-power cores into 1U of space because of the heat dissipation. Multi-node blades are cheaper than 2U workhorses. Rack space is expensive for a lot of reasons. Just look at the Atom HPC servers: I bet the Atom would score pretty low in performance-per-watt versus even the LP XEON, but its sheer size and thermal envelope fit it in places the XEON can't.
Frankly, I'd be surprised if the low-power XEON saved "energy" at the same workloads versus full-power, given that both are on the same architecture. LP XEONs are really an architecture choice, and greasing the transition to many-cores via horizontal scaling. A good desktop analogy would be, "Is one super-fast core better than a slower multi-core?" Fortunately for the datacenter most servers only need one or the other.
Also, with physical nodes scaling out horizontally, entire nodes can be powered down during down times, with significant power savings. This is software-bound, and I haven't seen this in action yet, but it's a direction nonetheless.
Without getting into all of the details, I think a proper TCO analysis is in order. This article seems to really only touch on the actual power-consumption, where there are really no surprises. The full-power peaks performance a little better, and the LP stays within a tighter thermal-envelope.
The value of low-power is really low-heat in the datacenter. I'd like to see something that covers node density and cooling costs as well. A datacenter with all LP-servers is unrealistic, seeing as how some applications that scale vertically will dictate higher-performing processors. It would be nice to see what the total cost would be for, say a 2,000 node data center with 80% LP population versus 20% LP population. The TDP suggests a 1/3 drop in cooling costs and 1/3 better density.