Server Clash: DELL's Quad Opteron DELL R815 vs HP's DL380 G7 and SGI's Altix UV10
by Johan De Gelas on September 9, 2010 7:30 AM EST- Posted in
- IT Computing
- AMD
- Intel
- Xeon
- Opteron
Power Extremes: Idle and Full Load
Now that we have real OEM servers in the lab for all platforms, we can finally perform a decent power consumption comparison. All servers have 64GB and the disk configuration is also exactly the same (four SSDs). In the first test we report the power consumption running vApus Mark II, which means that the servers are working at 95-99% CPU load. Please note that although the CPU load is high, we are not measuring maximum power: you can attain higher power consumption numbers by using a floating point intensive benchmark such as Linpack. But for virtualized—mostly integer—workloads, this should be more or less the maximum power draw.
We test with redundant power supplies working. So the Dell R815 uses 1+1 1100W PSUs, the SQI Altix UV10 uses 2+2 850W PSU and the HP uses the 1+1 460W PSUs.
You might think that the four 850 W PSU (2+2) are a serious disadvantage for the SGI server, but they are an advantage. The DELL and HP servers split their load over the two PSUs, resulting in somewhat lower efficiency, while the redundant PSUs of the SGI server consume exactly … 0 Watt. The power distribution board of the SGI Altix UV 10 /QSSC-4R has a very “cool” feature called “cold redundancy”. Although the redundancy is fully functional, the 2 redundant PSUs do not consume anything until you pull the active PSUs out.
The Dell R815 consumes less than two HP DL380 G7s, so the performance/watt ratio is competitive with the dual Xeon platform and without any doubt superior to the quad Xeon platform. If you compare the R815 with two Opterons with the HP DL380, you will notice that the R815 server is very efficient . The dual Opteron is hardly consuming more than the HP dual Xeon server while it has an 1100W PSU (not ideal when you are consuming only 360W) and contains of course a slightly more complex quad socket board. The quad socket R815 server is thus very efficient as the difference with a dual socket Xeon server is minimal.
Comparing the dual with the quad Opteron 6174 power numbers, we notice a relatively high increase in power: 244 Watt. So for each Opteron that we add, we measure 122 W at the wall. This 122 W includes a few Watts of PSU losses, VRM and DIMM wiring losses. So the real power consumed by the processor is probably somewhere between 100 and 110W. Which is much closer to the TDP (115W) than the ACP (80W) of this CPU.
Idle power measurements are hardly relevant for consolidated virtualized environments but they are a very interesting point of reference.
As you can see, it is not only the 130W TDP Xeon 7560 that make the SGI Altix UV10/QSSR-4R consume so much. We measure 372W difference between idle and full load, which is about 93W per CPU. That is not a huge difference if you consider that the difference is 350W for the four Opterons, and 170W for the two Xeons 5670. The Xeon 7500 CPU is capable of power gating the cores and will not consume much in idle. So we may say that the difference is not made by the CPUs: all CPU will consume in the range of 90-110W.
The problem is the complexity of the server. The QSSC-4R/ SGI Altix UV 10 pays a price for its scalability and serviceability: the memory riser boards alone consume almost 20W per board. So eight memory boards can add up to 160W. Being able to power 11 PCIe cards means that your power budget grows even more as the complexity of the I/O board is higher and the engineers have to size the power supply for the use of many more I/O cards and memory. The result is that the performance ratio of the quad Xeon 7500 is rather mediocre: you need three times the power of an HP DL 380 G7 and you only get twice the performance. At idle, it is even worse.
The Opteron 6174 needs a bit more power than its 80W ACP tag promises, but the performance/Watt ratio is very good, on par with the HP DL 380 G7. You need almost two HP DL380 G7 to achieve the same performance, but the DELL R815 needs 10% less power than two HP DL380 G7. So the DELL R815 is definitely a match for two DL380 G7s in the performance/watt category. And it beats two HP DL380 G7 with a healthy margin in other departments: CAPEX ($14000 with 128GB versus 2 x $9000 with 64GB), OPEX costs (only one machine to set up and manage), and rack space (2U vs 4U).
But… maximum power and minimum power are not very realistic. How about a real world scenario?
Facebook
Twitter
RSS
51 Comments
View All Comments
JohanAnandtech - Friday, September 10, 2010 - link
Thanks, appreciate you took the time to let us know. We went through 5 weeks of intensive testing and my eyes still hurt from looking at the countless excel sheets, with endless power and response time readings. ;-)FourthLiver - Thursday, September 9, 2010 - link
at the end of page 12, you allude to a performance per watt analysis. looks like you forgot to put it up. i'm chomping at the bit to see those numbers!please disregard me if i failed to rtfa correctly. Anandtech is the best; your (all of you collectively) articles are brilliant and correct down to the smallest details. This is another article that was an absolute joy to read. :]
JohanAnandtech - Thursday, September 9, 2010 - link
Well you can't really calculate it, as it depends on the situation. On low load loads, the system that consumes the less, is the winner, on the condition that the response times stay low. But of course, if your systems are running at low load all time, there might be something wrong: you should have bought more RAM and consolidated more VMs per system.At higher loads, the power consumption at high load divided by the throughput (vApusmark) is close to the truth. But it is definitely not the performance/watt number for everyone
It depends on your workloads. The more critical processing power (think response time SLA) is, the more the last mentioned calculation makes sense. The more we are talking about lots of lightly loaded VMs (like authentification servers, fileservers etc.), the more simply looking at the energy consumed at page 12 make sense.
mino - Thursday, September 9, 2010 - link
First, congratulations to a great article !Now to the small ammount of mess in there:
"the CPUs consume more than the ACP ratings that AMD mentions everywhere"
1) Avegare CPU Power (ACP) is NOT supposed/marketed to represent 100% load power use
Wikipedia: "The average CPU power (ACP), is a scheme to characterize power consumption of new central processing units under "average" daily usage..."
2) 122W at the wall and 110W at the CPU ??? Are you telling us the PSU's are 95% along with VRM/power/fans at 95% efficiency ? (0.95*0.95*1.22=1.10)
. Sorry to spoil the party but that is NOT the case. 122W at wall means 100W at CPU at the most realistically 95W.
Otherwise a great work. Keep is up!
JohanAnandtech - Friday, September 10, 2010 - link
"1) Avegare CPU Power (ACP) is NOT supposed/marketed to represent 100% load power useWikipedia: "The average CPU power (ACP), is a scheme to characterize power consumption of new central processing units under "average" daily usage...""
You are right. But what value does it have? As an admin I want to know what the maximum could be realistically (TDP is the absolute maximum for non-micro periods) and if you read between the lines that is more or less what AMD communicated (see their white paper). if it is purely "average", it has no meaning, because average power can be a quite a bit lower as some servers will run at 30% on average, others at 60%.
These PSU are supposed to be 92-94% efficient and AFAIK the VRMs are at least 90% efficient. So 122 x 0.92 x 0.90 = 101 W.
mino - Saturday, September 11, 2010 - link
Well, I was bit unslept when writing it but anyway. So got a bit harser than should have.In my experience the ACP values pretty well represent your average loaded server (<= 80% load). But that is not the point.
AMD created ACP in a response to the fact that their TDP numbers are conservative while Intel's are optimistic. That was the main cause wery well known to you as well.
Call me an ass but I certainly do not remember AT bitching about Intel TDPs no bein representative (during last 6 years at least).
And we all know too well that those NEVER represented the real power use of their boxen nor did they EVER represented what the "TDP" moniker stands for.
Currently the situation is as such that identical 2P AMD box with 80W ACP has ~ the same power requirements as 2P Intel box with 80W TDP. You have just proven that.
Therefore I believe it would be fair to stop bitching about AMD (or Intel) cheating in marketing (both do) and just say whether the numbers are comparable or not.
Arguing about spin wattage is not really needed.
JohanAnandtech - Monday, September 13, 2010 - link
"Arguing about spin wattage is not really needed. "I have to disagree. The usual slogan is "don't look at TDP, look at measurements". What measurments? The totally unrealistic SPECpower numbers?
It is impossible for review sites to test all CPUs. So it is up to vendors to gives us a number that does not have to be accurate on a few percent, but that let us select CPUs quickly.
Customers should have one number that allows them to calculate worst case numbers which are realistic (heavily load webserver for example, not a thermal virus). So all CPU vendors should agree on a standard. That is not bitching, but is a real need of the sysadmins out there.
mino - Thursday, September 9, 2010 - link
One thing I would love to see is having the lowest end HP server put to its paces.So far it seems to us a the best option for vCenter hosting in small environments (with FT Vm's hosting vCenter).
Maybe even run 1-tile vAPUS (v1? perhaps) on it ?
m3rdpwr - Thursday, September 9, 2010 - link
I would have prepared to have had the DL385 G7 compared.They can be had with 8 and 12 core CPU's.
We have close to 200 HP servers of all models, rack and blades.
Many running vm in our Data Center.
-Mario
duploxxx - Friday, September 10, 2010 - link
same here, we moved also to 385g7 with the new 8-12core cpu's, Nice servers with huge core count since we never run more vCPU then pCPU in a system. Dell 815 looks like a good solution also, it was mentioned in the review the BL685 and DL585 are way more expensive.