Facebook's "Open Compute" Server tested
by Johan De Gelas on November 3, 2011 12:00 AM ESTBenchmark Configuration
HP Proliant DL380 G7
| CPU | Two Intel Xeon X5650 at 2.66 GHz |
| RAM | 6 x 4GB Kingston DDR3-1333 FB372D3D4P13C9ED1 |
| Motherboard | HP proprietary |
| Chipset | Intel 5520 |
| BIOS version | P67 |
| PSU | 2 x HP PS-2461-1C-LF 460W HE |
We have three servers to test. The first is our own standard off-the-shelf server, and HP DL380G7. This server is the natural challenger for the Facebook design, as it is one of the most popular and efficient general purpose servers.
As this server is targeted at a very broad public, it cannot be as lean and mean as the Open Compute servers.
Facebook's Open Compute Xeon version
| CPU | Two Intel Xeon X5650 at 2.66 GHz |
| RAM | 6 x 4GB Kingston DDR3-1333 FB372D3D4P13C9ED1 |
| Motherboard | Quanta Xeon Opencompute 1.0 |
| Chipset | Intel 5500 Rev 22 |
| BIOS version | F02_3A16 |
| PSU | Power-One SPAFCBK-01G 450W |
The Open Compute Xeon server is configured as close to our HP DL380 G7 as possible.
Facebook's Open Compute AMD version
| CPU | Two AMD Opteron Magny-Cour 6128 HE at 2.0 GHz |
| RAM | 6 x 4GB Kingston DDR3-1333 FB372D3D4P13C9ED1 |
| Motherboard | Quanta AMD Open Compute 1.0 |
| Chipset | |
| BIOS version | F01_3A07 |
| PSU | Power-One SPAFCBK-01G 450W |
The benchmark numbers of the AMD Open Compute server are only included for your information. There is no direct comparison possible with the other two systems. The AMD system is better equipped than the Intel, as it has more DIMM slots and uses HE CPUs.
Common Storage system
Each server has an adaptec 5085 PCIe 8x (driver aacraid v1.1-5.1[2459] b 469512) connecting to six Cheetah 300GB 15000 RPM SAS disks in a Promise JBOD J300s.
Software configuration
VMware ESXi 5.0.0 (b 469512 - VMkernel SMP build-348481 Jan-12-2011 x86_64). All vmdks use thick provisioning, independent, and persistent. Power policy is Balanced Power.
Other notes
Both servers were fed by a standard European 230V (16 Amps max.) powerline. The room temperature was monitored and kept at 23°C.
Facebook
Twitter
RSS
67 Comments
View All Comments
JohanAnandtech - Thursday, November 3, 2011 - link
It is an ATI ES 1000, that is a server/thin client chip. That chip is only 2D. I can not find the power specs, but considering that the chip does not even need a heatsink, I think this chip consumes maybe 1W in idle.mczak - Thursday, November 3, 2011 - link
ES 1000 is almost the same as radeon 7000/ve (no that's not HD 7000...) (some time in the past you could even force 3d in linux with the open-source driver though it usually did not work). The chip also has dedicated ram chip (though only 16bit wide memory interface) and I'm not sure how good the powersaving methods of it are (probably not downclocking but supporting clock gating) - not sure if it fits into 1W at idle (but certainly shouldn't be much more).JohanAnandtech - Thursday, November 3, 2011 - link
I can not find any good tech resources on the chip, but I can imagine that AMD/ATI have shrunk the chip since it's appearance in 2005. If not, and the chip does consume quite a bit, it is a bit disappointing that server vendors still use it as the videochip is used very rarely. You don't need a videochip for RDP for example.mczak - Thursday, November 3, 2011 - link
I think the possibility of this chip being shrunk since 2005 is 0%. The other question is if it was shrunk from rv100 or if it's actually the same - even if it was shrunk it probably was a well mature process like 130nm in 2005 otherwise it's 180nm.At 130nm (estimated below 20 million transistors) the die size would be very small already and probably wouldn't get any smaller due to i/o anyway. Most of the power draw might be due to i/o too so shrink wouldn't help there neither. It is possible though it's really below 1W (when idle).
Taft12 - Thursday, November 3, 2011 - link
A shrink WOULD allow production of many more units on each wafer. Since almost every server shipped needs an ES1000 chip, demand is consistently on the order of millions per year.mczak - Thursday, November 3, 2011 - link
There's a limit how much i/o you can have for a given die size (actually the limiting factor is not area but circumference so making it rectangular sort of helps). i/o pads apparently don't shrink well hence if your chip has to have some size because you've got too many i/o pads a shrink will do nothing but make it more expensive (since smaller process nodes are generally more expensive per area).Being i/o bound is quite possible for some chips though I don't know if this one really is - it's got at least display outputs, 16bit memory interface, 32bit pci interface and the required power/ground pads at least.
In any case even at 180nm the chip should be below 40mm² already hence the die size cost is probably quite low compared to packaging, cost of memory etc.
Penti - Saturday, November 5, 2011 - link
It's the integrated BMC/ILO solution which also includes a GPU that would use more power then the ES1000 any how. That is also what is lacking in the simple Google / Facebook compute-node setup. They don't need that kind off management and can handle that a node goes offline.haplo602 - Thursday, November 3, 2011 - link
It seems to me that the HP server is doing as well as the Facebook ones. Considering it has more featuers (remote management, integrated graphics) and a "common" PSU.JohanAnandtech - Thursday, November 3, 2011 - link
The HP does well. However, if you don't need integrated graphics and you hardly use the BMC at all, you still end up with a server that wastes power on features you hardly use.twhittet - Thursday, November 3, 2011 - link
I would assume cost is also a major factor. Why pay for so many features you don't need? Manufacturing costs should be lower if they actually build these in bulk.