Benchmark Configuration

Since Supermicro claims that these servers are capable of operating at inlet temperatures of 47°C (117 °F) while supporting Xeons with 135 W TDPs, we tested with two extreme processors. First off is the Xeon E5 2650L at 1.8GHz with a low 70W TDP and a very low Tcase of 65°C. It's low power but is highly sensitive to high temperatures. Second, we tested with the fastest Xeon E5 available: the Xeon E5 2697 v2. The TDP is 130W for 12 cores at 2.7GHz and Tcase is 86°C. This is a CPU that needs a lot of power but it's also resistant to high temperatures.

Supermicro 6027R-73DARF (2U Chassis)

CPU Two Intel Xeon processor E5-2697 v2 (2.7GHz, 12c, 30MB L3, 130W)

Two Intel Xeon processor E5-2650L v2 (1.7GHz, 10c, 25MB L3, 70 W)
RAM 64GB (8x8GB) DDR3-1600 Samsung M393B1K70DH0-CK0
Internal Disks 8GB flash disk to boot up, 1 GbE link to iSCSI SAN
Motherboard Supermicro X9DRD-7LN4F
Chipset Intel C602J
BIOS version R 3.0a (December the 6th, 2013)
PSU Supermicro 740W PWS-741P-1R (80+ Platinum)

All C-states are enabled in both the BIOS and ESXi.

How We Tested Temperature vs. CPU Load
Comments Locked

48 Comments

View All Comments

  • lwatcdr - Thursday, February 20, 2014 - link

    Here in south florida it would probably be cheaper. The water table is very high and many wells are only 35 feet deep.
  • rrinker - Tuesday, February 11, 2014 - link

    It's been done already. I know I've seen it in an article on new data centers in one industry publication or another.
    A museum near me recently drilled dozens of wells under their parking lot for geothermal cooling of the building. Being large with lots of glass area, it got unbearably hot during the summer months. Now, while it isn't as cool as you might set your home air conditioning, it is quite comfortable even on the hottest days, and the only energy is for the water pumps and fans. Plus it's better for the exhibits, reducing the yearly variation in temperature and humidity. Definitely a feasible approach for a data center.
  • noeldillabough - Tuesday, February 11, 2014 - link

    I was actually talking about this today; the big cost for our data centers is Air Conditioning; what if we had a building up north (arctic) where the ground is alway frozen even in summer? Geothermal cooling for free, by pumping water through your "radiator".

    Not sure about the environmental impact this would do, but the emptiness that is the arctic might like a few data centers!
  • superflex - Wednesday, February 12, 2014 - link

    The enviroweenies would scream about you defrosting the permafrost.
    Some slug or bacteria might become endangered.
  • evonitzer - Sunday, February 23, 2014 - link

    Unfortunately, the cold areas are also devoid of people and therefore internet connections. You'll have to figure the cost of running fiber to your remote location, as well as how your distance might affect latency. If you go into permafrost area, there are additional complications as constructing on permafrost is a challenge. A datacenter high in the Mountains but close to population centers would seem a good compromise.
  • fluxtatic - Wednesday, February 12, 2014 - link

    I proposed this at work, but management stopped listening somewhere between me saying we'd need to put a trench through the warehouse floor to outside the building, and that I'd need a large, deep hole dug right next to building, where I would bury several hundred feet of copper pipe.

    I also considered using the river that's 20' from the office, but I'm not sure the city would like me pumping warm water into their river.
  • Varno - Tuesday, February 11, 2014 - link

    You seem to be reporting on the junction temperature which is reported by most measurement programs rather than the cast temperature that is impossible to measure directly without interfering with the results. How have you accounted for this in your testing?
  • JohanAnandtech - Tuesday, February 11, 2014 - link

    Do you mean case temperature? We did measure the outlet temperature, but it was significantly lower than Junction temperature. For the Xeon 2697 v2, it was 39-40 °C at 35°C inlet, 45°C at 40°C inlet.
  • Kristian Vättö - Tuesday, February 11, 2014 - link

    Google's usage of raw seawater for cooling of their data center in Hamina, Finland is pretty cool IMO. Given that the specific heat capacity of water is much higher than air's, it more efficient for cooling, especially in our climate where seawater is always relatively cold.
  • JohanAnandtech - Tuesday, February 11, 2014 - link

    I admit, I somewhat ignored the Scandinavian datacenters as "free cooling" is a bit obvious there. :-)

    I thought some readers would be surprised to find out that even in Sunny California free cooling is available most of the year.

Log in

Don't have an account? Sign up now