Benchmark Configuration

All tests were done on Ubuntu Server 14.04 LTS. We tested the HP Moonshot remotely. Our special thanks goes out to the team of HP EMEA Moonshot Discovery Lab of Grenoble (France). We tested both the Supermicro MicroCloud and the different motherboard configurations in our lab.

ASRock's C2750D4I

CPU Intel Atom C2750
RAM 4x 8GB DDR3 @1600 or
4x 16GB DDR3 @1333 (Intelligent Memory)
Internal Disks 1x Intel MLC SSD710 200GB
Motherboard ASRock C2750D4I
PSU Supermicro PWS-502 (80+)

Intel's Xeon E3-1200 v3 – ASUS P9D-MH

CPU Intel Xeon processor E3-1240 v3
Intel Xeon processor E3-1230L v3
RAM 4x 8GB DDR3 @1600
Internal Disks 1x Intel MLC SSD710 200GB
Motherboard ASUS P9D-MH
PSU Supermicro PWS-502 (80+)

Intel's Xeon E3-1200 v2

CPU Intel Xeon processor E3-1220 v2
Intel Xeon processor E3-1265L v2
RAM 4x 8GB DDR3 @1600
Internal Disks 1x Intel MLC SSD710 200GB
Motherboard Intel S1200BTL
PSU Supermicro PWS-502 (80+)

Supermicro's MicroCloud SYS-5038ML-H8TRF

We enabled four nodes, each with an Intel Xeon E3-1230L v3. Each node was configured with:

CPU Intel Xeon processor E3-1230L v3
RAM 4x 8GB DDR3 @1600
Internal Disks 1x Intel MLC SSD710 200GB
Motherboard Super X10SLD-F
PSU Dual Supermicro PWS-1K62P-1R (1.6 KW, 80+ Platinum) for 4 nodes

We first tested with only one PSU, but that did not work out as the firmware kept all Xeons at their minimum clock speed of 800 MHz. Only with both PSUs active were the Xeon able to use all their p-states. Supermicro confirmed that four active nodes should be enough to make the PSU run efficiently.

HP Moonshot

We tested two different cartridges: the m400 and the m300. Below you can find the specs of the m400:

CPU/SoC AppliedMicro X-Gene 2.4
RAM 8x 8GB DDR3 @ 1600
Internal Disks m2 2280 Solid State 120GB
Cartridge m400

And the m300:

CPU/SoC Atom C2750 2.4
RAM 8x 8GB DDR3 @ 1600
Internal Disks m2 2280 Solid State 120GB
Cartridge m300

Other Notes

Both servers are fed by a standard European 230V (16 Amps max.) power line. The room temperature is monitored and kept at 23°C by our Airwell CRACs. We use the Racktivity ES1008 Energy Switch PDU to measure power consumption in our lab. We used the HP Moonshot ILO to measure the power consumption of the cartridges.

Quick Overview of the SoCs Bandwidth
Comments Locked

47 Comments

View All Comments

  • JohanAnandtech - Tuesday, March 10, 2015 - link

    Thanks! It is been a long journey to get all the necessary tests done on different pieces of hardware and it is definitely not complete, but at least we were able to quantify a lot of paper specs. (25 W TDP of Xeon E3, 20W Atom, X-Gene performance etc.)
  • enzotiger - Tuesday, March 10, 2015 - link

    SeaMicro focused on density, capacity, and bandwidth.

    How did you come to that statement? Have you ever benchmark (or even play with) any SeaMicro server? What capacity or bandwidth are you referring to? Are you aware of their plan down the road? Did you read AMD's Q4 earning report?

    BTW, AMD doesn't call their server as micro-server anymore. They use the term dense server.
  • Peculiar - Tuesday, March 10, 2015 - link

    Johan, I would also like to congratulate you on a well written and thorough examination of subject matter that is not widely evaluated.

    That being said, I do have some questions concerning the performance/watt calculations. Mainly, I'm concerned as to why you are adding the idle power of the CPUs in order to obtain the "Power SoC" value. The Power Delta should take into account the difference between the load power and the idle power and therefore you should end up with the power consumed by the CPU in isolation. I can see why you would add in the chipset power since some of the devices are SoCs and do no require a chipset and some are not. However, I do not understand the methodology in adding the idle power back into the Delta value. It seems that you are adding the load power of the CPU to the idle power of the CPU and that is partially why you have the conclusion that they are exceeding their TDPs (not to mention the fact that the chipset should have its own TDP separate from the CPU).

    Also, if one were to get nit picky on the power measurements, it is unclear if the load power measurement is peak, average, or both. I would assume that the power consumed by the CPUs may not be constant since you state that "the website load is a very bumpy curve with very short peaks of high CPU load and lots of lows." If possible, it may be more beneficial to measure the energy consumed over the duration of the test.
  • JohanAnandtech - Wednesday, March 11, 2015 - link

    Thanks for the encouragement. About your concerns about the perf/watt calculations. Power delta = average power (high web load measured at 95% percentile = 1 s, an average of about 2 minutes) - idle power. Since idle power = total idle of node, it contains also the idle power of the SoC. So you must add it to get the power of the SoC. If you still have doubts, feel free to mail me.
  • jdvorak - Friday, March 13, 2015 - link

    The approach looks absolutely sound to me. The idle power will be drawn in any case, so it makes sense to add it in the calculation. Perhaps it would also be interesting to compare the power consumed by the differents systems at the same load levels, such as 100 req/s, 200 req/s, ... (clearly, some higher loads will not be achievable by all of them).

    Johan, thanks a lot for this excellent, very informative article! I can imagine how much work has gone into it.
  • nafhan - Wednesday, March 11, 2015 - link

    If these had 10gbit - instead of gbit - NICs, these things could do some interesting stuff with virtual SANs. I'd feel hesitant shuttling storage data over my primary network connection without some additional speed, though.

    Looking at that moonshot machine, for instance: 45 x 480 SSD's is a decent sized little SAN in a box if you could share most of that storage amongst the whole moonshot cluster.

    Anyway, with all the stuff happening in the virtual SAN space, I'm sure someone is working on that.
  • Casper42 - Wednesday, April 15, 2015 - link

    Johan, do you have a full Moonshot 1500 chassis for your testing? Or are you using a PONK?

Log in

Don't have an account? Sign up now