Turning our eyes back towards SC13 and supercomputers one more time this week, we have the biannual supercomputer rankings, which are released every spring and every fall. These rankings are composed of the Top500 and Green500, which rank supercomputers based on total performance and power efficiency respectively. These lists are admittedly mostly for bragging rights, but they do give us an interesting look at what the top computers in the world currently look like.

Top500 Top 6 Supercomputers
Supercomputer Architecture Performance (Rmax, TFLOPS) Power Consumption (kW) Efficiency (MFLOPS/W)
Tianhe-2 Xeon + Xeon Phi 33862.7 17808 1901.5
Titan Opteron + Tesla 17590.0 8209 2142.7
Sequoia BlueGene/Q 17173.2 7890 2176.5
K Computer SPARC64 10510.0 12660 830.1
Mira BlueGene/Q 8586.6 3945 2176.5
Piz Daint Xeon + Tesla 6271.0 1753 3185.9

Starting with the Top500, the top of the list remains unchanged from what we saw in June. Further supercomputers have come online in the last 6 months, but none large enough in scale to compete with the top supercomputers. This means that the Xeon + Xeon Phi powered Tianhe-2 remains as the top supercomputer in the world, offering a staggering 33.8 PFLOPS of performance, almost twice that of the next fastest computer. This also continues Intel’s rule of the top of the list, with the Xeon powered supercomputer having now held the top for two iterations of the list.

Moving down the list, we have the Opteron + Tesla based Titan, the Bluegene/Q based Sequoia and Mira, and the SPARC64 based K Computer. The first new supercomputer on the list comes in at #6, and that’s the Swiss National Computing Center’s Piz Daint. Piz Daint was first announced back in March and is now fully up and running, based on a Cray XC30 which in turn is a combined Xeon + Tesla system. Piz Daint weighs in at 6.2 PFLOPS, thereby making it the most powerful supercomputer in Europe. This also marks the second Tesla K20 system to make the top 10, tying Intel’s pair of Xeon Phi systems but still well behind the 4 Bluegene/Q systems that occupy the top of the list.

Also released this week was the Green500 list, which is a derivation of the Top500 list sorted by power efficiency instead of total performance. Since the Green500 list isn’t decided on raw performance it often sees a much higher rate of turnover than the Top500 list, as smaller scale computers can enter the top of the list if they’re efficient enough.

Green500 Top 5 Supercomputers
Supercomputer Architecture Performance (Rmax, TFLOPS) Power Consumption (kW) Efficiency (MFLOPS/W)
TSUBAME-KFC Xeon + Tesla 150.4 27.7 4503.1
Wilkes Xeon + Tesla 239.9 52.6 3631.8
HA-PACS TCA Xeon + Tesla 277.1 78.7 3517.8
Piz Daint Xeon + Tesla 6271.0 1753.6 3185.9
Romeo Xeon + Tesla 254.9 81.4 3130.9

This November has seen exactly the above happen, with a number of smaller installations coming online and pushing older systems further down the list. For June you needed an efficiency level of 2,299 MFLOPS/W to make the top 10; for November you need 2358 MFLOPS/W. Which doesn’t sound like a huge difference, but it was enough to push systems 4 through 10 off of the top 10 list, meaning most of the list is composed of new supercomputers.

Interestingly, every new system on the list is some kind Xeon + Tesla K20 configuration, which in turn has led to the entire top 10 now being composed of those systems. In fact you have to go down past #11 – the Xeon + FirePro S10000 based SANAM computer – to find a system that isn’t GPU powered. The Green500 list is based on total power consumption – system plus cooling – so it isn’t just a measurement of system power efficiency, but except in the cases of Eurora, TSUBAME-KFC, and other computers that are using exotic cooling, everyone else is using standard air conditioning and as such system power efficiency is the deciding factor. To that end it’s clear that from an efficiency perspective GPUs have the market cornered, edging out both traditional CPU setups and even Xeon Phi setups (as GPU-like as they are).

Looking at some of the individual systems, the new champion of the Green500 is an interesting and admittedly small scale system out of Japan called TSUBAME-KFC. At 27.78 kW in power consumption and 150 TFLOPS, TSUBAME-KFC isn’t very powerful, but at 4,503 MFLOPS/W it’s by far the most efficient system on the list. The secret – and to our knowledge this is the only ranked supercomputer like this – is that TSUBAME-KFC is the only oil immersion supercomputer on the list. As we’ve seen in the past with systems such as Eurora, supercomputer builders are playing with increasingly exotic forms of cooling in order to keep power consumption down and to further increase efficiency, as the largest scale installations are bounded more by power/cooling needs than they are equipment costs.

TSUBAME-KFC (Kepler Fluid Cooling, we’re told) takes this idea to the extreme by using non-conductive oil rather than air as the heat exchange medium. Oil doesn't have quite the immense heat capacity of water, but it allows for full immersion of the system nodes rather than requiring the use of tubing and water blocks. Coupled with system level optimizations undertaken by the system builders, such as turning down the GPU clockspeeds for efficiency rather than absolute performance, the builders were able to get the system up to 4,503 MFLOPS/W.

At 25% more efficient than the next closest supercomputer, TSUBAME-KFC is probably a safe bet for holding the top spot for another iteration. Any significant move on the list will require equally exotic cooling, or more likely a newer installation using newer processors, which is admittedly just a matter of time. Whether a traditional air cooled system can beat that mark remains to be seen, but we certainly expect all the usual parties – including the new IBM/NVIDIA partnership – to be potential contenders as their new processors and new manufacturing processes come online.

POST A COMMENT

28 Comments

View All Comments

  • MrSpadge - Saturday, November 23, 2013 - link

    That's the heat conductivity you're talking about. Don't know the numbers for such oils, though. Reply
  • ShieTar - Monday, November 25, 2013 - link

    Why would there be more mass? Most mineral oils have a lower density than water. Its actually the heat capacity per volume that makes water the best heat storage system. Reply
  • ddriver - Saturday, November 23, 2013 - link

    Got to love AT's eye-poking bias, I've been used to "top 5" or "top 10", but top 6? Now that is a curious choice. I bet it was intended as a top 5 chart, just like the second, but was expanded to 6 just to squeeze one more intel reference inside, god forbid AT put a "top 5" chart where intel is outmatched by ibm chips and has only one intel entry, just as much as amd. For the second chart 5 entries seem to suffice, probably because all systems are intel.

    Yeah, keep calling yourself "the most trusted" ... Articles without in-your-face brutally obvious bias at AT are becoming as rare as a hen's teeth. There is not even remote subtlety...
    Reply
  • Ryan Smith - Saturday, November 23, 2013 - link

    It would have been top 5, but since the only new supercomputer in the larger top 10 was Piz Daint at #6, I stretched it by one so that we could list those numbers and talk about it. Vulcan, SuperMUC, JUQUEEN, and Stampede were all online in June. Or would you rather we ignore the most powerful supercomputer in Europe? Reply
  • CPU-geek - Saturday, November 23, 2013 - link

    Is anandtech planning to write about Micron's new automata processor. Seems like a very interesting technology. Reply
  • SteelRing - Monday, November 25, 2013 - link

    They need to develop a special coolant liquid that is efficient at heat conduction yet has electrical insulation property. Most material that conducts heat does it by conducting electron, so this is super tricky. But if this is available, I can't see why all computer systems won't be submerged cooled from then on. Reply
  • dew111 - Monday, November 25, 2013 - link

    " In fact you have to go down past #11 – the Xeon + FirePro S10000 based SANAM computer – to find a system that isn’t GPU powered."

    Isn't the FirePro S10000 a GPU? It's not nvidia, which is probably what you meant, but it is definitely a GPU.
    Reply
  • dew111 - Monday, November 25, 2013 - link

    Oh, I see what you mean, number 11 is GPU powered, and number 12 isn't. But that's a bit confusing, as you mention the GPU powered on and not the first non-GPU powered supercomputer. Reply

Log in

Don't have an account? Sign up now