• What
    is this?
    You've landed on the AMD Portal on AnandTech. This section is sponsored by AMD. It features a collection of all of our independent AMD content, as well as Tweets & News from AMD directly. AMD will also be running a couple of huge giveaways here so check back for those.
    PRESENTED BY
POST A COMMENT

28 Comments

Back to Article

  • IanCutress - Friday, November 22, 2013 - link

    The problem with oil is replacing components, a key element of large installations that might suffer a failure a day. Replacing an oily component means you smell like bad fries afterwards. Or maybe it's another sort of oil. Any info on which hydrocarbon being used and the specific heat capacity? :) Reply
  • Ryan Smith - Friday, November 22, 2013 - link

    ExxonMobil SpectraSyn Polyalphaolefins (PAO) Reply
  • IanCutress - Friday, November 22, 2013 - link

    As far as I can tell, they're C4 to C22 alkenes that are polymerised and further hydrogenated to alkanes. Patented chemicals are sometimes the hardest to find info of, and I suspect that these are a range of PAOs rather than one single type. Reply
  • The Von Matrices - Friday, November 22, 2013 - link

    If the oil is pure alkanes and alkenes, it shouldn't have any smell at all. The aromatics and sulfurous compounds are the ones that have an odor. I work with FT Diesel in the lab, which this basically is, and there is no odor to it either. The petroleum Diesel with sulfur compounds and aromatics is the one that has the odor.

    I would think the biggest issue with replacing components is keeping the oil from spreading to everything in the room as components are moved around and replaced.
    Reply
  • IanCutress - Friday, November 22, 2013 - link

    Yes, the smell was a small bit of comedy :) I would imagine that if any sulphurous (natch, UK spelling ;]) links were present they'd attach pretty quickly to any exposed gold (excess pins etc) due to the low binding barrier therein.

    The name PAO is pretty generic, ExxonMobil have several SpectraSyn SKUs, and the datasheets on most of them had them as pretty viscous as far as I could tell. That wouldn't do much in terms of moving the oils around; I'd imagine there's a big-ass pump for each unit so push it through with fans along the way.
    Reply
  • lwatcdr - Saturday, November 30, 2013 - link

    My guess is that they remove the heat sink and keep the cooling loop intact when they swap parts. Reply
  • errorr - Friday, November 22, 2013 - link

    The data sheet I saw claimed it was white mineral oil (your dielectric alkenes) with their own patented additive mixture (usually fluorocarbons) and I would assume some antioxidant to prevent free radical breakdown of the alkenes.

    They claim there is no evaporation and that the liquid need never be topped off or raplaced. The sheet also said the liquid is pumped through a heat exchanger where it uses tap water (hot or cold) to cool the system.
    Reply
  • DanNeely - Friday, November 22, 2013 - link

    In large scale installations the minimum size of a swap out module is much larger than in your home computer. I don't have any specific information about super computer installs; but it's probably similar to how huge internet company data centers work. Ex When MS went to containerized data centers their minimum swapout unit was an entire shipping container of computers. Individual computers failing within one were just disabled via admin tools.

    This question on ServerFault was asking specifically about storage capacity; but the answers focused on the general infrastructure that the disks were part of:
    http://serverfault.com/q/535049/34835
    Reply
  • nafhan - Friday, November 22, 2013 - link

    Judging by the diagram above, the "swap out module" might not be much larger than a typical blade server, as it appears to be a self contained unit consisting of 2 CPU's, 4 GPU's and associated system board(s).

    Also, there's a decent chance that the module itself has no moving parts which are generally the first thing to go on a system. In the facility I work in, we only have a handful of non-disk or fan (or PS, which is usually the fan) related failures a year across a couple thousand servers. Get rid of the moving part failures, and we'd probably be swapping out less than one "module" a month at a location like mine.
    Reply
  • nathanddrews - Friday, November 22, 2013 - link

    So cool. Reply
  • marc1000 - Sunday, November 24, 2013 - link

    haha, oil-like cool! Reply
  • bobbozzo - Friday, November 22, 2013 - link

    Hi, how is the oil cooled?

    e.g. via air-cooled heatsinks/exchangers, or freon-cooled, or ?

    Thanks!
    Reply
  • errorr - Friday, November 22, 2013 - link

    Answered this elsewhere but the data sheet said heat exchanger to building water (hot or cold) hookup. So yay tap water. Reply
  • SuckRaven - Friday, November 22, 2013 - link

    My computer needs an oil change. =) Reply
  • mfenn - Friday, November 22, 2013 - link

    KFC obviously stands for Kentucky Fried Chicken. Kepler Fluid Cooling is just a respectable-sounding bacronym. ;) Reply
  • Belegost - Friday, November 22, 2013 - link

    I was hoping the KFC here meant the hot oil could be used for frying poultry. I am greatly disappointed. Reply
  • xdrol - Friday, November 22, 2013 - link

    You should check your facts.. Mineral oil has about one third heat capacity than water, just you can submerge chips into it (unlike water). Reply
  • errorr - Friday, November 22, 2013 - link

    Um, just a correction that I would only make on a site like this but oil has nowhere near the heat capacity of water. Besides pure hydrogen or helium only anhydrous ammonia exceeds the heat capacity of water.

    I believe they use "white" mineral oil (baby oil, liquid parrafin) with additives of Teflon like chemicals to boost heat capacity a bit. Still the heat capacity is still probably less than 1/3 of water.

    Of course there is much more mass so the heat capacity of the whole system is better than if you could somehow use distilled water in a sealed vessel.

    My understanding is also that the system is cooled by using tap water and a heat exchanger to answer someone's question.
    Reply
  • Tchamber - Friday, November 22, 2013 - link

    It may have lower heat capacity than water, but what it is good at is absorbing heat...I would guess that it absorbs heat at a higher rate, much like aluminum absorbs heat faster than steal. With enough volume, it has better cooling efficiency than any closed-loop H2O system. Reply
  • meacupla - Saturday, November 23, 2013 - link

    I doubt that, it is an oil after all and has nowhere near the same amount of bonds as water can achieve.

    the advantage this system has is sheer reservoir volume, its massive heat exchange surface area and because it is devoid of air, which performs worse than oil.
    Reply
  • MrSpadge - Saturday, November 23, 2013 - link

    That's the heat conductivity you're talking about. Don't know the numbers for such oils, though. Reply
  • ShieTar - Monday, November 25, 2013 - link

    Why would there be more mass? Most mineral oils have a lower density than water. Its actually the heat capacity per volume that makes water the best heat storage system. Reply
  • ddriver - Saturday, November 23, 2013 - link

    Got to love AT's eye-poking bias, I've been used to "top 5" or "top 10", but top 6? Now that is a curious choice. I bet it was intended as a top 5 chart, just like the second, but was expanded to 6 just to squeeze one more intel reference inside, god forbid AT put a "top 5" chart where intel is outmatched by ibm chips and has only one intel entry, just as much as amd. For the second chart 5 entries seem to suffice, probably because all systems are intel.

    Yeah, keep calling yourself "the most trusted" ... Articles without in-your-face brutally obvious bias at AT are becoming as rare as a hen's teeth. There is not even remote subtlety...
    Reply
  • Ryan Smith - Saturday, November 23, 2013 - link

    It would have been top 5, but since the only new supercomputer in the larger top 10 was Piz Daint at #6, I stretched it by one so that we could list those numbers and talk about it. Vulcan, SuperMUC, JUQUEEN, and Stampede were all online in June. Or would you rather we ignore the most powerful supercomputer in Europe? Reply
  • CPU-geek - Saturday, November 23, 2013 - link

    Is anandtech planning to write about Micron's new automata processor. Seems like a very interesting technology. Reply
  • SteelRing - Monday, November 25, 2013 - link

    They need to develop a special coolant liquid that is efficient at heat conduction yet has electrical insulation property. Most material that conducts heat does it by conducting electron, so this is super tricky. But if this is available, I can't see why all computer systems won't be submerged cooled from then on. Reply
  • dew111 - Monday, November 25, 2013 - link

    " In fact you have to go down past #11 – the Xeon + FirePro S10000 based SANAM computer – to find a system that isn’t GPU powered."

    Isn't the FirePro S10000 a GPU? It's not nvidia, which is probably what you meant, but it is definitely a GPU.
    Reply
  • dew111 - Monday, November 25, 2013 - link

    Oh, I see what you mean, number 11 is GPU powered, and number 12 isn't. But that's a bit confusing, as you mention the GPU powered on and not the first non-GPU powered supercomputer. Reply

Log in

Don't have an account? Sign up now