HPC: Fluid Dynamics with OpenFOAM

Computational Fluid Dynamics is a very important part of the HPC world. Several readers told us that we should look into OpenFOAM, and my lab was able to work with the professionals of Actiflow. Actiflow specializes in combining aerodynamics and product design. Calculating aerodynamics involves the use of CFD software, and Actiflow uses OpenFOAM  to accomplish this. To give you an idea what these skilled engineers can do, they worked with Ferrari to improve the underbody airflow of the Ferrari 599 and increase its downforce.

We were allowed to use one of their test cases as a benchmark, however we are not allowed to discuss the specific solver. All tests were done on OpenFOAM 2.2.1 and openmpi-1.6.3. The reason why we still run with OpenFOAM 2.2.1 is that our current test case does not work well with higher versions.

We also found AVX code inside OpenFoam 2.2.1, so we assume that this is one of the cases where AVX improves FP performance. 

OpenFOAM

As this is AVX code, the clock speed of our Xeon processors can be lower than Intel's official specifications, and turbo boost speeds are also lower. Despite the fact that on Broadwell the only cores that reduce their clock when running AVX code are the AVX-active cores themseves (the others can continue at higher speeds), OpenFOAM does not run appreciably faster on the top of the line Xeon E5 v4 than it did on the E5 v3.

It is not as if OpenFOAM does not scale: 22% more cores delivers 13% higher performance (E5-2699v4 vs E5-2695v4). No, our first impression is that the new Xeon v4 needs to lower the clockspeed more than the old one. The official specifications tell us that both the Xeon E5-2699 v4 and v3 should run AVX code at up to 2.6 GHz with all cores enabled. The reality is however that Broadwell runs at a lower clock on average. 

Spark Benchmarking NAMD
Comments Locked

112 Comments

View All Comments

  • Casper42 - Thursday, March 31, 2016 - link

    HPE just dropped the 64GB LRDIMMs a week or two back.
    They are now exactly 2x the 32GB LRDIMM as far as List Price goes.
    LRDIMMs across the board are 31% more expensive than RDIMMs.
  • wishgranter - Tuesday, April 5, 2016 - link

    http://www.techpowerup.com/221459/samsung-starts-m...
  • wishgranter - Tuesday, April 5, 2016 - link

    While introducing a wide array of 10nm-class DDR4 modules with capacities ranging from 4GB for notebook PCs to 128GB for enterprise servers, Samsung will be extending its 20nm DRAM line-up with its new 10nm-class DRAM portfolio throughout the year.
  • nathanddrews - Thursday, March 31, 2016 - link

    Perf/W is obviously a very exciting metric for server farmers and it generally exciting from a basic technology perspective, but it's absolute performance isn't amazing. Anyway, it's not like I'll be buying one anyway. LOL
  • asendra - Thursday, March 31, 2016 - link

    This interest me in so far as this would be the updated processors in a supposedly-coming-this-year Mac Pro refresh. Not that I would personally fork that much cash, but I'm interested to see how much of a jump they will make.

    But things seam rather bleak. No wonder they decided to wait 3 years for a refresh.
  • MrSpadge - Thursday, March 31, 2016 - link

    Not sure which years you're counting in, but for the majority of us it takes 1.5 years from 09/2014 to today.
    https://en.wikipedia.org/wiki/Haswell_%28microarch...
  • asendra - Thursday, March 31, 2016 - link

    Apple didn't update the MacPros with Haswell-EP. They are still using Ivy Bridge
  • tipoo - Thursday, March 31, 2016 - link


    Wonder what they'll do on the GPU side though. Too early for next generation 14nm FF GPUs from anyone, if Nvidia was even a choice due to OpenCL politics. Another GCN 1.0 part in 2016 would be...A bag of hurt.

    Still waiting on the high end 15" rMBP to have something better than GCN 1.0...The performance, shockingly, hasn't come all that far from even my Iris Pro model. Maybe double, which is something, but I'd like larger than that to upgrade from integrated...
  • extide - Thursday, March 31, 2016 - link

    Nah, if they refresh it late this year, like in august or something like that, then 14/16nm FF GPU's will be available.

    At worst we would get GCN 1.2, but yeah it would suck to see 28nm GPU's put in there...
  • mdriftmeyer - Thursday, March 31, 2016 - link

    On what planet do you not grasp FinFET 14nm end of June from AMD?

Log in

Don't have an account? Sign up now