HPC: OpenFoam

Computational Fluid Dynamics is a very important part of the HPC world. Several readers told us that we should look into OpenFoam, and my lab was able to work with the professionals of Actiflow. Actiflow specializes in combining aerodynamics and product design. Calculating aerodynamics involves the use of CFD software, and Actiflow uses OpenFoam to accomplish this. To give you an idea what these skilled engineers can do, they worked with Ferrari to improve the underbody airflow of the Ferrari 599 and increase its downforce.

We were allowed to use one of their test cases as a benchmark, but we are not allowed to discuss the specific solver. All tests were done on OpenFoam 2.2.1 and openmpi-1.6.3.

Many CFD calculations do not scale well on clusters, that is unless you use InfiniBand. InfiniBand switches are quite expensive and even then there are limits to scaling. We do not have an InfiniBand switch in the lab, unfortunately. Although it's not as low latency as InfiniBand, we do have a good 10G Ethernet infrastructure, which performs rather well. So we can compare our newest Xeon server with a basic cluster.

We also found AVX code inside OpenFoam 2.2.1, so we assume that this is one of the cases where AVX improves FP performance.

OpenFoam Benchmark

Unless you recompile and tune your code for AVX2, the new Xeon E7 v3 is hardly faster than the previous one. The reason may be that the new Xeon can sometimes go as low as 2.1 GHz running AVX code due to the immense power load AVX2 workloads can cause, while the older E7 v2 is capable of sustaining 2.8 GHz.

Linux Kernel Compile HPC Watts per Job
Comments Locked

146 Comments

View All Comments

  • PowerTrumps - Saturday, May 9, 2015 - link

    Oracle has been unable to develop a power core let alone a processor. What they have done is created servers with many cores and many threads albeit weak cores/threads. The S3 core was an improvement and no reason to think the S4 won't be decent either. However, the M7 will come (again, true to form) with 32 cores per socket. It will be like 8 mini clusters of 4 cores because they are unable to develop a single SMP chip with shared resources across all of the cores. As such, these mini clusters will have their own resources which will lead to latency and inefficiencies. Oracle is a software business and their goal is to run software on either the most cores possible or the most inefficient. They have both of these bases covered with their Intel and SPARC business.

    Also, performance per Watt is important for Intel because what you see is what you get. With Power though, when you have strong single thread performance, strong multi-thread performance and tremendous consolidation efficiency due to Power Hypervisor efficiency means ~200W doesn't matter when you can consolidate 2, 4 maybe 10 Intel chips at 135W each into a single Power chip because of this hypervisor efficiency.
  • tynopik - Friday, May 8, 2015 - link

    pg4 - datam ining
  • der - Friday, May 8, 2015 - link

    Woo...we're bout to have another GHz War here!
  • usernametaken76 - Friday, May 8, 2015 - link

    I'm sure you mean figuratively. We've been stuck between 4-5 GHz on POWER architecture for closing in on a decade.
  • zamroni - Friday, May 8, 2015 - link

    My conclusion is Samsung should buy AMD to reduce Intel dominance.
  • alpha754293 - Friday, May 8, 2015 - link

    It would have been interesting to see the LS-DYNA benchmark results again (so that you can compare it against some of the tests that you've ran previously). But very interesting...
  • JohanAnandtech - Friday, May 8, 2015 - link

    Give me some help and we'll do that again on an update version :-)
  • alpha754293 - Tuesday, May 12, 2015 - link

    Not a problem. You have my email address right? And if not, I'll just send you another email and we can get that going again. :) Thanks.
  • andychow - Friday, May 8, 2015 - link

    If Samsung bought AMD, they would lose the licence for both x86 and x86_64 production. It would in fact ensure Intel's dominance of the market.
  • Kevin G - Friday, May 8, 2015 - link

    The x86 license can be transferred as long as Intel signs off on the deal (and it is in their best interest to do so). What will probably happen is that if any company buys AMD, the new owner will enter a cross licensing agreement with Intel.

Log in

Don't have an account? Sign up now