Benchmark Configuration and Methodology

For our look at the ThunderX2, all of our testing was conducted on Ubuntu Server 17.10, Linux kernel 4.13 64 bit. Normally we would use an LTS version, but since the Cavium shipped with that Ubuntu version, we did not want to take any unnecessary risks by changing the OS. The compiler that ships with this distribution is GCC 7.2.

Unfortunately however, our AMD EPYC system has missed the deadline for this article. We ran into problems with that system right up to press time and are still debugging the matter. But in short, the system did not perform well after we performed a kernel upgrade.

Finally, you will notice that the DRAM capacity varies among our server configurations. The reason is simple: Intel's system has 6 memory channels, while Cavium's ThunderX2 has 8 memory channels.

Gigabyte - Cavium "Saber"

CPU Two Cavium ThunderX2 CN9980 (32 cores at 2.2 - 2.5 GHz)
RAM 512 GB (16x32GB) Micron Reg. DDR4 @2666
Internal Disks SANDISK Cloudspeed Gen II 800 GB
Motherboard Cavium Sabre
BIOS version 18/2/2018
PSU Dual 1600W 80+ Platinum

 

Intel's Xeon "Purley" Server – S2P2SY3Q (2U Chassis)

CPU Two Intel Xeon Platinum 8176  (28 cores at 2.1 GHz, 165W)
RAM 384 GB (12x32 GB) Hynix DDR4-2666
Internal Disks SAMSUNG MZ7LM240 (bootdisk)
Intel SSD3710 800 GB (data)
Motherboard Intel S2600WF (Wolf Pass baseboard)
Chipset Intel Wellsburg
BIOS version 9/02/2017
PSU 1100W PSU (80+ Platinum)

The typical BIOS settings can be seen below. I should also note that we have both hyperthreading and Intel's virtualization technology enabled.

Other Notes

Both servers are fed by a standard European 230V (16 Amps max.) power line. The room temperature is monitored and kept at 23°C by our Airwell CRACs.

Energy Consumption

One thing that concerned us was the fact that the Gigabyte "Saber" system consumed 500W while simply running Linux (so mostly idle). Under load however the system consumed around 800W, which is in line with our expectations, as we have two 180W TDP chips inside. So as is typically the case for early test systems, we are not able to do any accurate power comparisons.

In fact, Cavium claims that the actual systems from HP, Gigabyte and others will be far more power efficient. The "Sabre" testing system we received had several power management problems: immature fan management firmware, a BMC bug, and an oversized (1600W) PSU.

The ThunderX2 SKUs: 16 to 32 Cores Memory Subsystem Measurements
POST A COMMENT

98 Comments

View All Comments

  • DrizztVD - Wednesday, May 23, 2018 - link

    It amazes me how the one big advantage ARM could have is the power efficiency, yet no power efficiency numbers in this review? It's like someone just isn't thinking about what can best showcase the ARM advantage and testing it. Reply
  • boeush - Thursday, May 24, 2018 - link

    You must have missed this bit:

    "So as is typically the case for early test systems, we are not able to do any accurate power comparisons.

    In fact, Cavium claims that the actual systems from HP, Gigabyte and others will be far more power efficient."

    This was an early (and apparently quite buggy, especially from the power management standpoint) test system. It's not representative of final production systems in these respects, so doing what you request on it would only put a very crude lower bound on efficiency, at best.

    That's why the final section of the write-up has a title ending in ": so far"... (obviously, there will be more to come if/when real production-quality systems are available for benchmarking/analysis.)
    Reply
  • ZolaIII - Thursday, May 24, 2018 - link

    It's broken currently on the MB. If you want to see real power/performance metrics for a SoC made on comparable lithography to the lintels 14 nm (aka TSMC 10nm) & with optimised software read this:
    https://blog.cloudflare.com/neon-is-the-new-black/
    Reply
  • drwho9437 - Wednesday, May 23, 2018 - link

    Thanks Johan, I've been reading since Ace's. I can't believe it has been more almost 20 years. Even though I don't work in this market I still read everything you write. Reply
  • JohanAnandtech - Friday, May 25, 2018 - link

    It was indeed almost 20 years ago that I published my first article about the K6-2 vs Pentium MMX. And Anand's star was about to rise with the launch of the K6-3 :-). Reply
  • Spatz - Wednesday, May 30, 2018 - link

    Wow. Aces hardware... that used to be my go to for hardware reviews back in the day. I can’t believe your still at it! This article was great. Keep up the good work. Reply
  • beginner99 - Thursday, May 24, 2018 - link

    So it for sure is an option. however I d not get the focus on price. The CPU cost is a small fraction of the total server cost and a tiny if infrastructure cost (network, HVAC,...) is included. Add to that the software and data running on that server and if your CPU is 5% faster at same power it costing $5000 more might be totally worth it. Reply
  • Apple Worshipper - Thursday, May 24, 2018 - link

    Errmm... does ARM feature SMT now? Reply
  • Ryan Smith - Thursday, May 24, 2018 - link

    Not in Arm's own cores. But in Cavium's ThunderX2, yes. Reply
  • sgeocla - Thursday, May 24, 2018 - link

    What's up with EPYC comparison missing in almost all benchmarks?
    EPYC has been out for a while and the only benchmarks are from almost a year ago?
    Reply

Log in

Don't have an account? Sign up now