Benchmark Configuration and Methodology

For our look at the ThunderX2, all of our testing was conducted on Ubuntu Server 17.10, Linux kernel 4.13 64 bit. Normally we would use an LTS version, but since the Cavium shipped with that Ubuntu version, we did not want to take any unnecessary risks by changing the OS. The compiler that ships with this distribution is GCC 7.2.

Unfortunately however, our AMD EPYC system has missed the deadline for this article. We ran into problems with that system right up to press time and are still debugging the matter. But in short, the system did not perform well after we performed a kernel upgrade.

Finally, you will notice that the DRAM capacity varies among our server configurations. The reason is simple: Intel's system has 6 memory channels, while Cavium's ThunderX2 has 8 memory channels.

Gigabyte - Cavium "Saber"

CPU Two Cavium ThunderX2 CN9980 (32 cores at 2.2 - 2.5 GHz)
RAM 512 GB (16x32GB) Micron Reg. DDR4 @2666
Internal Disks SANDISK Cloudspeed Gen II 800 GB
Motherboard Cavium Sabre
BIOS version 18/2/2018
PSU Dual 1600W 80+ Platinum

 

Intel's Xeon "Purley" Server – S2P2SY3Q (2U Chassis)

CPU Two Intel Xeon Platinum 8176  (28 cores at 2.1 GHz, 165W)
RAM 384 GB (12x32 GB) Hynix DDR4-2666
Internal Disks SAMSUNG MZ7LM240 (bootdisk)
Intel SSD3710 800 GB (data)
Motherboard Intel S2600WF (Wolf Pass baseboard)
Chipset Intel Wellsburg
BIOS version 9/02/2017
PSU 1100W PSU (80+ Platinum)

The typical BIOS settings can be seen below. I should also note that we have both hyperthreading and Intel's virtualization technology enabled.

Other Notes

Both servers are fed by a standard European 230V (16 Amps max.) power line. The room temperature is monitored and kept at 23°C by our Airwell CRACs.

Energy Consumption

One thing that concerned us was the fact that the Gigabyte "Saber" system consumed 500W while simply running Linux (so mostly idle). Under load however the system consumed around 800W, which is in line with our expectations, as we have two 180W TDP chips inside. So as is typically the case for early test systems, we are not able to do any accurate power comparisons.

In fact, Cavium claims that the actual systems from HP, Gigabyte and others will be far more power efficient. The "Sabre" testing system we received had several power management problems: immature fan management firmware, a BMC bug, and an oversized (1600W) PSU.

The ThunderX2 SKUs: 16 to 32 Cores Memory Subsystem Measurements
POST A COMMENT

98 Comments

View All Comments

  • JohanAnandtech - Thursday, May 24, 2018 - link

    I have been trouble shooting a Java problem for the last 3 weeks now - for some reason our specific EPYC test system has some serious performance issues after we upgraded to kernel 4.13. This might be a hardware/firmware... issue. I don't know. I just know that the current tests are not accurate. Reply
  • npz - Thursday, May 24, 2018 - link

    Large Pages should be used whenever possible on Intel. You do waste some more memory, but it's worth it for most workloads as can be seen in your Intel Java benchmark. We've tested it for IO devices and enabling large pages in drivers to do DMA to shows a big difference for some high throughput devices. Reply
  • junky77 - Thursday, May 24, 2018 - link

    What? A 2.5GHZ ARM core is around 60-70% of a 3.8GHZ Skylake core?? For 3.8GHZ, the ARM is probably at least as fast? Reply
  • Wilco1 - Thursday, May 24, 2018 - link

    Probably around 90% since performance doesn't scale linearly with frequency. Note these are throughput parts so won't clock that high. However a 7nm version might well reach 3GHz. Reply
  • AJ_NEWMAN - Thursday, May 24, 2018 - link

    If Caviums tweaked 16nm hits 3GHz - it would to be unreasonable to aim for 4GHz for a 7nm part.

    With 2.3 times as many transistors available - it will be interesting to see what else they beef up?

    HIgher IPC? 64 cores? 16 memory controllers? CCIX - or perhaps they will compete with Fujitsu and add some Supercomputer centric hardware?

    AJ
    Reply
  • meta.x.gdb - Thursday, May 31, 2018 - link

    Wonder why the VASP code limped along on ThunderX2 while OpenFOAM saw such gains. I'm pretty familiar with both codes. VASP is mostly doing density functional theory, which is FFT-heavy... Reply
  • Meteor2 - Tuesday, June 26, 2018 - link

    All I want to say (all I can say) is that Anandtech has some of the best writers and commenters in this field. Fantastic article, and fantastic discussion. Reply
  • paldU - Saturday, July 7, 2018 - link

    A typo in Page 2. "it terms of performance per dollar" should be " in terms of performance per dollar". Reply

Log in

Don't have an account? Sign up now