Benchmark Configuration and Methodology

For our look at the ThunderX2, all of our testing was conducted on Ubuntu Server 17.10, Linux kernel 4.13 64 bit. Normally we would use an LTS version, but since the Cavium shipped with that Ubuntu version, we did not want to take any unnecessary risks by changing the OS. The compiler that ships with this distribution is GCC 7.2.

Unfortunately however, our AMD EPYC system has missed the deadline for this article. We ran into problems with that system right up to press time and are still debugging the matter. But in short, the system did not perform well after we performed a kernel upgrade.

Finally, you will notice that the DRAM capacity varies among our server configurations. The reason is simple: Intel's system has 6 memory channels, while Cavium's ThunderX2 has 8 memory channels.

Gigabyte - Cavium "Saber"

CPU Two Cavium ThunderX2 CN9980 (32 cores at 2.2 - 2.5 GHz)
RAM 512 GB (16x32GB) Micron Reg. DDR4 @2666
Internal Disks SANDISK Cloudspeed Gen II 800 GB
Motherboard Cavium Sabre
BIOS version 18/2/2018
PSU Dual 1600W 80+ Platinum


Intel's Xeon "Purley" Server – S2P2SY3Q (2U Chassis)

CPU Two Intel Xeon Platinum 8176  (28 cores at 2.1 GHz, 165W)
RAM 384 GB (12x32 GB) Hynix DDR4-2666
Internal Disks SAMSUNG MZ7LM240 (bootdisk)
Intel SSD3710 800 GB (data)
Motherboard Intel S2600WF (Wolf Pass baseboard)
Chipset Intel Wellsburg
BIOS version 9/02/2017
PSU 1100W PSU (80+ Platinum)

The typical BIOS settings can be seen below. I should also note that we have both hyperthreading and Intel's virtualization technology enabled.

Other Notes

Both servers are fed by a standard European 230V (16 Amps max.) power line. The room temperature is monitored and kept at 23°C by our Airwell CRACs.

Energy Consumption

One thing that concerned us was the fact that the Gigabyte "Saber" system consumed 500W while simply running Linux (so mostly idle). Under load however the system consumed around 800W, which is in line with our expectations, as we have two 180W TDP chips inside. So as is typically the case for early test systems, we are not able to do any accurate power comparisons.

In fact, Cavium claims that the actual systems from HP, Gigabyte and others will be far more power efficient. The "Sabre" testing system we received had several power management problems: immature fan management firmware, a BMC bug, and an oversized (1600W) PSU.

The ThunderX2 SKUs: 16 to 32 Cores Memory Subsystem Measurements


View All Comments

  • Eris_Floralia - Wednesday, May 23, 2018 - link

    The L2$ for SKX should be 1MB (256+768KiB), 16-way. Reply
  • Ryan Smith - Wednesday, May 23, 2018 - link

    Right you are. Thanks! Reply
  • danjme - Wednesday, May 23, 2018 - link

    Mental. Reply
  • Duncan Macdonald - Wednesday, May 23, 2018 - link

    The CPU may be much cheaper than the equivalent Intel CPU - however on the price of a complete server there would be almost no difference as the vast majority of the price of a server is in other items (RAM, storage, network, software etc). To take a significant share, the performance needs to be better than Intel CPUs on both a per thread and a per socket basis. Potential users will look at this CPU - see that it is not faster than Intel on a per thread basis and is also not X86-64 compatible and turn away with a shrug. A price difference of under 5% for a complete server is not enough to justify the risks of going from x86-64 to ARM. Reply
  • BurntMyBacon - Thursday, May 24, 2018 - link

    Perhaps you are correct and the lack of per thread performance will not allow Cavium to take a "significant' share of the market from Intel. However, at this point, getting even a small amount of market penetration in the server market is a significant achievement for an ARM vendor. This processor doesn't need to take a "significant" share from Intel to be successful. It just needs to establish a solid foothold. Given the data, I think it has a good chance of succeeding in that.

    The bigger question in my mind is how Intel will respond. They already have the ability to make a many lite core accelerator as demonstrated by the Xeon Phi line. Will they bring this tech to their CPU lineup, create a new accelerator based on this tech to handle applications that use many light threads, create a new many small core CPU based on Goldmont Plus (or Tremont), or will they consider the ARM threat insignificant enough to ignore.
  • boeush - Wednesday, May 23, 2018 - link

    "(*) EPYC and Xeon E5 V4 are older results, run on Kernel 4.8 and a slightly older Java 1.8.0_131 instead of 1.8.0_161. Though we expect that the results would be very similar on kernel 4.13 and Java 1.8.0_161"

    What about Spectre/Meltdown mitigation patches? Were they in effect for 'older' results?
  • boeush - Wednesday, May 23, 2018 - link

    To elaborate: if those numbers really are from July 2017, then they don't reflect true performance in a server context any longer (servers are where Spectre/Meltdown patches would be applied most.). Since the performance impact of Spectre/Meltdown is greatest on speculative execution and memory loads/prefetching, I'd guess those super-aggressive memory subsystem performance numbers, as well as single thread IPC advantages that Intel's CPUs claim in your benchmarks, are not really entirely applicable any longer. Reply
  • HStewart - Wednesday, May 23, 2018 - link

    Spectre has been proved to effect other CPU's than Intel and even effects ARM and AMD.,

    Image on this article states that this CPU supports Fully Out of Order execution. So with my understanding of Spectre that this CPU also has issues.

    To be honest I not sure how much the whole Spectre/Meltdown stuff is in this real world. It probably cause more harm in the computer industry than help.
  • Manch - Thursday, May 24, 2018 - link

    Commentor: Blah Blah Blah Spectre?
    HStewart: Shill Shill Shill must defend Intel by any means...
  • lmcd - Thursday, May 24, 2018 - link

    Commentor: reasonable position taken
    Manch: *banned for unreasonable, offensive comments*

Log in

Don't have an account? Sign up now