The Ampere Altra Review: 2x 80 Cores Arm Server Performance Monster
by Andrei Frumusanu on December 18, 2020 6:00 AM EST- Posted in
- Servers
- Neoverse N1
- Ampere
- Altra
Test Bed and Setup - Compiler Options
For the rest of our performance testing, we’re disclosing the details of the various test setups:
Ampere "Mount Jade" - Dual Altra Q80-33
Obviously, for the Ampere Altra system we’re using the provided Mount Jade server as configured by Ampere.
The system features 2 Altra Q80-33 processors within the Mount Jade DVT motherboard from Ampere.
In terms of memory, we’re using the bundled 16 DIMMs of 32GB of Samsung DDR4-3200 for a total of 512GB, 256GB per socket.
| CPU | 2x Ampere Altra Q80-33 (3.3 GHz, 80c, 32 MB L3, 250W) |
| RAM | 512 GB (16x32 GB) Samsung DDR4-3200 |
| Internal Disks | Samsung MZ-QLB960NE 960GB Samsung MZ-1LB960NE 960GB |
| Motherboard | Mount Jade DVT Reference Motherboard |
| PSU | 2000W (94%) |
The system came preinstalled with CentOS 8 and we continued usage of that OS. It’s to be noted that the server is naturally Arm SBSA compatible and thus you can run any kind of Linux distribution on it.
Ampere makes special note of Oracle’s active support of their variant of Oracle Linux for Altra, which makes sense given that Oracle a few months ago announced adoption of Altra systems for their own cloud-based offerings.
The only other note to make of the system is that the OS is running with 64KB pages rather than the usual 4KB pages – this either can be seen as a testing discrepancy or an advantage on the part of the Arm system given that the next page size step for x86 systems is 2MB – which isn’t feasible for general use-case testing and something deployments would have to decide to explicitly enable.
The system has all relevant security mitigations activated, including SSBS (Speculative Store Bypass Safe) against Spectre variants.
AMD - Dual EPYC 7742
For our AMD system, unfortunately we had hit some issues with our Daytona reference server motherboard, and moved over to a test-bench setup on a SuperMicro H11DSI0.
We’re also equipping the system with 256GB per socket of 8-channel/DIMM DDR4-3200 memory, matching the Altra system.
| CPU | 2x AMD EPYC 7742 (2.25-3.4 GHz, 64c, 256 MB L3, 225W) |
| RAM | 512 GB (16x32 GB) Micron DDR4-3200 |
| Internal Disks | OCZ Vector 512GB |
| Motherboard | SuperMicro H11DSI0 |
| PSU | EVGA 1600 T2 (1600W) |
As an operating system we’re using Ubuntu 20.10 with no further optimisations. In terms of BIOS settings we’re using complete defaults, including retaining the default 225W TDP of the EPYC 7742’s, as well as leaving further CPU configurables to auto, except of NPS settings where it’s we explicitly state the configuration in the results.
The system has all relevant security mitigations activated against speculative store bypass and Spectre variants.
Intel - Dual Xeon Platinum 8280
For the Intel system we’re also using a test-bench setup with the same SSD and OS image actually – we didn’t have enough RAM to run both systems concurrently.
Because the Xeons only have 6-channel memory, their maximum capacity is limited to 384GB of the same Micron memory, running at a default 2933MHz to remain in-spec with the processor’s capabilities.
| CPU | 2x Intel Xeon Platinum 8280 (2.7-4.0 GHz, 28c, 38.5MB L3, 205W) |
| RAM | 384 GB (12x32 GB) Micron DDR4-3200 (Running at 2933MHz) |
| Internal Disks | OCZ Vector 512GB |
| Motherboard | ASRock EP2C621D12 WS |
| PSU | EVGA 1600 T2 (1600W) |
The Xeon system was similarly run on BIOS defaults on an ASRock EP2C621D12 WS with the latest firmware available.
The system has all relevant security mitigations activated against the various vulnerabilities.
Compiler Setup
For compiled tests, we’re using the release version of GCC 10.2. The toolchain was compiled from scratch on both the x86 systems as well as the Altra system. We’re using shared binaries with the system’s libc libraries.
Facebook
Twitter
RSS
148 Comments
View All Comments
Wilco1 - Monday, December 21, 2020 - link
Using Zen 2 is not correct since it uses much larger transistors. Using Kirin 990 5G density gives an estimate of 330mm^2 for Graviton 2. The size of N1 cores has been published for 7nm, so we know it is 1.4mm^2. You're right that PCIe lanes would add to it as well - assuming the PHYs have the same size as DDR PHYs at the same speed, 64 lanes would be about 12-15mm^2. That would increase it to about 365mm^2.milli - Monday, December 21, 2020 - link
Kirin 990 5G uses N7+. Altra uses N7.Not only is the process different but they're also totally different categories of products concerning transistor density. A mobile SOC can be very dense. It barely has any IO (which is not transistor dense). Also GPU, DPU, IMG, ... all are extremely dense.
Kirin 990 5G is 90MTr/mm^2.
No way a server class SOC is going to be more than 60MTr/mm2.
Renoir = 62, Navi 21 = 52, Zen2 = 54, Vega 20 = 40, Navi 10 = 41.
Ampere isn't going to magically break 60.
"The size of N1 cores has been published for 7nm, so we know it is 1.4mm^2"
Those are ARM numbers and that is only if you use high density libraries.
Wilco1 - Monday, December 21, 2020 - link
Arm servers don't need high performance libraries - even mobile phones clock over 3 GHz using high density libraries. See https://images.anandtech.com/doci/13959/03_Infra%2... (note 3.1GHz and 1.4mm^2 with 1MB L2 on 7nm is ~100MT/mm^2)Using ~90MT/mm^2 for 7nm is reasonable since that is the reported density of recent 7nm chips (Kirin 990 5G is 91, 4G is 88 - the older 980 gets 93). Mobile SoCs already have a large amount of IO and analog logic and we are multiplying that amount by 3x.
This shows how stupid it is to use high performance libraries in server chips - they don't need to run at 5GHz!
milli - Monday, December 21, 2020 - link
We have different opinions but there's only one true fact: the die size is not disclosed. So anything anyone says is just a pure guess. You can't throw it around as fact.milli - Monday, December 21, 2020 - link
Navi 10/20 chips run at < 2Ghz and are 40MTr/mm. Just because Altra runs at 3.3Ghz, doesn't mean that it doesn't use HPL.Josh128 - Friday, December 18, 2020 - link
Exactly-- no way in hell this thing is just 350mm^2. The package is huge. Why would a 350mm^2 die need such a giant package?Wilco1 - Friday, December 18, 2020 - link
The package is only 16% larger than EPYC. Do you see any opportunity to reduce the huge number of pins? There are 8 memory channels plus full 128 PCIe lanes.mode_13h - Sunday, December 20, 2020 - link
Yes, the problem Altra Max will likely face is more memory bottlenecks. Also, I wonder if they'll have to dial clocks down, a little, to keep the power-efficiency numbers attractive.Wilco1 - Monday, December 21, 2020 - link
Altra Max drops max frequency to 3GHz, but it's not clear whether the TDP stays the same.Gondalf - Friday, December 18, 2020 - link
Are you sure :). Come on