The Ampere Altra Max Review: Pushing it to 128 Cores per Socket
by Andrei Frumusanu on October 7, 2021 8:00 AM EST- Posted in
- Servers
- Arm
- Neoverse N1
- Ampere
- Altra Max
SPECjbb MultiJVM - Java Performance
Moving on from SPECCPU, we shift over to SPECjbb2015. SPECjbb is a from ground-up developed benchmark that aims to cover both Java performance and server-like workloads, from the SPEC website:
“The SPECjbb2015 benchmark is based on the usage model of a worldwide supermarket company with an IT infrastructure that handles a mix of point-of-sale requests, online purchases, and data-mining operations. It exercises Java 7 and higher features, using the latest data formats (XML), communication using compression, and secure messaging.
Performance metrics are provided for both pure throughput and critical throughput under service-level agreements (SLAs), with response times ranging from 10 to 100 milliseconds.”
The important thing to note here is that the workload is of a transactional nature that mostly works on the data-plane, between different Java virtual machines, and thus threads.
We’re using the MultiJVM test method where as all the benchmark components, meaning controller, server and client virtual machines are running on the same physical machine.
The JVM runtime we’re using is OpenJDK 15 on both x86 and Arm platforms, although not exactly the same sub-version, but closest we could get:
EPYC & Xeon systems:
openjdk 15 2020-09-15
OpenJDK Runtime Environment (build 15+36-Ubuntu-1)
OpenJDK 64-Bit Server VM (build 15+36-Ubuntu-1, mixed mode, sharing)
Altra system:
openjdk 15.0.1 2020-10-20
OpenJDK Runtime Environment 20.9 (build 15.0.1+9)
OpenJDK 64-Bit Server VM 20.9 (build 15.0.1+9, mixed mode, sharing)
Furthermore, we’re configuring SPECjbb’s runtime settings with the following configurables:
SPEC_OPTS_C="-Dspecjbb.group.count=$GROUP_COUNT -Dspecjbb.txi.pergroup.count=$TI_JVM_COUNT -Dspecjbb.forkjoin.workers=N -Dspecjbb.forkjoin.workers.Tier1=N -Dspecjbb.forkjoin.workers.Tier2=1 -Dspecjbb.forkjoin.workers.Tier3=16"
Where N=160 for 2S Altra test runs, N=128 for 1S Altra Max runs, N=80 for 1S Altra test runs, N=112 for 2S Xeon 8280, N=56 for 1S Xeon 8280, and N=128 for 2S and 1S on the EPYC system. The 75F3 system had the worker count reduced to 64 and 32 for 2S/1S runs, with the 7443, 7343 and 72F3 also having the same thread to core ratiio. The Xeon 8380 was running at N=140 for 2S Xeon 8380 and N=70 for 1S - the benchmark had been erroring out at higher thread counts.
In terms of JVM options, we’re limiting ourselves to bare-bone options to keep things simple and straightforward:
EPYC systems:
JAVA_OPTS_C="-server -Xms2g -Xmx2g -Xmn1536m -XX:+UseParallelGC "
JAVA_OPTS_TI="-server -Xms2g -Xmx2g -Xmn1536m -XX:+UseParallelGC"
JAVA_OPTS_BE="-server -Xms48g -Xmx48g -Xmn42g -XX:+UseParallelGC -XX:+AlwaysPreTouch"
Xeon Cascade Lake systems:
JAVA_OPTS_C="-server -Xms2g -Xmx2g -Xmn1536m -XX:+UseParallelGC"
JAVA_OPTS_TI="-server -Xms2g -Xmx2g -Xmn1536m -XX:+UseParallelGC"
JAVA_OPTS_BE="-server -Xms172g -Xmx172g -Xmn156g -XX:+UseParallelGC -XX:+AlwaysPreTouch"
Xeon Ice Lake (SNC1) & Altra systems:
JAVA_OPTS_C="-server -Xms2g -Xmx2g -Xmn1536m -XX:+UseParallelGC"
JAVA_OPTS_TI="-server -Xms2g -Xmx2g -Xmn1536m -XX:+UseParallelGC"
JAVA_OPTS_BE="-server -Xms192g -Xmx192g -Xmn168g -XX:+UseParallelGC -XX:+AlwaysPreTouch"
Xeon Ice Lake systems (SNC2):
JAVA_OPTS_C="-server -Xms2g -Xmx2g -Xmn1536m -XX:+UseParallelGC"
JAVA_OPTS_TI="-server -Xms2g -Xmx2g -Xmn1536m -XX:+UseParallelGC"
JAVA_OPTS_BE="-server -Xms96g -Xmx96g -Xmn84g -XX:+UseParallelGC -XX:+AlwaysPreTouch"
The reason the Xeon CLX system is running a larger back-end heap is because we’re running a single NUMA node per socket, while for the Altra and EPYC we’re running four NUMA nodes per socket for maximised throughput, meaning for the 2S figures we have 8 backends running for the Altra and EPYC and 2 for the Xeon, and naturally half of those numbers for the 1S benchmarks.
For the Ice Lake system, I ran both SNC1 (one NUMA node) as SNC2 (two nodes), with the corresponding scaling in the back-end memory allocation.
The back-ends and transaction injectors are affinitised to their local NUMA node with numactl –cpunodebind and –membind, while the controller is called with –interleave=all.
The max-jOPS and critical-jOPS result figures are defined as follows:
"The max-jOPS is the last successful injection rate before the first failing injection rate where the reattempt also fails. For example, if during the RT-curve phase the injection rate of 80000 passes, but the next injection rate of 90000 fails on two successive attempts, then the max-jOPS would be 80000."
"The overall critical-jOPS is computed by taking the geomean of the individual critical-jOPS computed at these five SLA points, namely:
• Critical-jOPSoverall = Geo-mean of (critical-jOPS@ 10ms, 25ms, 50ms, 75ms and 100ms response time SLAs)
During the RT curve building phase the Transaction Injector measures the 99th percentile response times at each step level for all the requests (see section 9) that are considered in the metrics computations. It then computes the Critical-jOPS for each of the above five SLA points using the following formula:
(first * nOver + last * nUnder) / (nOver + nUnder) "
That’s a lot of technicalities to explain an admittedly complex benchmark, but the gist of it is that max-jOPS represents the maximum transaction throughput of a system until further requests fail, and critical-jOPS is an aggregate geomean transaction throughput within several levels of guaranteed response times, essentially different levels of quality of service.
On the SLA graph with the response times across increasing load, we see the Altra Max not differ too much from its predecessor, though it’s simply showcasing improved performance.
In our initial review of the Q80-33 last year, we had noted that the chip offered lacklustre performance metrics in the SPECjbb benchmark, and though this was somehow attributed to Java performance on AArch64. After spending some more time trying to debug the issue prior to this review, I surprisingly found out that the issue, at least in our configuration, was due to the Altra chip’s operating mode. While we first had tested the chip in quadrant mode, where the chip is partitioned in respective four NUMA nodes, and running four SPECjbb back-ends (one per NUMA node). Running the chip in monolithic mode as an experiment, surprisingly resolved all our performance issues with SPECjbb, with the Q80-33 now running in line where Arm had expected the system to land, which meant increase in the max-jOPS metric and a more massive increase in the critical-jOPS metric as we’ll see in a bit. Unfortunately, trading in one issue with another, we ran into other issues on the 2-socket test scenario where the test ran into issues at large thread counts. The 2S Q80-33 figures here only stresses 130 cores, while I wasn’t able at all to get 2S M128-30 figures at reasonable core counts, so I completely omitted results here.
Focusing on the 1-socket results instead, both the Q80-33 and the new M128-30 now showcase much better performance than what we had seen in the first review of the Altra. The new M128-30 sees a +26% boost in peak throughput performance compared to the Q80-33, however, the chip still lags behind AMD’s flagship EPYC 7763.
The Altra Max here not only is able to increase performance through more cores, but it’s also just outright able to make more usage of its system TDP of 250W. The workload being more data-plane bound, which is also the reason it scales well with SMT, had the effect that the 80-cores on the Q80-33 were running at lower execution resources and lower power, averaging at ~180W, quite below the 250W TDP. The per-core utilisation doesn’t change on the M128-30, but throwing more cores at the matter does help saturate more of the available TDP and result in more performance.
The new critical-jOPS figures for this review are massively improved, with the Q80-33 now posting double that of what we had originally measured. The new M128-30 now further pushes this up, reaching an impressive figure of 96k-jOPS, significantly above the second-best CPU which is the EPYC 7763 at 73k. These new results are much more in line with what we had expected of a single large monolithic CPU design, and showcases the Altra Max in its best possible light and what Ampere tries to focus on – better performance predictability than what the competition can offer with SMT.
We’ll continue to try to figure out what’s happening with the system in the dual-socket scenarios – it’s possible we’re hitting some sort of inherent scaling issue with SPECjbb at these massive thread counts.
Facebook
Twitter
RSS
60 Comments
View All Comments
Wilco1 - Sunday, October 10, 2021 - link
> You need only note the cases where Max significantly underperforms, relative to its 80-core sibling, to see where the cache reduction is likely an issue.There are regressions in 4 of the 10 integer benchmarks - only mcf is significant. However if you look closely, Altra Max still beats/equals the 8380 in 3 out of those 4. Clearly a 40MB L3 is not large enough for these benchmarks, so would you also call that a "major liability"? EPYC beats all by a huge margin in these 4, so clearly 256MB L3 works well, but it's also way too expensive for a monolithic die.
> The reason why there are so many different benchmarks is that you can't just seize on the aggregate numbers to tell the whole story.
No. The aggregate result averages out the extremes and is a better prediction for average performance. For example Altra Max is slower than Altra on gcc_r and far behind EPYC. However in LLVM compilation Altra Max beats Altra by ~20% and is pretty much equal to the 7763. So in real world tests EPYC's huge caches don't help nearly as much as the gcc_r subtest suggests.
mode_13h - Sunday, October 10, 2021 - link
> There are regressions in 4 of the 10 integer benchmarks - only mcf is significant.When you have 45% more core x GHz, *any* regression is significant! By that token, we should also be marking the xz test as underperforming, since it's only a ~20% improvement.
It's also convenient to seize on specint, when it suffers regressions on 7 of the 12 specfp tests.
> Altra Max still beats/equals the 8380 in 3 out of those 4.
> Clearly a 40MB L3 is not large enough for these benchmarks,
> so would you also call that a "major liability"?
This seems like a rather disingenuous point. To say anything about the 8380's cache, we'd need to see a comparison against other Ice Lake CPUs with a different core-to-cache ratio.
> No. The aggregate result averages out the extremes and is a better
> prediction for average performance.
It's a flawed inference to conclude that "the average workload" will match an unweighted average of a set of intentionally disparate workloads.
Furthermore, people less & less buy hardware for "the average workload".
> in LLVM compilation Altra Max beats Altra by ~20% and is pretty much equal to the 7763.
You can't just cherry-pick the best results of each memory configuration. If you're going to deal in aggregates, then you need to aggregate the results per-configuration.
> So in real world tests EPYC's huge caches don't help nearly as much as the gcc_r subtest suggests.
As a matter of fact, the monolithic vs. quadrant results would argue the opposite, in your chosen example of LLVM. Furthermore, what qualifies LLVM compilation as more "real world" than the gcc test?
schujj07 - Monday, October 11, 2021 - link
"The Altra Max wins the more useful critical-jOPS benchmark by over 30%"What are you talking about? In both critical-jOPS & max-jOPS the 2s 7763 is on the top of the chart. We cannot try to extrapolate possible performance on the Altra Max due to "Unfortunately, trading in one issue with another, we ran into other issues on the 2-socket test scenario where the test ran into issues at large thread counts. The 2S Q80-33 figures here only stresses 130 cores, while I wasn’t able at all to get 2S M128-30 figures at reasonable core counts, so I completely omitted results here."
Per-core performance matters a lot. There are A LOT of programs, especially databases, that are licensed on a per core metric. This means I need 8 cores of Altra Max to equal the performance I get from an Epyc 4c that will kill my licensing cost. Those added cores could easily double the license cost and those license are often times MUCH more expensive than the server itself. It is obvious you don't work in industry as this is common knowledge.
Overall the Altra Max is interesting but nothing more than that. It won't be a player in industry until the per core performance is at least double what it currently is and there is enterprise software able to take advantage of it. Basically Altra Max is like IBM Power and that is niche at best.
Wilco1 - Tuesday, October 12, 2021 - link
Altra Max is still at the top for 1S critical-jOPS - that's not invalidated by missing 2S results.If you worked in the industry, you would know that per-core licenses have a multiplier based on CPU type to level out performance differences. In cases where per-core performance really matters and you completely disable SMT (for example for high-frequency trading), you would not consider these many-core servers at all but get 8 or 16 core CPUs with significantly higher bandwidth, cache and power per core.
It seems you misunderstand the target market completely. You probably also call Graviton 2 a niche eventhough it is already a significant percentage of AWS and growing fast. And that with just 64 cores and far lower per-core performance than Altra...
schujj07 - Tuesday, October 12, 2021 - link
How about we do some math instead. Compared to the 1S 80 core, the 1S Max gets 42% better performance, THP disabled, or 30% better performance THP enabled for 60% more cores in your beloved critical-jOPS. Compare that to the Epyc 7763 which gets 105% better THP disabled and 102% THP enabled. Even the older 80C only adds 62% despite doubling its cores. Based on that alone best case scenario is the 2S Altra Max ties the Epyc 2S in critical-jOPS. Sure it is beating the 1S 7763 but it barely beats the 2S 7443 a 24c/48t CPU.I do work in industry as a VMware Admin. Unless you are running Oracle, most of these will be run on systems up to 32c/64t to max out your VMware license. If you have specific needs you can get the higher frequency parts that also are up to 32c AMD or 28c Intel. The difference in costs for Windows DataCenter for the core additional core licenses is saved by reducing the number of physical hosts. What software has "per-core licenses have a multiplier based on CPU type to level out performance differences?" That sounds like they are going to charge your X for Xeon Scalable Gen 1 but Y for Gen 2 and Z for AMD. That doesn't happen. MS SQL Server charges per core with a base license of 4 cores. Now if I need 8 cores on the Altra Max to equal the performance of an Epyc at 4 cores I have doubled my license cost.
Overall ARM with under 10% total market share IS a niche player. They need to get to the same per core performance & have software available for it to be an actual alternative. Until that happens companies will play around with it but nothing serious in the data center environment.
mode_13h - Tuesday, October 12, 2021 - link
> You probably also call Graviton 2 a nicheDon't put words in people's mouths. If you want to know whether @schujj07 considers it a niche, you can certainly ask.
Kangal - Thursday, October 7, 2021 - link
This is basically a 3GHz Cortex-A76 (Neoverse N1), running in a 128-core tandem, and built with a more efficient/expensive Monolithic Socket based on TSMC's 7nm node. Sounds neat.I enjoyed seeing the older generation which was basically a 2GHz Cortex-A73, running in 64-core tandem, and built on TSMC's 16nm node. Was quiet value-for-money, at least in its time.
Seems like this new version is giving Intel's Core-i, decent competition in the single-threaded work. Since Intel is having some issues with their own node, and can't clock too high. Whilst AMD has a clear advantage here. When it comes to total/multi-threaded performance, ARM wins through sheer grunt of all those extra cores. Overall, it is a competitive choice for today and the next few years.
What will be interesting is when they bump it up to the Cortex-A78 (Neoverse V1) and use something like TSMC's 5nm node which should bring it to full-parity on the single-threaded performance against Intel. Or to the next best thing, ARM v9, using the Cortex-X2 (Neoverse N2) on the same TSMC 5nm node. But I share my previous concerns that the first-generation of (USA) ARM v9 is going to be quiet disappointing, but I'm optimistic about the (European) second-generation. I think then we should see more tangible benefits, when combined with the TSMC 3nm node, which should bring it on parity to AMD's cores on the single-threaded characteristic. Exciting times ahead. And yes, I know I am over-simplifying things here.
SarahKerrigan - Thursday, October 7, 2021 - link
Previous Ampere parts weren't 64-core, 2GHz, or Cortex-A73. They were a custom (and bad) core, 32 per socket, at 3.3GHz.Neoverse V1 is based on the Cortex-X1, not the Cortex-A78. Neoverse N2 is based on the Cortex-A710, not the Cortex-X2.
Kangal - Friday, October 8, 2021 - link
Sorry, by "older generation" I was talking about the Amazon Graviton one, not the previous Ampere Version.The proper upgrade from the Cortex-A76 is the Cortex-A78.
The Cortex-A78 is the base micro-architecture, with the Cortex-X1 being a slightly modified derivative of it, and the Neoverse-V1 is a further slightly modified version of that. That's why I worded it in that way. Whilst ARM claims a divergence between the Cortex-A710, Cortex-X2, and Neoverse-N2... I think we will end up seeing them much more closer in-common than different.
SarahKerrigan - Friday, October 8, 2021 - link
The Graviton1 was 16 Cortex-A72 at 2.3GHz.