Power Consumption, Thermals, and Noise

Whenever we’ve tested big processors in the past, especially those designed for the super high power consumption tasks, they’ve all come with pre-prepared test systems. For the 28-core Intel Xeon W-3175X, rated at 255W, Intel sent a whole system with a 500 W Asetek liquid cooler as well as a second liquid cooler in case we were overclocking the system. When I tested the Core i9-9990XE, a 14-core 5 GHz ‘auction-only’ processor, the system integrator shipped a full 1U server with a custom liquid cooler to deal with the 400W+ thermals.

As with the Armari Magnetar X64T, the custom liquid cooling set up is an integral part of the system offering in order to achieve the high overclocked frequencies that the company promises.

Armari calls the solution its FWLv2, ‘designed to support a fully unlocked Threadripper 3990X at maximum PBO with all 64 cores sustained up to 4.1 GHz’. The solution consists of a custom monoblock created in partnership with EKWB designed to fit both the CPU and the VRM on the ASRock motherboard specifically. There is also additional convective heatsinks for the VRM to aid in additional cooling as the system also offers airflow through the chassis. This cooling loop hooks up to an EKWB Coolstream 420x45mm triple radiator, three EK-Vardar 140ER EVO fans, and a high-performance cooling pump with a custom performance profile. Armari claims a 3x better flow-rate than the best all-in-one liquid cooling solution on the market, with 200% better cooling performance and a lower noise profile (at a given power) due to the chassis design.

The system also includes two internal 140mm 1500 RPM Noctua fans, for additional airflow over the add-in components, and an angled 80mm low-noise SanAce fan mounted specifically for the memory and the VRM area of the motherboard.

As mentioned on the first page, the 420x45mm radiator is mounted on a swing arm inside the custom chassis. This makes it very easy to open the side of the case and perform maintenance. The chassis is a mix of aluminum inside and a steel frame, and weighs 18 kg / 39.7 lbs, but has handles on the top that hide inside the case, making it very easy to move but also look flush with the design. To be honest, this is a very nice chassis – it’s big, but given what it has to cool and the workstation element of it all, it is more than suitable. Externally, there are no RGB LEDs – a simple light on the top for the power/reset buttons, and blue accents at the front.

As you can probably see inside, there’s no aesthetic to pander to, especially when these systems are only meant be opened for maintenance. The standard Armari 3-year warranty for the UK (1year RTB, 2/3rd year parts + labor) includes a free full-system checkup and coolant replacement during that period.

With all that said, an overclocked 3990X is a bit of a beast, both in power consumption and cooling requirements. Armari told us going into this review that we’ll likely see a series of different power consumptions based on the workload, especially when it comes to sustained codes.

Our normal look into power consumption is typically with our y-cruncher test, which deals solely in integers:

For this test, the system was CPU over 400 W from start to finish, and the peak power was 505 W. CPU temperature averaged at 70 ºC and peaked at 82 ºC, with the CPU frequency average at 4002 MHz.

For a less dense workload that involves a mixture of math, we turn to our Agisoft test. This involves converting 2D images to 3D models, and involves four algorithmic stages – some fully multi-threaded, and others that are more serially coded.

The bulk of the test was done at around 270 W, with a single peak at 375 W. CPU temperatures never broke 50ºC.

Where we saw the real power was in our 3DPMavx floating point math test. This uses AVX2 like y-cruncher, but in a much denser calculation.

So this test runs a loop for 10 seconds, then idles for 10 seconds, hence the up and down part. There are six different loops, so each one is having a different effect on the power based on the instruction density. The test then repeats – I’ve cut the graph at 300 seconds to get a clear view

The peak power is 640 W – there is no rest when the workload is this heavy, and the CPU very quickly idles down to 70 W where possible. The peak temperature with a workload this heavy, even in small 10 second bursts, was 89ºC. Depending on the exact nature of the instructions, we saw sustained all-core frequencies at 3925 MHz to 4025 MHz.

As another angle to this story, I ran a script to plot the power consumption during the AIDA64 integer stress test, and cycled through 0 threads loaded up to 256 threads loaded, with two minutes load followed by two minutes idle.

The power consumption peaks at 450 W, as with the previous integer test, and we can see a slow steady rise from 106 W with one thread loaded up to 64 threads loaded. In this instance, AIDA64 is clever enough to split threads onto separate cores. In a normal 3990X scenario, we’re going to be seeing about 3.2 W per core at full load – in this instance, we’re approaching 6 W per core. For floating point math, where we see those 640 W peaks, it’s closer to 9 W per core. Bearing in mind that some of the consumer Ryzen Zen 2 processors are similarly running 9-12 W per core at full load, this is a bit wild.

Now, mentioning something like 640 W running at 10 seconds already got the CPU to 89ºC, the next question is what happens to the system when that load is sustained. Depending on the use case, some software might focus on INT workloads while others prefer FP. I attached a wall power meter to the system and fired up an 8K Blender render, and left the system on for over 10 minutes.

As you can see from the video, the software starts at around 4050 MHz, and slowly decreases over time to keep everything in check to about 3850 MHz. During this time, the system averaged about 900 W at the wall, with ~935 W peak. In this time, the temperature didn’t go above 92ºC, and with an audio meter at a distance of 1 ft, I measured 45-49 dB (compared to an idle of 36 dB).

To pile even more on, I turned to software that could load up the overclocked processor as well as the Quadro RTX 6000 in the system. TheaRender is our benchmark of choice here – it’s a physically based global illumination render that can find how many samples per pixel it can calculate in any given time. It works on CPU and GPU simultaneously.

This pushed the system hard. The benchmark can take up to 20 minutes or more, and the wall meter peaked at 1167 W for the full system. This was the one and only time I heard the cooling fans kick into high gear, at 52-55 dB. Thermals on the CPU were measured at 96 ºC, which seems to be a pseudo-ceiling. Even with that said, the processor was still running at 3850 MHz all-core.

I was running the system in a 100 sq ft room, and so pushing 1000 W for a long time without adequate air flow will warm up the room. I left my benching scripts on overnight, especially the per-thread power loading, to notice the next morning the room was warm. For any users planning to run sustained workloads in a small office, take note. Even though the system is much quieter than other workstation-class systems I’ve tested, should there be an option to place the system in an air-conditioned environment external to the work desk, this might be preferable.

On stability, throughout all of our testing, there was nothing to mention - there wasn't a single hint of instability. On speaking with Armari, the company said that this is down to AMD's own internal DVFS when implmenting a high level PBO-based overclock: I was told that because this system was built from the ground up to accommodate this power, along with the custom tweaks, and the fact that AMD's voltage and frequency tracking metrics always ensured a stable system (as long as the temperature and BIOS is managed), then they can build 10, 20, or 50 systems in a row and not experience any issues. All systems are pre-tested extensively before shipping with customer-like workloads as well. 

Science and Simulation Performance Armari Magnetar X64T: Cinebench go brrrr
Comments Locked

96 Comments

View All Comments

  • Spunjji - Friday, September 11, 2020 - link

    There's nothing more sad than a lack of imagination.
  • Everett F Sargent - Wednesday, September 9, 2020 - link

    "Unfortunately due to how quickly this system was rebuilt for this review, the system I was sent was using DDR4-3200 at CL20, as some of the original memory was accidentally splashed with coolant, and Armari wanted to ensure I wouldn’t have any issues with the system."

    Unfortunately, they could not give one of these systems. In fact, they would have to give me $15K and this system, call it a warranty in reverse. :)
  • PeachNCream - Thursday, September 10, 2020 - link

    That's the sort of trouble you have with using a little company like this as a supplier. They toss together shoddy bits and pieces because they don't retain proper inventory to support builds (and likely cannot afford to do so) and then for press-related reviews, they make absolutely sure there will not be issues with stability. Can we be certain they take the same care with hardware sold to non-journalist buyers or do those customers get the coolant-soaked memory and less stability testing? Who ensures there is adequate quality control in a smaller system builder like this?

    Chasing so-called "world records" in benchmarks in the hopes of increasing productivity by purchasing little-builder-overclocked hardware is asking to shoot your own business operation in the foot when the hardware fails. I would be very reluctant to trust my company's workload on out-of-spec computers cobbled together by a company that can't even be bothered to keep coolant fluids off the memory.
  • Ian Cutress - Thursday, September 10, 2020 - link

    I received this system three or four days ago - I wasn't even in contact with the company a week ago. I suggested a review if a system was ready, and so they scrambled to action. Obviously they weren't going to send me a system that a customer had already ordered, delaying someone with a financial contract, and so they pulled out the system they built for the record and wanted re-test and update so I had the latest components. Best laid plans and all that, combined with a quick turn-around requirement, might lend more readily to non-ideal situations, but given what was done in that time (I was told that included a couple of engineers 7am-11pm the day before), it was actually very well executed.
  • Spunjji - Friday, September 11, 2020 - link

    "...by a company that can't even be bothered to keep coolant fluids off the memory."
    God forbid you ever see what goes on in the large system integrators' build rooms..!

    Accidents happen, it's how they're dealt with that marks a bad supplier from a good one. "Something went wrong and we had to alter the spec" is infinitely better than "Oh well, the fluid's mostly inert, bung the RAM in and hope it works and blame the customer if it goes wrong."
  • Arutius - Friday, September 11, 2020 - link

    I applaud this Company for its honesty and hustle. They are a small organization that cares about what they do. They appreciated Ian being available to them as a small business and are thankful for the exposure. Many readers here work in large organizations and have no appreciation for the real-world stress ( what is the balance of the checkbook today and where will we be in 1 month for cash and sales) on small businesses in this hyper-competitive atmosphere. Try sitting in the chair of the person who personally values each customer, who without they are shortly destined to fail.
  • MrVibrato - Thursday, September 10, 2020 - link

    Yeah, curiously they couldn't replace the "splashed" memory with modules of the same type. There are a small company and i don't expect them to have big inventory. If they don't have stock of these memory modules for at least two or three rigs like these, it tells me this is a cash-strapped operation that probably pays part orders from the advance payments from customers ordering one of their systems. And such vendors are always a major head-ache, because you pay for the system and then you wait and wait and don't get your ordered goods because the vendor struggles to place some order for some component with some supplier, not to mention what could happen if you need to do a RMA or warranty exchange through such a vendor. Mind you, i cannot tell with certainty that Armari is such a company. But this occurence would be a major red flag for me if i were shopping for a boutique workstation (as little sense as that would make, as Peach has explained already; and i would also add uncertainty about the ability to do quick-turnaround on-premises support/service, which larger vendors -while occassionally also being a bitch about- usually don't struggle with in and around large metropolitan areas wherever in the country)
  • MrVibrato - Thursday, September 10, 2020 - link

    Oh well, *There* should be *They*, and

    *it tells me this is a cash-strapped operation*

    should be

    *it tells me this is perhaps a cash-strapped operation*
  • MrVibrato - Thursday, September 10, 2020 - link

    Never mind what i just wrote. I should have checked their website beforehand. Their product range seems to big and too broad to be one of those no-inventory boutique vendors. Mea culpa!
  • Spunjji - Friday, September 11, 2020 - link

    Perhaps check before leaving the big stinky comment full of bold assumptions on the small system integrator's offering?

Log in

Don't have an account? Sign up now