The new Opteron 6300: Finally Tested!
by Johan De Gelas on February 20, 2013 12:03 AM ESTLS-DYNA Power Consumption
For HPC buyers, peak power tends to be a very important metric. As HPC systems are run at close to or equal to 100% CPU load, the energy consumption is at its peak for a long time. Peak power thus also determines the cooling and energy requirements. This is in sharp contrast with most other servers, where calculating the power and amps based on the peak load of a complete rack is considered wasteful as it is highly unlikely that all servers will hit 100% CPU load at the same time. We took the 95th percentile of our power numbers.
Note that the Xeon E5 numbers are not directly comparable to the Opteron numbers as the CPUs are tested in servers with different form factors. We will tackle that in the next test. Let us focus on the Opteron results for now.
AMD has made some real progres here. At the same clock, the total power consumption is 6% lower. Even at a 200MHz higher clock the peak power is very slightly—but consistently—lower (2%).
Of course, we also want to compare the AMD and Intel CPUs directly. To do this, we always run the fans at maximum speed. That way, the fans always consume the same amount of power. We then test with one and two CPUs, while keeping the amount of memory (64GB) the same. This way we measure how much extra power you consume at the wall when you add a second CPU. This number thus includes the voltage regulators (which can amount to up to 10% of the total server power) and the PSU inefficiency.
The Intel Xeon has a TDP of 95W, but even with a very FP intensive application it does not get anywhere near that number. About 75W out of those 94W are consumed by the CPU, as measured by our Hardware Monitoring Software that reads out the MSRs. We are still working on our version for the AMD platform (AMD's documentation is a bit late), but we estimate that the Opteron 6376 consumes about 110W and the Opteron 6380 needs about 120W. That means that AMD's top CPUs are probably consuming a bit more than their TDP indicates if you push the FP unit hard.
We also tried to measure idle power. Take the numbers with a grain of salt, but we measured about 19-20W for the Opteron 6380 (p-states disabled), 17-18W for the Opteron 6376 and 16-17W for the Xeon.
Facebook
Twitter
RSS
55 Comments
View All Comments
JohanAnandtech - Wednesday, February 20, 2013 - link
Per Core. So with the -np 32 setting. I have tried less before, but the LS-DYNA really likes the extra load and store units of the second unit. so 32 MPI processes give a 30% boost.alpha754293 - Wednesday, February 20, 2013 - link
That's a healthy boost! It's amazing how these technologies are maturing to the point where conventional wisdom that they might be starved for FPU resources isn't enough to slow them down.It'd be interesting to see whether it makes a difference if you were to let the OS handle the job/process scheduling or whether manual intervention can help reduce some of the thread/process migration overhead, especially across 16 FPUs.
dmytty - Friday, February 22, 2013 - link
Before the benchmarks came in, I looked at an HPC build for the new 6300 series. I saw the real sweet spot for AMD being the 6344 which is a CPU of type 2.6 Ghz @ 12 or 6 core (depending how you define a 'core'). Anandtech never mentioned this CPU in the review. (?)In simple $ terms for the CPU, it's the E5 2640 @ $815 vs the Opteron 6344 @ $415. So how does AMD not have a decisive price advantage?
In 4S land the price advantage widens.
AMD 4S...I priced a build (not including a case) at ~$3860 for 4 x 6344, 128 GB RAM, mobo and PSU. Note that this mobo could also go out to 256 GB. CPU cost is 4 x 6344 = $1660. Again, total system cost was $3860.
Intel 4S...I picked the 4607 as being the best 'bang for the buck processor'. It's 6 cores @ 2.2 Ghz. However, at $885 per 4607 processor the CPU cost is more than double than AMD (ie 4x Intel 4607 = $3540). The same build cost using the Intel 4607 would then be $5740.
AMD @ $3860 vs Intel @ $5740. Why did Anandtech not talk about 4S? Why no mention of the 6344?
dmytty - Friday, February 22, 2013 - link
I forgot to mention that the Intel board I specified for 4S build was $1200 whereas the AMD board was $800.A marketing person would call the system cost comparison ~$3k vs $~6k.
Again, am I missing something?
The AMD 6344 based 4S system has a ~19% clockspeed advantage and comes in at 62% of the cost.
Worried about electrical cost? You can buy 700 watts of PV solar panels with the cost savings between AMD and Intel. 700 watts peak = ~3.5 kwh daily output. That would nicely mitigate (and then some) the 20 watts/cpu difference (80 watts total with 4x cpu) between AMD 6344 and Intel 4607. You would net about 1.5 kwh/day electricity going with the AMD + PV over the slightly more efficient Intel.
geok1ng - Monday, May 19, 2014 - link
this is all and good, but looking at spec int 2006 results, Abu Dhabi still does not match performance/watt of Magni Cours opterons. And the best competition AMd can offer against 2 gen old Xeons is still the Opteron 6180.