The 2017 GPU Benchmark Suite & the Test

Paired with our RX Vega 64 and 56 review is a new benchmark suite and new testbed. The 2017 GPU suite features new games, as well as new compute and synthetic benchmarks.

Games-wise, we have kept Grand Theft Auto V and Ashes of the Singularity: Escalation. Joining them is Battlefield 1, DOOM, Tom Clancy’s Ghost Recon Wildlands, Warhammer 40,000: Dawn of War III, Deus Ex: Mankind Divided, F1 2016, and Total War: Warhammer. All-in-all, these games span multiple genres, differing graphics workloads, and contemporary APIs, with a nod towards modern and relatively intensive games. Additionally, we have retired the venerable Crysis 3 as our mainline power-testing game in favor of Battlefield 1.

AnandTech GPU Bench 2017 Game List
Game Genre Release Date API(s)
Battlefield 1 FPS Oct. 2016 DX11
(DX12)
Ashes of the Singularity: Escalation RTS Mar. 2016 DX12
(DX11)
DOOM (2016) FPS May 2016 Vulkan
(OpenGL 4.5)
Ghost Recon Wildlands FPS/3PS Mar. 2017 DX11
Dawn of War III RTS Apr. 2017 DX11
Deus Ex: Mankind Divided RPG/Action/Stealth Aug. 2016 DX11
(DX12)
Grand Theft Auto V Action/Open world Apr. 2015 DX11
F1 2016 Racing Aug. 2016 DX11
Total War: Warhammer TBS/Real-time tactics May 2016 DX11 + DX12

In terms of data collection, measurements were gathered either using built-in benchmark tools or with AMD's open-source Open Capture and Analytics Tool (OCAT), which is itself powered by Intel's PresentMon. 99th percentiles were obtained or calculated in a similar fashion: OCAT natively obtains 99th percentiles, GTA V's built-in benchmark include 99th percentiles, and both Ashes: Escalation and Total War: Warhammer's built-in benchmark outputs raw frame time data. Dawn of War III continutes to suffer from its misconfigured built-in benchmark calculations and so its native data cannot be used. In general, we prefer 99th percentiles over minimums, as they more accurately represent the gaming experience and filter out outliers that may not even be true results of the graphics card.

We are continuing to use the best API for a given card when given a choice. As before, we use DirectX 12 for Ashes of the Singularity: Escalation, being natively designed for that API. For DOOM (2016), using Vulkan is an improvement to OpenGL 4.5 across the board, and for those not in-the-know, Vulkan is roughly comparable to OpenGL in the same way DX12 is to DX11. We also stick to DX11 for Battlefield 1, with the persistent DX12 performance issues in mind, and similar reasoning follows with Deus Ex: Mankind Divided, where DX12 did not appear to give the best performance for RX Vega.

In the same vein, we have used DX12 for Total War: Warhammer when testing AMD cards, but we are still sussing out the exact effects on the Vega cards. With Vega running Total War: Warhammer, neither API seems to be absolutely better performing than the other, and we are continuing to investigate.

2017 GPU Compute and Synthetics

We have also updated our compute and synthetics suites, which are now as follows:

  • Compute: Blender 2.79 - BlenchMark
  • Compute: CompuBench 2.0 – Level Set Segmentation 256
  • Compute: CompuBench 2.0 – N-Body Simulation 1024K
  • Compute: CompuBench 2.0 – Optical Flow
  • Compute: Folding @ Home Single Precision
  • Compute: Geekbench 4 – GPU Compute – Total Score
  • Synthetics: TessMark, Image Set 4, 64x Tessellation
  • Synthetics: VRMark Orange
  • Synthetics: Beyond3D Suite – Pixel Fillrate
  • Synthetics: Beyond3D Suite – Integer Texture Fillrate (INT8)
  • Synthetics: Beyond3D Suite – Floating Point Texture Fillrate (FP32)

Testing with Vega

Testing was done with default configurations with respect to the High-Bandwidth Cache Controller (HBCC) and BIOS/power profiles. By default, HBCC is disabled in Radeon Software. As for power profiles, both Vega 64 and 56 come with primary and secondary VBIOS modes, each having three profiles in WattMan: Power Saver, Balanced, and Turbo. By default, both cards use the primary VBIOS' Balanced power profile.

GPU Power Limits for RX Vega Power Profiles
  Radeon RX Vega 64 Air Radeon RX Vega 56
Primary VBIOS Secondary VBIOS Primary VBIOS Secondary VBIOS
Power Saver 165W 150W 150W 135W
Balanced 220W 200W 165W 150W
Turbo 253W 230W 190W 173W

A small switch on the cards can be toggled away from the PCIe bracket for the lower power secondary VBIOS. In Radeon WattMan, a slider permits switching between Power Saver, Balanced, Turbo, and Custom performance profiles. In total, each card has six different power profiles to choose from. RX Vega 64 Liquid has its own set of six profiles as well, ranging from 165W to 303W. We don't expect Turbo mode to significantly change results: for Turbo vs. Balanced, AMD themselves cited a performance increase of about 2% at 4K.

The New 2017 GPU Skylake-X Testbed

Last, but certainly not least, we have a new testbed running these benchmarks and games. For that reason, historical results cannot be directly compared with the results in this review.

CPU: Intel Core i7-7820X @ 4.3GHz
Motherboard: Gigabyte X299 AORUS Gaming 7
Power Supply: Corsair AX860i
Hard Disk: OCZ Toshiba RD400 (1TB)
Memory: G.Skill TridentZ DDR4-3200 4 x 8GB (16-18-18-38)
Case: NZXT Phantom 630 Windowed Edition
Monitor: LG 27UD68P-B
Video Cards: AMD Radeon RX Vega 64 (Air Cooled)
AMD Radeon RX Vega 56
AMD Radeon RX 580
AMD Radeon R9 Fury X
NVIDIA GeForce GTX 1080 Ti Founders Edition
NVIDIA GeForce GTX 1080 Founders Edition
NVIDIA GeForce GTX 1070 Founders Edition
Video Drivers: NVIDIA Release 384.65
AMD Radeon Software Crimson ReLive Edition 17.7.2 (for non-Vega cards)
AMD Radeon Software Crimson Press Beta 17.30.1051
OS: Windows 10 Pro (Creators Update)
Competitive Positioning, Radeon Packs, & Crytocurrency Demand Battlefield 1
Comments Locked

213 Comments

View All Comments

  • BrokenCrayons - Monday, August 14, 2017 - link

    The hypothetical APU that contains Zen, Polaris/Vega, and HBM2 would be interesting if AMD can keep the power and heat down. Outside of the many cores Threadripper, Zen doesn't do badly on power versus performance so something like 4-6 CPU cores plus a downclocked and smaller GPU would be good for the industry if the package's TDP ranged from 25-95W for mobile and desktop variants.

    By itself though, Vega is an inelegant and belated response to the 1080. It shares enough in common with Fiji that it strikes me as an inexpensive (to engineer) stopgap that tweaks GCN just enough to keep it going for one more generation. I'm hopeful that AMD will have a better, more efficient design for their next generation GPU. The good news is that with the latest product announcements, AMD will likely avoid bankruptcy and get a bit healthier looking in the near term. Things were looking pretty bad for them until Ryzen's announcement, but we'll need to see a few more quarters of financials that ideally show a profit in order to be certain the company can hang in there. I'm personally willing to go out on a limb and say AMD will be out of the red in Q1 of FY18 even without tweaking the books on a non-GAAP basis. Hopefully, they'll have enough to pay down liabilities and invested in the R&D necessary to stay competitive. With process node shrinks coming far less often these days, there's an several years' long opening for them right now.
  • TheinsanegamerN - Monday, August 14, 2017 - link

    " It shares enough in common with Fiji that it strikes me as an inexpensive (to engineer) stopgap that tweaks GCN just enough to keep it going for one more generation. "

    We thought the same thing about polaris. I think the reality is that AMD cannot afford to do a full up arch, and can only continue to tweak GCN in an attempt to stay relevant.

    They still have not done a Maxwell-Esq redesign of their GPUs streamlining them for consumer use. They continue to put tons of compute in their chips which is great, but it restricts clock rates and pushes power usage sky high.
  • mapesdhs - Monday, August 14, 2017 - link

    I wonder if AMD decided it made more sense to get back into the CPU game first, then focus later on GPUs once the revenue stream was more healthy.
  • Manch - Tuesday, August 15, 2017 - link

    Just like there CPU's it's a jack of all trades design. Cheaper R&D to use one chip for many but you got to live with the trade offs.

    The power requirement doesn't bother me. Maybe after the third party customs coolers, I'll buy one if it's the better deal. I have a ventilated comm closet. All my equipment stays in there, including my PCs. I have outlets on the wall to plug everything else into. Nice and quiet regardless of what I run.
  • Sttm - Monday, August 14, 2017 - link

    That Battlefield 1 Power Consumption with Air, is that actually correct? 459 watts.... WTF AMD.
  • Aldaris - Monday, August 14, 2017 - link

    Buggy driver? Something is totally out of whack there.
  • Ryan Smith - Monday, August 14, 2017 - link

    Yes, that is correct.

    I also ran Crysis 3 on the 2016 GPU testbed. That ended up being 464W at the wall.
  • haukionkannel - Monday, August 14, 2017 - link

    Much better than I expected!
    Nice to see competition Also in GPU highend. I was expecting the Vega to suffer deeply in DX11, but it is actuallu doing very nice in those titles... I am really surpriced!
  • Leyawiin - Monday, August 14, 2017 - link

    A day late and a dollar short (and a power pig at that). Shame. I was hoping for a repeat of Ryzen's success, but they'll sell every one they make to miners so I guess its still a win.
  • Targon - Monday, August 14, 2017 - link

    I would love to see a proper comparison between an AMD Ryzen 7 and an Intel i7-7700k at this point with Vega to see how they compare, rather than testing only on an Intel based system, since the 299X is still somewhat new. All of the Ryzen launch reviews were done on a new platform, and the AMD 370X is mature enough where reviews will be done with a lot more information. Vega is a bit of a question mark in terms of how well it does when you compare between the two platforms. Even how well drivers should have matured in how well the 370X chipset deals with the Geforce 1080 is worth looking at in my opinion.

    I've had the thought, without resources, that NVIDIA drivers may not do as well on an AMD based machine compared to an Intel based machine, simply because of driver issues, but without a reasonably high end video card from AMD, there has been no good way to do a comparison to see if some of the game performance differences between processors could have been caused by NVIDIA drivers as well.

Log in

Don't have an account? Sign up now