The 2018 GPU Benchmark Suite & the Test

Another year marks another update to our GPU benchmark suite. This time, however, is more in line with a maintenance update than it is a complete overhaul. Although we've done some extended compute and deep learning benchmarking in the past year, and even some HDR gaming impressions, our compute and synthetic lineup remains largely the same. But before getting into the details, let's start with the bulk of benchmarking, and the biggest reason for these cards anyhow: games.

Joining the 2018 game list is Far Cry 5, Wolfenstein II, Final Fantasy XV and Middle-earth: Shadow of War. We are also bringing in F1 2018 and Total War: Warhammer II. Returning from last year is Battlefield 1, Ashes of the Singularity: Escalation, and Grand Theft Auto V. All-in-all, these games span multiple genres, differing graphics workloads, and contemporary APIs, with a nod towards modern and relatively intensive games.

AnandTech GPU Bench 2018 Game List
Game Genre Release Date API(s)
Battlefield 1 FPS Oct. 2016 DX11
(DX12)
Far Cry 5 FPS Mar. 2018 DX11
Ashes of the Singularity: Escalation RTS Mar. 2016 DX12
(DX11, Vulkan)
Wolfenstein II: The New Colossus FPS Oct. 2017 Vulkan
Final Fantasy XV: Windows Edition JRPG Mar. 2018 DX11
Grand Theft Auto V Action/Open world Apr. 2015 DX11
Middle-earth: Shadow of War Action/RPG Sep. 2017 DX11
F1 2018 Racing Aug. 2018 DX11
Total War: Warhammer II RTS Sep. 2017 DX11
(DX12)

That said, Ashes as a DX12 trailblazer may not be as hot and fresh as it once was, especially considering that the pace of DX12 and Vulkan adoption in new games has waned. The circumstances are worth an investigation on their own, but the learning curve required in modern low-level API and the subsequent return may not be convincing right now. As a more general remark, most developers and publishers tend not to advertise or document DX12 support as much as they used to, nor is it clearly labelled in game specifications as many times DX11 is the unmentioned default.

Particularly for NVIDIA and GeForce RTX, pushing DXR and raytracing means pushing DX12, of which DXR is a component. The API has a backstop in the form of Xbox consoles and Windows 10, and if multi-GPU is to make a comeback, whether that's via compatible workloads (VR), flexible usage (ray tracing workload topologies), or just the plain old inevitability of Moore's Law. So this is less likely to be the slow end of DX12.

In terms of data collection, measurements were gathered either using built-in benchmark tools or with AMD's open-source Open Capture and Analytics Tool (OCAT), which is itself powered by Intel's PresentMon. 99th percentiles were obtained or calculated in a similar fashion, as OCAT natively obtains 99th percentiles. In general, we prefer 99th percentiles over minimums, as they more accurately represent the gaming experience and filter out any artificial outliers.

We've also swapped out Blenchmark, which seems to have been abandoned in terms of updates, in favor of a BMW render from the Blender Institute Cycles Benchmark, and a more recent one from a Cycles benchmark developer on Blenderartists.org. There were concerns with Blenchmark's small tile size, which is not very applicable to GPUs, and in terms of usability we also ran into some GPU detection errors which were linked to inaccurate Blenchmark Python code.

Otherwise, we are also keeping an eye on a few trends and upcoming developments:

  • MLPerf machine learning benchmark suite
  • Blender Benchmark
  • Futuremark's 3DMark DirectX Raytracing benchmark
  • DXR and Vulkan raytracing extension support in games

Another point is that we do not have a permanent HDR monitor for our testbed, which would be necessary to incorporate HDR game testing in the near future; 5 games in our list actually support HDR. And as we look at technologies that enhance or alter image quality (e.g. HDR, Turing's DLSS), we will want to find a better way of comparing differences. This is particularly tricky with HDR as screenshots are inapplicable and even taking accurate photographs will most likely be viewed on an SDR screen. With DLSS, there is a built-in reference quality based on 64x supersampling, which in deep learning terms is the 'ground truth'; an intuitive solution would be to use a neural network based method of analyzing quality differences, but that is likely beyond our scope.

The following tech demos and test applications were provided via NVIDIA:

  • Star Wars 'Reflections' Demo (includes real time ray tracing and DLSS support)
  • Final Fantasy XV Official Benchmark (includes DLSS support)
  • Asteroids Demo (features mesh shading and variable LOD)
  • Epic Infiltrator Demo (features DLSS)

The Testbed

Because NVIDIA is not productizing any other reference-quality GeForce RTX 2080 Ti and 2080 card besides the Founders Editions, which are non-reference by specifications, we've gone ahead and emulated the true reference specifications with a 90MHz downclock and lowering the TDP by roughly 10W. This is to keep comparisons standardized and apples-to-apples, as we always look at reference-to-reference results.

In a classic case of Murphy's Law, our usual PSU started malfunctioning around the time of the review, but given the time constraints we couldn't do a 1:1 replacement in time. As it is a digital PSU, we were beginning to use it for PCIe power readings to augment system measurements, but for now we will have to stick power draw at the wall. For the time being, we've swapped it out with another high-quality and high-wattage PSU.

CPU: Intel Core i7-7820X @ 4.3GHz
Motherboard: Gigabyte X299 AORUS Gaming 7 (F9g)
Power Supply: Corsair AX860i
EVGA 1000 G3
Hard Disk: OCZ Toshiba RD400 (1TB)
Memory: G.Skill TridentZ DDR4-3200 4 x 8GB (16-18-18-38)
Case: NZXT Phantom 630 Windowed Edition
Monitor: LG 27UD68P-B
Video Cards: AMD Radeon RX Vega 64 (Air Cooled)
NVIDIA GeForce RTX 2080 Ti Founders Edition
NVIDIA GeForce RTX 2080 Founders Edition

NVIDIA GeForce GTX 1080 Ti Founders Edition
NVIDIA GeForce GTX 1080 Founders Edition
NVIDIA GeForce GTX 980 Ti
NVIDIA GeForce GTX 980
Video Drivers: NVIDIA Release 411.51 Press
AMD Radeon Software Adrenalin Edition 18.9.1
OS: Windows 10 Pro (April 2018 Update)
Spectre/Meltdown Mitigations Yes, both
Meet The GeForce RTX 2080 Ti & RTX 2080 Founders Editions Cards Battlefield 1
Comments Locked

337 Comments

View All Comments

  • Spunjji - Friday, September 21, 2018 - link

    If that were true then Nvidia could have left off the RTX parts this time around and created a GPU that offers a simple ~30% performance improvement at roughly the same retail cost.

    Following that, the die-area benefits from 7nm could have been spent on both RTX features and another ~30% performance boost at a similar or slightly-higher cost. By then they could probably have added enough resources to at least manage high refresh rates at 1080p, if not 2.5K

    Instead they massively inflated their die for features that require you to accept resolutions and frame-rates that PC gaming left behind 6 years ago.
  • mapesdhs - Wednesday, September 26, 2018 - link

    That last sentence is something I which tech sites would emphasise a lot more. It very much defines how those who normally buy into the higher tier tech now regard what they like doing and why. NVIDIA pushed hard to create the market for high-refresh gaming, 4K & VR, now suddenly they're trying to do an about-face. I can't see how it can work. I just bought a 27" 1440p IPS panel for 200 UKP, the cost of good screens has come down a lot, and now NVIDIA wants us to drop back down to 1080p? :D I get the impression the reaction of a great many is just laughter.
  • Gastec - Thursday, September 27, 2018 - link

    Ahaa! You are getting close :) Come on, just spell it: they want to "milk" us as much as possible before Moore's Law ends and we will completely stop upgrading our PC's and we'll just replace the defective part twice in a life time. No more billions of moneyz for Corporate Commander :)
  • Yojimbo - Wednesday, September 19, 2018 - link

    "Please, let's boycott Turing cards."

    Throw down your chains and resist!
  • Ranger1065 - Thursday, September 20, 2018 - link

    100%
  • Xex360 - Wednesday, September 19, 2018 - link

    These cards are a disappointment for the price, the 2080ti should be priced at most 800$, it just doesn't offer the performance required for justifying its price, worse here they compared it to the 1080ti FE which as GamerNexus pointed out is not ideal, for the cards are noticeably slower than other cards with proper cooling, so the 1080ti is at least as fast as the 2080.
    On the ray tracing side, I like the technology but it's not impressive enough to justify the hefty price tag, I'd rather have a real generational leap with a 2070 beating a 1080ti and a 2080ti having at least 70% more performance than having RT, it's a niche product and obly few games will benefit from it, and the whole DLSS isn't good either limited to only a few games, with more brut force we could achieve 4k and super sampling.
  • kron123456789 - Wednesday, September 19, 2018 - link

    "I'd rather have a real generational leap with a 2070 beating a 1080ti and a 2080ti having at least 70% more performance than having RT"
    That reminded me of a very old quote:
    "If I had asked people what they wanted, they would have said ‘faster horses.’" — Henry Ford
  • saikrishnav - Wednesday, September 19, 2018 - link

    That quote only makes sense if Nvidia came up with a "different" radical product than a graphical horse. They just made a slightly faster horse with a RTX ON button which nobody is ready to push yet i.e. developers. So, if you have a choice between a much faster horse and a RTX ON button - one would take a much faster horse. Now, when developers are ready to push the button/envelope, and sign on to the RTX, then this quote makes sense. Nvidia is asking customers to pay the price of new tech-adoption without show-casing the products that use it. They could have invested with devs and in games, to use the RTX, and then released it. But no, they want to fill in a gap until 7nm arrives.
  • Yojimbo - Wednesday, September 19, 2018 - link

    Nobody was ready to push the mass produced automobile button, yet, either. Do you think Ford started mass producing cars and then immediately there were roads and gas stations? No, at first horses could comfortably go many more places than cars could.

    His quote is entirely appropriate.

    There is no gap to fill before 7 nm arrives since AMD will have no competition. NVIDIA introduced this now because they see value in the product which will generate sales. Plus it will get the ball rolling on developers implementing the new technologies that are present in the architecture and will be present in future NVIDIA architectures.
  • BurntMyBacon - Thursday, September 20, 2018 - link

    Have to agree here. No only where automobiles extremely limited in where they could go on introduction, they were also very loud and considered disruptive to society with a large voice of opposition. These new cards at least have the benefit of being able to go anywhere their predecessors can while still enabling new capabilities.

    I very much agree that nVidia is using this architecture to "get the ball rolling" on the new tech. They are probably very much aware that sales of RTX cards will be lower until they can fit a meaningful amount of the new hardware resources into a mainstream chip. Though, given the size of the chips and typical associated yields, nVidia may still end up selling every chip they can make.

Log in

Don't have an account? Sign up now