The 2018 GPU Benchmark Suite & the Test

Another year marks another update to our GPU benchmark suite. This time, however, is more in line with a maintenance update than it is a complete overhaul. Although we've done some extended compute and deep learning benchmarking in the past year, and even some HDR gaming impressions, our compute and synthetic lineup remains largely the same. But before getting into the details, let's start with the bulk of benchmarking, and the biggest reason for these cards anyhow: games.

Joining the 2018 game list is Far Cry 5, Wolfenstein II, Final Fantasy XV and Middle-earth: Shadow of War. We are also bringing in F1 2018 and Total War: Warhammer II. Returning from last year is Battlefield 1, Ashes of the Singularity: Escalation, and Grand Theft Auto V. All-in-all, these games span multiple genres, differing graphics workloads, and contemporary APIs, with a nod towards modern and relatively intensive games.

AnandTech GPU Bench 2018 Game List
Game Genre Release Date API(s)
Battlefield 1 FPS Oct. 2016 DX11
(DX12)
Far Cry 5 FPS Mar. 2018 DX11
Ashes of the Singularity: Escalation RTS Mar. 2016 DX12
(DX11, Vulkan)
Wolfenstein II: The New Colossus FPS Oct. 2017 Vulkan
Final Fantasy XV: Windows Edition JRPG Mar. 2018 DX11
Grand Theft Auto V Action/Open world Apr. 2015 DX11
Middle-earth: Shadow of War Action/RPG Sep. 2017 DX11
F1 2018 Racing Aug. 2018 DX11
Total War: Warhammer II RTS Sep. 2017 DX11
(DX12)

That said, Ashes as a DX12 trailblazer may not be as hot and fresh as it once was, especially considering that the pace of DX12 and Vulkan adoption in new games has waned. The circumstances are worth an investigation on their own, but the learning curve required in modern low-level API and the subsequent return may not be convincing right now. As a more general remark, most developers and publishers tend not to advertise or document DX12 support as much as they used to, nor is it clearly labelled in game specifications as many times DX11 is the unmentioned default.

Particularly for NVIDIA and GeForce RTX, pushing DXR and raytracing means pushing DX12, of which DXR is a component. The API has a backstop in the form of Xbox consoles and Windows 10, and if multi-GPU is to make a comeback, whether that's via compatible workloads (VR), flexible usage (ray tracing workload topologies), or just the plain old inevitability of Moore's Law. So this is less likely to be the slow end of DX12.

In terms of data collection, measurements were gathered either using built-in benchmark tools or with AMD's open-source Open Capture and Analytics Tool (OCAT), which is itself powered by Intel's PresentMon. 99th percentiles were obtained or calculated in a similar fashion, as OCAT natively obtains 99th percentiles. In general, we prefer 99th percentiles over minimums, as they more accurately represent the gaming experience and filter out any artificial outliers.

We've also swapped out Blenchmark, which seems to have been abandoned in terms of updates, in favor of a BMW render from the Blender Institute Cycles Benchmark, and a more recent one from a Cycles benchmark developer on Blenderartists.org. There were concerns with Blenchmark's small tile size, which is not very applicable to GPUs, and in terms of usability we also ran into some GPU detection errors which were linked to inaccurate Blenchmark Python code.

Otherwise, we are also keeping an eye on a few trends and upcoming developments:

  • MLPerf machine learning benchmark suite
  • Blender Benchmark
  • Futuremark's 3DMark DirectX Raytracing benchmark
  • DXR and Vulkan raytracing extension support in games

Another point is that we do not have a permanent HDR monitor for our testbed, which would be necessary to incorporate HDR game testing in the near future; 5 games in our list actually support HDR. And as we look at technologies that enhance or alter image quality (e.g. HDR, Turing's DLSS), we will want to find a better way of comparing differences. This is particularly tricky with HDR as screenshots are inapplicable and even taking accurate photographs will most likely be viewed on an SDR screen. With DLSS, there is a built-in reference quality based on 64x supersampling, which in deep learning terms is the 'ground truth'; an intuitive solution would be to use a neural network based method of analyzing quality differences, but that is likely beyond our scope.

The following tech demos and test applications were provided via NVIDIA:

  • Star Wars 'Reflections' Demo (includes real time ray tracing and DLSS support)
  • Final Fantasy XV Official Benchmark (includes DLSS support)
  • Asteroids Demo (features mesh shading and variable LOD)
  • Epic Infiltrator Demo (features DLSS)

The Testbed

Because NVIDIA is not productizing any other reference-quality GeForce RTX 2080 Ti and 2080 card besides the Founders Editions, which are non-reference by specifications, we've gone ahead and emulated the true reference specifications with a 90MHz downclock and lowering the TDP by roughly 10W. This is to keep comparisons standardized and apples-to-apples, as we always look at reference-to-reference results.

In a classic case of Murphy's Law, our usual PSU started malfunctioning around the time of the review, but given the time constraints we couldn't do a 1:1 replacement in time. As it is a digital PSU, we were beginning to use it for PCIe power readings to augment system measurements, but for now we will have to stick power draw at the wall. For the time being, we've swapped it out with another high-quality and high-wattage PSU.

CPU: Intel Core i7-7820X @ 4.3GHz
Motherboard: Gigabyte X299 AORUS Gaming 7 (F9g)
Power Supply: Corsair AX860i
EVGA 1000 G3
Hard Disk: OCZ Toshiba RD400 (1TB)
Memory: G.Skill TridentZ DDR4-3200 4 x 8GB (16-18-18-38)
Case: NZXT Phantom 630 Windowed Edition
Monitor: LG 27UD68P-B
Video Cards: AMD Radeon RX Vega 64 (Air Cooled)
NVIDIA GeForce RTX 2080 Ti Founders Edition
NVIDIA GeForce RTX 2080 Founders Edition

NVIDIA GeForce GTX 1080 Ti Founders Edition
NVIDIA GeForce GTX 1080 Founders Edition
NVIDIA GeForce GTX 980 Ti
NVIDIA GeForce GTX 980
Video Drivers: NVIDIA Release 411.51 Press
AMD Radeon Software Adrenalin Edition 18.9.1
OS: Windows 10 Pro (April 2018 Update)
Spectre/Meltdown Mitigations Yes, both
Meet The GeForce RTX 2080 Ti & RTX 2080 Founders Editions Cards Battlefield 1
Comments Locked

337 Comments

View All Comments

  • eddman - Thursday, September 20, 2018 - link

    It still doesn't justify their prices. Great cards, finally ray-tracing for games, horribly cutthroat prices.
  • Yojimbo - Saturday, September 22, 2018 - link

    So don't buy it, eddman. In the end the only real justification for prices is what people are willing to pay. If one isn't able to make a product cheaply enough for it to be sold for what people are willing to pay then the product is a bad product.

    I don't understand why you are so worried about the price. Or why you think they are "cut-throat". A cut-throat price is a very low price, not a high one.
  • eddman - Sunday, September 23, 2018 - link

    There is a wealthy minority who'd pay that much, and? It's only "justified" if you are an nvidia shareholder.

    The cards are overpriced compared to last gen and that's an absolute fact. Your constant defending of nvidia's pricing is certainly not a normal consumer behavior.
  • mapesdhs - Wednesday, September 26, 2018 - link

    Yojimbo is right that an item is only ever worth what someone is willing to pay, so in that sense NVIDIA can do what it likes, in the end it's up to the market, to consumers, whether the prices "make sense", ie. whether people actually buy them. In this regard the situation we have atm is largely that made by gamers themselves, because even when AMD released competitive products (whether by performance, value, or both), people didn't buy them. There are even people saying atm they hope AMD can release something to compete with Turing just so NVIDIA will drop its prices and thus they can buy a cheaper NVIDIA card; that's completely crazy, AMD would be mad to make something if that's how the market is going to respond.

    What's interesting this time though is that even those who in the past have been happy to buy the more expensive cards are saying they're having major hesitation about buying Turing, and the street cred which used to be perceived as coming with buying the latest & greatest has this time largely gone, people are more likely to react like someone is a gullible money pumped moron for buying these products ("More money than sense!", as my parents used to say). By contrast, when the 8800 GTX came out, that was a huge leap over the 7800 and people were very keen to get one, those who could afford it. Having one was cool. Ditto the later series right through to Maxwell (though a bit of a dip with the GTX 480 due to heat/power). The GTX 460 was a particularly good release (though the endless rebranding later was annoying). Even Pascal was a good bump over what had come before.

    Not this time though, it's a massive price increase for little gain, while the headline features provide sub-60Hz performance at a resolution far below what NVIDIA themselves have been pushing as desirable for the last 5 years (the focus has been on high frequency monitors, 4K and VR); now NVIDIA is trying to roll back the clock, which won't work, especially since those who've gotten used to high frequency monitors physically cannot go back (ref New Scientist, changes in the brain's vision system).

    Thus, eddman is right that the card's are overpriced in a general sense, as they don't remotely match what the market has come to expect from NVIDIA based on previous releases. However, if gamers don't vote with their wallets then nothing will change. Likewise, if AMD releases something just as good, or better value, but gamers don't buy them, then again nothing will change, we'll be stuck with this new expensive normal.

    I miss the Fermi days, buy two GTX 460s to have better performance than a GTX 580, didn't cost much, games ran great, and the lesser VRAM didn't bother me anyway as I wasn't using an uber monitor. Now we have cards that cost many hundreds that don't even support multi-GPU. It's as daft as Intel making the cost entry point to >= 40 PCIe lanes much higher than it was with X79 (today it's almost 1000 UKP); an old cheapo 4820K can literally do things a 7820X can't. :D

    Alas though, again it boils down to individual choice. Some want the fastest possible and if they can afford it then that's up to them, it's their free choice, we don't have the right to tell people they shouldn't buy these cards. It's their money afterall (anything else is communism). It is though an unfortunate reality that if the cards do sell well then NVIDIA will know they can maintain this higher priced and more feature restricted strategy, while selling the premium parts to Enterprise. Btw, it amazes me how people keep comparing the 2080 to the 1080 Ti even though the former has less RAM; how is that an upgrade in the product stack? (people will respond with ray tracing! Ray tracing! A feature which can't be used yet and runs too slow to be useful anyway, and with an initial implementation that's a pretty crippled implementation of the idea aswell).And why doesn't the 2080 Ti have more than 11GB? It really should, unless NVIDIA figures that if they can indeed push people back to 1080p then 11GB is enough anyway, which would be ironic.

    I'm just going to look for a used 1080 Ti, more than enough for my needs. For those with much older cards, a used 980 Ti or 1070, or various AMD cards, are good options.

    Ian.
  • Yojimbo - Wednesday, September 19, 2018 - link

    Yes, exactly. A very appropriate quote.
  • Skiddywinks - Thursday, September 20, 2018 - link

    No reason Ford couldn't have done both though. There is no technological reason nVidia could not have released a GTX 2080 Ti as well. But they know they couldn't charge as much, and the vast majority of people would not buy the RTX version. Instead, it makes their 1080 Ti stock look much more appealing to for value oriented gamers, helping them shift that stock as well as charge a huge price for the new cards.

    It's really great business, but as a gamer and not a stockholder, I'm salty.
  • Spunjji - Friday, September 21, 2018 - link

    Ford didn't invent the car, though. Ford invented a way to make them cheaper.

    Ford's strategy was not to make a new car that might do something different one day and then charge through the effing nose for it.
  • Gastec - Thursday, September 27, 2018 - link

    That quote applies perfectly to our digital electronic World: we want to go faster from point A to point B. To do that, Henry Ford gave us a car (a faster "horse"). We want the same from GPUs and CPU's, to be faster. Prettier sure, pink even. But first just make it fast.
  • Writer's Block - Monday, October 1, 2018 - link

    Except there is no evidence he said that - it is a great statement though, and conveys the intended message well
  • Hxx - Wednesday, September 19, 2018 - link

    overall dissapointing performance. RTX 2080 is a flat out bad buy at $800+ when 1080 ti custom boards are as low as $600. the RTX 2080 TI is a straight up ripoff when consumers can easily surpass its performance with 2 x 1080 TIs. I agree on the conclusion though that you are buying hardware that you wont take adavantage of yet but still, if Nvidia wants to push this hardware to all gamers, they need to drop the pricing in line with their performance otherwise not many will buy into the hype.

Log in

Don't have an account? Sign up now