The 2018 GPU Benchmark Suite & the Test

Another year marks another update to our GPU benchmark suite. This time, however, is more in line with a maintenance update than it is a complete overhaul. Although we've done some extended compute and deep learning benchmarking in the past year, and even some HDR gaming impressions, our compute and synthetic lineup remains largely the same. But before getting into the details, let's start with the bulk of benchmarking, and the biggest reason for these cards anyhow: games.

Joining the 2018 game list is Far Cry 5, Wolfenstein II, Final Fantasy XV and Middle-earth: Shadow of War. We are also bringing in F1 2018 and Total War: Warhammer II. Returning from last year is Battlefield 1, Ashes of the Singularity: Escalation, and Grand Theft Auto V. All-in-all, these games span multiple genres, differing graphics workloads, and contemporary APIs, with a nod towards modern and relatively intensive games.

AnandTech GPU Bench 2018 Game List
Game Genre Release Date API(s)
Battlefield 1 FPS Oct. 2016 DX11
(DX12)
Far Cry 5 FPS Mar. 2018 DX11
Ashes of the Singularity: Escalation RTS Mar. 2016 DX12
(DX11, Vulkan)
Wolfenstein II: The New Colossus FPS Oct. 2017 Vulkan
Final Fantasy XV: Windows Edition JRPG Mar. 2018 DX11
Grand Theft Auto V Action/Open world Apr. 2015 DX11
Middle-earth: Shadow of War Action/RPG Sep. 2017 DX11
F1 2018 Racing Aug. 2018 DX11
Total War: Warhammer II RTS Sep. 2017 DX11
(DX12)

That said, Ashes as a DX12 trailblazer may not be as hot and fresh as it once was, especially considering that the pace of DX12 and Vulkan adoption in new games has waned. The circumstances are worth an investigation on their own, but the learning curve required in modern low-level API and the subsequent return may not be convincing right now. As a more general remark, most developers and publishers tend not to advertise or document DX12 support as much as they used to, nor is it clearly labelled in game specifications as many times DX11 is the unmentioned default.

Particularly for NVIDIA and GeForce RTX, pushing DXR and raytracing means pushing DX12, of which DXR is a component. The API has a backstop in the form of Xbox consoles and Windows 10, and if multi-GPU is to make a comeback, whether that's via compatible workloads (VR), flexible usage (ray tracing workload topologies), or just the plain old inevitability of Moore's Law. So this is less likely to be the slow end of DX12.

In terms of data collection, measurements were gathered either using built-in benchmark tools or with AMD's open-source Open Capture and Analytics Tool (OCAT), which is itself powered by Intel's PresentMon. 99th percentiles were obtained or calculated in a similar fashion, as OCAT natively obtains 99th percentiles. In general, we prefer 99th percentiles over minimums, as they more accurately represent the gaming experience and filter out any artificial outliers.

We've also swapped out Blenchmark, which seems to have been abandoned in terms of updates, in favor of a BMW render from the Blender Institute Cycles Benchmark, and a more recent one from a Cycles benchmark developer on Blenderartists.org. There were concerns with Blenchmark's small tile size, which is not very applicable to GPUs, and in terms of usability we also ran into some GPU detection errors which were linked to inaccurate Blenchmark Python code.

Otherwise, we are also keeping an eye on a few trends and upcoming developments:

  • MLPerf machine learning benchmark suite
  • Blender Benchmark
  • Futuremark's 3DMark DirectX Raytracing benchmark
  • DXR and Vulkan raytracing extension support in games

Another point is that we do not have a permanent HDR monitor for our testbed, which would be necessary to incorporate HDR game testing in the near future; 5 games in our list actually support HDR. And as we look at technologies that enhance or alter image quality (e.g. HDR, Turing's DLSS), we will want to find a better way of comparing differences. This is particularly tricky with HDR as screenshots are inapplicable and even taking accurate photographs will most likely be viewed on an SDR screen. With DLSS, there is a built-in reference quality based on 64x supersampling, which in deep learning terms is the 'ground truth'; an intuitive solution would be to use a neural network based method of analyzing quality differences, but that is likely beyond our scope.

The following tech demos and test applications were provided via NVIDIA:

  • Star Wars 'Reflections' Demo (includes real time ray tracing and DLSS support)
  • Final Fantasy XV Official Benchmark (includes DLSS support)
  • Asteroids Demo (features mesh shading and variable LOD)
  • Epic Infiltrator Demo (features DLSS)

The Testbed

Because NVIDIA is not productizing any other reference-quality GeForce RTX 2080 Ti and 2080 card besides the Founders Editions, which are non-reference by specifications, we've gone ahead and emulated the true reference specifications with a 90MHz downclock and lowering the TDP by roughly 10W. This is to keep comparisons standardized and apples-to-apples, as we always look at reference-to-reference results.

In a classic case of Murphy's Law, our usual PSU started malfunctioning around the time of the review, but given the time constraints we couldn't do a 1:1 replacement in time. As it is a digital PSU, we were beginning to use it for PCIe power readings to augment system measurements, but for now we will have to stick power draw at the wall. For the time being, we've swapped it out with another high-quality and high-wattage PSU.

CPU: Intel Core i7-7820X @ 4.3GHz
Motherboard: Gigabyte X299 AORUS Gaming 7 (F9g)
Power Supply: Corsair AX860i
EVGA 1000 G3
Hard Disk: OCZ Toshiba RD400 (1TB)
Memory: G.Skill TridentZ DDR4-3200 4 x 8GB (16-18-18-38)
Case: NZXT Phantom 630 Windowed Edition
Monitor: LG 27UD68P-B
Video Cards: AMD Radeon RX Vega 64 (Air Cooled)
NVIDIA GeForce RTX 2080 Ti Founders Edition
NVIDIA GeForce RTX 2080 Founders Edition

NVIDIA GeForce GTX 1080 Ti Founders Edition
NVIDIA GeForce GTX 1080 Founders Edition
NVIDIA GeForce GTX 980 Ti
NVIDIA GeForce GTX 980
Video Drivers: NVIDIA Release 411.51 Press
AMD Radeon Software Adrenalin Edition 18.9.1
OS: Windows 10 Pro (April 2018 Update)
Spectre/Meltdown Mitigations Yes, both
Meet The GeForce RTX 2080 Ti & RTX 2080 Founders Editions Cards Battlefield 1
POST A COMMENT

338 Comments

View All Comments

  • Bp_968 - Sunday, December 2, 2018 - link

    Even though the review is older and this comment is a few months old I just wanted to jump in and say "hah, look, eddman was right!" Now that the Titan RTX leaks are showing up. Lol. They didn't even wait for supply to stabilize on the 2080ti before dropping the titan.

    Plus, if the 2080 replaced the 1080ti then why is it more expensive and no faster? That would be a first even for Nvidia..
    Reply
  • PeachNCream - Thursday, September 20, 2018 - link

    The model numbers aren't that significant. NVIDIA could just have easily released a 2080, a 2070, and a 2060 by putting different labels on the boxes of the 2080 Ti, the 2080, and the 2070 for instance. The Ti, the Titan, all of those are long standing marketing identities that buyers now automatically associate with a certain relative scale of performance among other GPUs of the same generation. NVIDIA can play upon buyer expectations by releasing various products to fill those expectations in the way that best advances the company's interest. Any company with enough brand recognition can easily do the same. Consider Intel's long-running i-series CPU numbering. The fact that something labeled as a Ti came out at a certain time isn't an example of technological development, but a way of meeting customer expectations in reflection of the MSRP. We would have balked much more at $1200 for the exact same product if it was labeled as a plain vanilla 2080 and the current vanilla 2080 was branded as a 2070. Instead, we say, "Well, the 2080 Ti is really expensive, but at least its a Ti so that makes it a little bit more reasonable." Reply
  • eddman - Thursday, September 20, 2018 - link

    Model numbers are significant in the way that they point out the models in the same successive line up. That's the entire point of them.

    I and a lot of people are not in this "we" you talk about. Again, nvidia themselves compare it to 1080 Ti every chance they get, so I do not see why I should in any way think its price is "reasonable".

    That's not how past generational leaps worked, even for 8800 GTX. We got massive performance gains AND usually new rendering features at similar MSRPs or maybe a bit higher. The difference this time is that AMD has left the building, for now.
    Reply
  • PeachNCream - Thursday, September 20, 2018 - link

    Don't misunderstand me. I'm not implying that the price is okay or that anyone should find it reasonable to stomach a $1200 MSRP for a mere graphics card. I also agree that part of the pricing problem is due to an absence of credible competition from AMD. I'm just arguing that the people in the NVIDIA marketing department may justify the price in part by slapping a Ti label on the box so consumers are less likely to balk during checkout. The reality is that we're getting a step sideways in performance for a noteworthy increase in TDP due to the addition of capabilities that may or may not actually add much value because said features are too demanding to play nicely at high resolutions and because there are not indications that the software side will move to take advantage of said features. At best, the addition of the hardware won't be very compelling until the next generation of GPUs after Turing when its likely that performance will pick up a bit.

    Then again, who am I to talk? I play PC games on a laptop with an HD 4000 infrequently and end up mostly gaming on my ancient dual core Kitkat era phone that I've been keeping as a cheap wireless mini tablet. To me, PC gaming became an overly pricey sink of my limited, single parent free time. I'd rather bank my spare money in something that yields interest over time than throw it into gaming hardware that's obsolete in a matter of a few years. That and my kids me to be both of their parents these days since that worthless ex of mine schlepped off to marry some woman in Canada. *grumble*
    Reply
  • tamalero - Thursday, September 20, 2018 - link

    More like that they are pricing their high end cards like they are flagship cards.
    The 2080 Founders seems identical in price to a 1080TI. That is unacceptable. Specially when they are almost identical in performance (going slower in most games by a few small points).

    They(Nvidia) just want to clear the huge build up of PASCAL cards.. by charging insanity for those who are willing to claim to be "gamers" with money. period.
    Reply
  • tamalero - Thursday, September 20, 2018 - link

    "You, like so many others don't get it. nVidia has re-worked their product lines. Didn't you notice how the Ti came out at the same time as the 2080?"
    What the hell does this has to do? Nothing for the consumer again.
    Reply
  • tamalero - Thursday, September 20, 2018 - link

    "Die size is not irrelevant to consumers because increased die size means increased cost to manufacture. Increased cost to manufacture means a pressure for higher prices. The question is what you get in return for those higher prices.
    "
    You're repeating the same.
    Die size means NOTHING to a consumer. It means something for the manufacturer because it costs THEM.
    If the die doesnt benefit anything at all (Fermi) compared to smaller dies that offer almost the same performance (Pascal). Why would the consumer have to pay MORE for LESS?

    New tech is nothing if there is nothing to show. And there is NOTHING to show right now.
    by the time raytracing becomes really viable, the new generation of cards will be out.
    Reply
  • Spunjji - Friday, September 21, 2018 - link

    This x1000. These cards are a necessary step towards getting the technology out there, but I'm thoroughly unconvinced that it is a good idea for anyone to buy them. The sacrifice in die area was too great, for far too little benefit. Given the strong indications that 1080p ~45fps is where real-time raytracing will be at right now, I just don't care. They sold me on high-resolution and high-framerate because those actually affect how much enjoyment I get from my games. I'm not interested in that rug being pulled from under my feet *and* paying damn near double price for the privilege. Reply
  • Morawka - Wednesday, September 19, 2018 - link

    Doesn't TSMC charge their customers by the wafer nowadays? Reply
  • PopinFRESH007 - Wednesday, September 19, 2018 - link

    how does that matter? Are you suggesting that magically makes the die size irrelevant? If you have a 300mm wafer and you double the die size, you also halve the number of die per wafer. This would also ignore yield. A larger die is more costly to produce because you get fewer die per wafer and increase the probability of having a defect within a die. Reply

Log in

Don't have an account? Sign up now