The 2018 GPU Benchmark Suite & the Test

Another year marks another update to our GPU benchmark suite. This time, however, is more in line with a maintenance update than it is a complete overhaul. Although we've done some extended compute and deep learning benchmarking in the past year, and even some HDR gaming impressions, our compute and synthetic lineup remains largely the same. But before getting into the details, let's start with the bulk of benchmarking, and the biggest reason for these cards anyhow: games.

Joining the 2018 game list is Far Cry 5, Wolfenstein II, Final Fantasy XV and Middle-earth: Shadow of War. We are also bringing in F1 2018 and Total War: Warhammer II. Returning from last year is Battlefield 1, Ashes of the Singularity: Escalation, and Grand Theft Auto V. All-in-all, these games span multiple genres, differing graphics workloads, and contemporary APIs, with a nod towards modern and relatively intensive games.

AnandTech GPU Bench 2018 Game List
Game Genre Release Date API(s)
Battlefield 1 FPS Oct. 2016 DX11
(DX12)
Far Cry 5 FPS Mar. 2018 DX11
Ashes of the Singularity: Escalation RTS Mar. 2016 DX12
(DX11, Vulkan)
Wolfenstein II: The New Colossus FPS Oct. 2017 Vulkan
Final Fantasy XV: Windows Edition JRPG Mar. 2018 DX11
Grand Theft Auto V Action/Open world Apr. 2015 DX11
Middle-earth: Shadow of War Action/RPG Sep. 2017 DX11
F1 2018 Racing Aug. 2018 DX11
Total War: Warhammer II RTS Sep. 2017 DX11
(DX12)

That said, Ashes as a DX12 trailblazer may not be as hot and fresh as it once was, especially considering that the pace of DX12 and Vulkan adoption in new games has waned. The circumstances are worth an investigation on their own, but the learning curve required in modern low-level API and the subsequent return may not be convincing right now. As a more general remark, most developers and publishers tend not to advertise or document DX12 support as much as they used to, nor is it clearly labelled in game specifications as many times DX11 is the unmentioned default.

Particularly for NVIDIA and GeForce RTX, pushing DXR and raytracing means pushing DX12, of which DXR is a component. The API has a backstop in the form of Xbox consoles and Windows 10, and if multi-GPU is to make a comeback, whether that's via compatible workloads (VR), flexible usage (ray tracing workload topologies), or just the plain old inevitability of Moore's Law. So this is less likely to be the slow end of DX12.

In terms of data collection, measurements were gathered either using built-in benchmark tools or with AMD's open-source Open Capture and Analytics Tool (OCAT), which is itself powered by Intel's PresentMon. 99th percentiles were obtained or calculated in a similar fashion, as OCAT natively obtains 99th percentiles. In general, we prefer 99th percentiles over minimums, as they more accurately represent the gaming experience and filter out any artificial outliers.

We've also swapped out Blenchmark, which seems to have been abandoned in terms of updates, in favor of a BMW render from the Blender Institute Cycles Benchmark, and a more recent one from a Cycles benchmark developer on Blenderartists.org. There were concerns with Blenchmark's small tile size, which is not very applicable to GPUs, and in terms of usability we also ran into some GPU detection errors which were linked to inaccurate Blenchmark Python code.

Otherwise, we are also keeping an eye on a few trends and upcoming developments:

  • MLPerf machine learning benchmark suite
  • Blender Benchmark
  • Futuremark's 3DMark DirectX Raytracing benchmark
  • DXR and Vulkan raytracing extension support in games

Another point is that we do not have a permanent HDR monitor for our testbed, which would be necessary to incorporate HDR game testing in the near future; 5 games in our list actually support HDR. And as we look at technologies that enhance or alter image quality (e.g. HDR, Turing's DLSS), we will want to find a better way of comparing differences. This is particularly tricky with HDR as screenshots are inapplicable and even taking accurate photographs will most likely be viewed on an SDR screen. With DLSS, there is a built-in reference quality based on 64x supersampling, which in deep learning terms is the 'ground truth'; an intuitive solution would be to use a neural network based method of analyzing quality differences, but that is likely beyond our scope.

The following tech demos and test applications were provided via NVIDIA:

  • Star Wars 'Reflections' Demo (includes real time ray tracing and DLSS support)
  • Final Fantasy XV Official Benchmark (includes DLSS support)
  • Asteroids Demo (features mesh shading and variable LOD)
  • Epic Infiltrator Demo (features DLSS)

The Testbed

Because NVIDIA is not productizing any other reference-quality GeForce RTX 2080 Ti and 2080 card besides the Founders Editions, which are non-reference by specifications, we've gone ahead and emulated the true reference specifications with a 90MHz downclock and lowering the TDP by roughly 10W. This is to keep comparisons standardized and apples-to-apples, as we always look at reference-to-reference results.

In a classic case of Murphy's Law, our usual PSU started malfunctioning around the time of the review, but given the time constraints we couldn't do a 1:1 replacement in time. As it is a digital PSU, we were beginning to use it for PCIe power readings to augment system measurements, but for now we will have to stick power draw at the wall. For the time being, we've swapped it out with another high-quality and high-wattage PSU.

CPU: Intel Core i7-7820X @ 4.3GHz
Motherboard: Gigabyte X299 AORUS Gaming 7 (F9g)
Power Supply: Corsair AX860i
EVGA 1000 G3
Hard Disk: OCZ Toshiba RD400 (1TB)
Memory: G.Skill TridentZ DDR4-3200 4 x 8GB (16-18-18-38)
Case: NZXT Phantom 630 Windowed Edition
Monitor: LG 27UD68P-B
Video Cards: AMD Radeon RX Vega 64 (Air Cooled)
NVIDIA GeForce RTX 2080 Ti Founders Edition
NVIDIA GeForce RTX 2080 Founders Edition

NVIDIA GeForce GTX 1080 Ti Founders Edition
NVIDIA GeForce GTX 1080 Founders Edition
NVIDIA GeForce GTX 980 Ti
NVIDIA GeForce GTX 980
Video Drivers: NVIDIA Release 411.51 Press
AMD Radeon Software Adrenalin Edition 18.9.1
OS: Windows 10 Pro (April 2018 Update)
Spectre/Meltdown Mitigations Yes, both
Meet The GeForce RTX 2080 Ti & RTX 2080 Founders Editions Cards Battlefield 1
POST A COMMENT

338 Comments

View All Comments

  • Santoval - Wednesday, September 19, 2018 - link

    The problem is that it does not bring those things to the current table but is going to bring them to a future table. Essentially they expect you to buy a graphics cards that no current game can support its advanced features merely on faith that it both will deliver them in the future *and* that they will be will be worth the very high premium.

    If there is one ultimate unwritten rule when buying computers, computer parts or anything really, it must be this one : Never buy anything based on promises of *future* capabilities - always make your purchasing decisions based on what the products you buy can deliver *now*. All experienced computer and console consumers, in particular, must have that maxim engraved on their brain after having been burnt by so many broken promises.
    Reply
  • Writer's Block - Monday, October 1, 2018 - link

    That is certainly true; 'we promise', politicians and companies selling their shit use it a lot... And break it about as often. Reply
  • Inteli - Wednesday, September 19, 2018 - link

    It's not that the price increase wasn't warranted, at least from the transistor count perspective, it's that there's not a lot to show for it.

    Many more transistors...concentrated in Tensor cores and RTX cores, which aren't being touched in current games. The increased price is for a load of baggage that will take at least a year to really get used (and before you say it, 3 games is not "really getting used"). We're used to new GPUs performing better in current games for the same price, not performing the same in current games for the same price (and I'm absolutely discounting everything before 2008 because that was 10 years ago and the expectations of what a new μArch should bring have changed).

    I get the whole "future of gaming" angle you're pushing, and it's a perfectly valid reason to buy these new GPUs, but don't act like an apples-to-apples comparison of performance *right now* is the "wrong way of looking at it". How the card performs right now is an important metric for a lot of people, and will influence their decision. Especially when we're talking a potential price difference of $100+ (with sales on 1080 Ti's, and FE 2080 prices). Obviously there isn't a valid comparison for the 2080 Ti, but anyone who can drop $1300 on a GPU probably doesn't care too much about the price tag.
    Reply
  • Flunk - Thursday, September 20, 2018 - link

    Nvidia is charging what they are because they have no competition at the top end. That's it, nothing else. They're taking in the cash today in preparation for having to price more competitively later. Reply
  • just4U - Thursday, September 20, 2018 - link

    Flunk, we are talking Nvidia here.. typically speaking they don't lower prices to compete.. Sometimes they bump to high and to few bite.. but that's about it. The last time they lowered prices to compete was the 400 series but they'd just come off getting zonked by amd for basically 2 generations.. and when they went to the 500s series it was fairly competitive with amd.. (initially they were better but Amd continued to improve their 5000/6000 series.. til it was consistently beating Nvidia.. did they lower prices? NO.. not one bit..)

    TNT cards were competitive and cheap.. but once Nvidia knocked off all other contenders (aside from AMD) and started in with their geforce line they have always carried premiums regardless competition or not.
    Reply
  • eddman - Thursday, September 20, 2018 - link

    GTX 280, launched at $650 because they thought AMD couldn't do much. AMD came up with 4870. What happened? Nvidia cut the card's price to $500 a mere month after launch. So yes, they do cut prices to compete. Reply
  • Dragonstongue - Thursday, September 20, 2018 - link

    13.6 and 18.6 (bln transistor estimated) die size of 454/754mm2 (2080/2080Ti) 12nm
    7.2 and 12 (bln transistor estimated) die size of 314/471 (1070/1080-1080Ti/TitanX) 16nm

    yes it is "expensive" no doubt about that, but, it is Nv we are talking about, there is a reason they are way over valued as they are, they produce as cheaply as possible and rack them up in price as much as they can even when their actual cards shipped are no where near the $$$ figure they report as it should.

    also, if anything else, they always have and always will BS the numbers to make themselves ALWAYS appear "supreme" no matter if it is actual power used, TDP, API features, or transistor count etc etc etc.

    as far as the ray tracing crap...if they used an open source style so that everyone can use the exact same ray tracing engine so they can be directly compared to see how good they are or not then it might be "worthy" but, it is Nv they are and continue to be "it has to be our way or you don't play" type approach...I remember way back when with PhysX (which Nv boug out Ageia to do it) when Radeons were able to use it (before Nv took the ability away) they ran circles around comparable Nv cards AND used less cpu and power to do it.

    Nv does not want to get "caught" in their BS, so they find nefarious ways around everything, and when you have a massive amount of $$$$$$$$$$$$$$$$ floating everything you do, it is not hard for them to "buy silence" Intel has done so time and time again, Nv does so time and time again........blekk

    DLSS or whatever the fk they want to call it, means jack shit when only specific cards will be able to use it instead of being a truly open source initiative where everyone/everything gets to show how good they are (or not) and also stand to gain benefit from others putting effort into making it as good as it possibly can be...there is a reason why Nv barely supports Vulkan, because they are not "in control" it is way too easy to "prove them wrong"..funny because Vulkan has ray tracing "built in"

    IMO if they are as good as they claim they are, they would do everything in the light to show they are "the best" not find ways to "hide" what they are doing.....their days are numbered....hell their stock price just took a hit....good IMHO because they should not be over $200 anyways, $100, maybe, but they absolutely should not be valued above others whos financials and product shipment as magnitudes larger.
    Reply
  • Spunjji - Friday, September 21, 2018 - link

    Remind me why consumers should give a rats-ass about die size, other than its visible effects of price and performance.

    If you want to sell me a substantially larger, more expensive chip that performs a little better for a lot more money, a better reason is needed than "maybe it will make some games that aren't out yet really cool in a way that we refuse to give you any performance indications about".

    Screw that.
    Reply
  • Writer's Block - Monday, October 1, 2018 - link

    They look poor value; good performance, sure. But a 1080ti offers the same for much less.
    They want me to buy promises! Seriously, promises are never worth the paper they are printed on - digital or the real stuff.
    Reply
  • Writer's Block - Monday, October 1, 2018 - link

    Oh and, yeh agree. Reply

Log in

Don't have an account? Sign up now