The 2018 GPU Benchmark Suite & the Test

Another year marks another update to our GPU benchmark suite. This time, however, is more in line with a maintenance update than it is a complete overhaul. Although we've done some extended compute and deep learning benchmarking in the past year, and even some HDR gaming impressions, our compute and synthetic lineup remains largely the same. But before getting into the details, let's start with the bulk of benchmarking, and the biggest reason for these cards anyhow: games.

Joining the 2018 game list is Far Cry 5, Wolfenstein II, Final Fantasy XV and Middle-earth: Shadow of War. We are also bringing in F1 2018 and Total War: Warhammer II. Returning from last year is Battlefield 1, Ashes of the Singularity: Escalation, and Grand Theft Auto V. All-in-all, these games span multiple genres, differing graphics workloads, and contemporary APIs, with a nod towards modern and relatively intensive games.

AnandTech GPU Bench 2018 Game List
Game Genre Release Date API(s)
Battlefield 1 FPS Oct. 2016 DX11
(DX12)
Far Cry 5 FPS Mar. 2018 DX11
Ashes of the Singularity: Escalation RTS Mar. 2016 DX12
(DX11, Vulkan)
Wolfenstein II: The New Colossus FPS Oct. 2017 Vulkan
Final Fantasy XV: Windows Edition JRPG Mar. 2018 DX11
Grand Theft Auto V Action/Open world Apr. 2015 DX11
Middle-earth: Shadow of War Action/RPG Sep. 2017 DX11
F1 2018 Racing Aug. 2018 DX11
Total War: Warhammer II RTS Sep. 2017 DX11
(DX12)

That said, Ashes as a DX12 trailblazer may not be as hot and fresh as it once was, especially considering that the pace of DX12 and Vulkan adoption in new games has waned. The circumstances are worth an investigation on their own, but the learning curve required in modern low-level API and the subsequent return may not be convincing right now. As a more general remark, most developers and publishers tend not to advertise or document DX12 support as much as they used to, nor is it clearly labelled in game specifications as many times DX11 is the unmentioned default.

Particularly for NVIDIA and GeForce RTX, pushing DXR and raytracing means pushing DX12, of which DXR is a component. The API has a backstop in the form of Xbox consoles and Windows 10, and if multi-GPU is to make a comeback, whether that's via compatible workloads (VR), flexible usage (ray tracing workload topologies), or just the plain old inevitability of Moore's Law. So this is less likely to be the slow end of DX12.

In terms of data collection, measurements were gathered either using built-in benchmark tools or with AMD's open-source Open Capture and Analytics Tool (OCAT), which is itself powered by Intel's PresentMon. 99th percentiles were obtained or calculated in a similar fashion, as OCAT natively obtains 99th percentiles. In general, we prefer 99th percentiles over minimums, as they more accurately represent the gaming experience and filter out any artificial outliers.

We've also swapped out Blenchmark, which seems to have been abandoned in terms of updates, in favor of a BMW render from the Blender Institute Cycles Benchmark, and a more recent one from a Cycles benchmark developer on Blenderartists.org. There were concerns with Blenchmark's small tile size, which is not very applicable to GPUs, and in terms of usability we also ran into some GPU detection errors which were linked to inaccurate Blenchmark Python code.

Otherwise, we are also keeping an eye on a few trends and upcoming developments:

  • MLPerf machine learning benchmark suite
  • Blender Benchmark
  • Futuremark's 3DMark DirectX Raytracing benchmark
  • DXR and Vulkan raytracing extension support in games

Another point is that we do not have a permanent HDR monitor for our testbed, which would be necessary to incorporate HDR game testing in the near future; 5 games in our list actually support HDR. And as we look at technologies that enhance or alter image quality (e.g. HDR, Turing's DLSS), we will want to find a better way of comparing differences. This is particularly tricky with HDR as screenshots are inapplicable and even taking accurate photographs will most likely be viewed on an SDR screen. With DLSS, there is a built-in reference quality based on 64x supersampling, which in deep learning terms is the 'ground truth'; an intuitive solution would be to use a neural network based method of analyzing quality differences, but that is likely beyond our scope.

The following tech demos and test applications were provided via NVIDIA:

  • Star Wars 'Reflections' Demo (includes real time ray tracing and DLSS support)
  • Final Fantasy XV Official Benchmark (includes DLSS support)
  • Asteroids Demo (features mesh shading and variable LOD)
  • Epic Infiltrator Demo (features DLSS)

The Testbed

Because NVIDIA is not productizing any other reference-quality GeForce RTX 2080 Ti and 2080 card besides the Founders Editions, which are non-reference by specifications, we've gone ahead and emulated the true reference specifications with a 90MHz downclock and lowering the TDP by roughly 10W. This is to keep comparisons standardized and apples-to-apples, as we always look at reference-to-reference results.

In a classic case of Murphy's Law, our usual PSU started malfunctioning around the time of the review, but given the time constraints we couldn't do a 1:1 replacement in time. As it is a digital PSU, we were beginning to use it for PCIe power readings to augment system measurements, but for now we will have to stick power draw at the wall. For the time being, we've swapped it out with another high-quality and high-wattage PSU.

CPU: Intel Core i7-7820X @ 4.3GHz
Motherboard: Gigabyte X299 AORUS Gaming 7 (F9g)
Power Supply: Corsair AX860i
EVGA 1000 G3
Hard Disk: OCZ Toshiba RD400 (1TB)
Memory: G.Skill TridentZ DDR4-3200 4 x 8GB (16-18-18-38)
Case: NZXT Phantom 630 Windowed Edition
Monitor: LG 27UD68P-B
Video Cards: AMD Radeon RX Vega 64 (Air Cooled)
NVIDIA GeForce RTX 2080 Ti Founders Edition
NVIDIA GeForce RTX 2080 Founders Edition

NVIDIA GeForce GTX 1080 Ti Founders Edition
NVIDIA GeForce GTX 1080 Founders Edition
NVIDIA GeForce GTX 980 Ti
NVIDIA GeForce GTX 980
Video Drivers: NVIDIA Release 411.51 Press
AMD Radeon Software Adrenalin Edition 18.9.1
OS: Windows 10 Pro (April 2018 Update)
Spectre/Meltdown Mitigations Yes, both
Meet The GeForce RTX 2080 Ti & RTX 2080 Founders Editions Cards Battlefield 1
Comments Locked

337 Comments

View All Comments

  • Qasar - Wednesday, September 19, 2018 - link

    just checked a local store, the lowest priced 2080 card, a gigabyte rtx 2080 is $1080, and thats canadian dollars... the most expensive RTX card ..EVGA RTX 2080 Ti XC ULTRA GAMING 11GB is $1700 !!!! again that's canadian dollars !! to make things worse.. that's PRE ORDER pricing, and have this disclaimer : Please note that the prices of the GeForce RTX cards are subject to change due to FX rate and the possibility of tariffs. We cannot guarantee preorder prices when the stock arrives - prices will be updated as needed as stock become available.
    even if i could afford these cards.. i think i would pass.. just WAY to expensive.. id prob grab a 1080 or 1080ti and be done with it... IMO... nvida is being a tad bit greedy just to protect and keep its profit margins.. but, they CAN do this.. cause there is no one else to challenge them...
  • PopinFRESH007 - Wednesday, September 19, 2018 - link

    would you care to share the bill of materials for the tu102 chip? Since you seem to suggest you know the production costs, and therefor know the profit margin which you suggest is a bit greedy.
  • Qasar - Wednesday, September 19, 2018 - link

    popin.. all i am trying to say is nvidia doesnt have to charge the prices they are charging.. but they CAN because there is nothing else out there to provide competition...
  • tamalero - Thursday, September 20, 2018 - link

    Please again explain how the cost of materials is somehow relevant on the price performance argument for consumers?

    Chips like R600, Fermi, similars.. were huge.. did it matter? NO, did performance matter? YES.
  • PopinFRESH007 - Thursday, September 20, 2018 - link

    I specifically replied to Qasar's claim "nvida is being a tad bit greedy just to protect and keep its profit margins.. but, they CAN do this" which is baseless unless they have cost information to know what their profit margins are.
  • Nagorak - Thursday, September 20, 2018 - link

    Nvidia is a public company. You can look up their profit margin and it is quite high.
  • Qasar - Thursday, September 20, 2018 - link

    PopinFRESH * sigh * i guess you will never understand the concept of " no competition, we can charge what ever we want, and people will STILL buy it cause it is the only option if you want the best or fastest " it has NOTHING to do with knowing cost info or what a companies profit margins are... but i guess you will never understand this....
  • just4U - Thursday, September 20, 2018 - link

    Ofcourse their being greedy. Since they saw their cards flying off the shelfs at 50% above MSRP earlier this year they know people are willing to pay.. so their pushing the limit. As they normally do.. this isn't new with Nvidia. Not sure why any are defending them.. or getting excessively mad about it. (..shrug)
  • mapesdhs - Wednesday, September 26, 2018 - link

    Effectively, gamers are complaining about themselves. Previous cards sold well at higher prices, so NVIDIA thinks it can push up the pricing further, and reduce product resources at the same time even when the cost is higher. If the cards do sell well then gamers only have themselves to blame, in which case nothing will change until *gamers* stop making it fashionable and cool to have the latest and greatest card. Likewise, if AMD does release something competitive, whether via price, performance or both, then gamers need to buy the damn things instead of just exploiting the lowered NVIDIA pricing as a way of getting a cheaper NVIDIA card. There's no point AMD even being in this market if people don't buy their products even when it does make sense to do so.
  • BurntMyBacon - Thursday, September 20, 2018 - link

    @PopinFRESH007: "would you care to share the bill of materials for the tu102 chip? Since you seem to suggest you know the production costs, and therefor know the profit margin which you suggest is a bit greedy."

    You have a valid point. It is hard to establish a profit margin without a bill of materials (among other things). We don't have a bill of materials, but let me establish some knowns so we can better assess.

    Typically, most supporting components on a graphics card are pretty similar to previous generation cards. Often times different designs used to do the same function are a cost cutting measure. I'm going to make an assumption that power transistors, capacitors, output connectors, etc. will remain nominally the same cost. So I'll focus on differences. The obvious is the larger GPU. This is not the first chip made on this process (TSMC 12nm) and the process appears to be a half node, so defect rates should be lower and the wafer cost should be similar to TSMC 14nm. On the other hand, the chip is still very large which will likely offset some of that yield gain and reduce the number of chips fabricated per wafer. Pascal was first generation chip produced on a new full node process (TSMC 14nm), but quite a bit smaller, so yields may have been higher and there were more chips fabricated per wafer. Also apparent is the newer GDDR6 memory tech, which will naturally cost more than GDDR5(X) at the moment, but clearly not as much as HBM2. The chips also take more power, so I'd expect a marginal increase for power related circuitry and cooling relative to pascal. I'd expect about the same cost here as for maxwell based chips, given similar power requirements.

    From all this, it sounds like graohics cards based on Turing chips will cost more to manufacture than Pascal equivalents. I it is probably not unreasonable to suggest that a TU106 may have a similar cost bill of materials to a GP102 with the note that the cost to produce the GP102 has most certainly dropped since introduction.

    I'll leave the debate on how greedy or not this is to others.

Log in

Don't have an account? Sign up now