The 2018 GPU Benchmark Suite & the Test

Another year marks another update to our GPU benchmark suite. This time, however, is more in line with a maintenance update than it is a complete overhaul. Although we've done some extended compute and deep learning benchmarking in the past year, and even some HDR gaming impressions, our compute and synthetic lineup remains largely the same. But before getting into the details, let's start with the bulk of benchmarking, and the biggest reason for these cards anyhow: games.

Joining the 2018 game list is Far Cry 5, Wolfenstein II, Final Fantasy XV and Middle-earth: Shadow of War. We are also bringing in F1 2018 and Total War: Warhammer II. Returning from last year is Battlefield 1, Ashes of the Singularity: Escalation, and Grand Theft Auto V. All-in-all, these games span multiple genres, differing graphics workloads, and contemporary APIs, with a nod towards modern and relatively intensive games.

AnandTech GPU Bench 2018 Game List
Game Genre Release Date API(s)
Battlefield 1 FPS Oct. 2016 DX11
(DX12)
Far Cry 5 FPS Mar. 2018 DX11
Ashes of the Singularity: Escalation RTS Mar. 2016 DX12
(DX11, Vulkan)
Wolfenstein II: The New Colossus FPS Oct. 2017 Vulkan
Final Fantasy XV: Windows Edition JRPG Mar. 2018 DX11
Grand Theft Auto V Action/Open world Apr. 2015 DX11
Middle-earth: Shadow of War Action/RPG Sep. 2017 DX11
F1 2018 Racing Aug. 2018 DX11
Total War: Warhammer II RTS Sep. 2017 DX11
(DX12)

That said, Ashes as a DX12 trailblazer may not be as hot and fresh as it once was, especially considering that the pace of DX12 and Vulkan adoption in new games has waned. The circumstances are worth an investigation on their own, but the learning curve required in modern low-level API and the subsequent return may not be convincing right now. As a more general remark, most developers and publishers tend not to advertise or document DX12 support as much as they used to, nor is it clearly labelled in game specifications as many times DX11 is the unmentioned default.

Particularly for NVIDIA and GeForce RTX, pushing DXR and raytracing means pushing DX12, of which DXR is a component. The API has a backstop in the form of Xbox consoles and Windows 10, and if multi-GPU is to make a comeback, whether that's via compatible workloads (VR), flexible usage (ray tracing workload topologies), or just the plain old inevitability of Moore's Law. So this is less likely to be the slow end of DX12.

In terms of data collection, measurements were gathered either using built-in benchmark tools or with AMD's open-source Open Capture and Analytics Tool (OCAT), which is itself powered by Intel's PresentMon. 99th percentiles were obtained or calculated in a similar fashion, as OCAT natively obtains 99th percentiles. In general, we prefer 99th percentiles over minimums, as they more accurately represent the gaming experience and filter out any artificial outliers.

We've also swapped out Blenchmark, which seems to have been abandoned in terms of updates, in favor of a BMW render from the Blender Institute Cycles Benchmark, and a more recent one from a Cycles benchmark developer on Blenderartists.org. There were concerns with Blenchmark's small tile size, which is not very applicable to GPUs, and in terms of usability we also ran into some GPU detection errors which were linked to inaccurate Blenchmark Python code.

Otherwise, we are also keeping an eye on a few trends and upcoming developments:

  • MLPerf machine learning benchmark suite
  • Blender Benchmark
  • Futuremark's 3DMark DirectX Raytracing benchmark
  • DXR and Vulkan raytracing extension support in games

Another point is that we do not have a permanent HDR monitor for our testbed, which would be necessary to incorporate HDR game testing in the near future; 5 games in our list actually support HDR. And as we look at technologies that enhance or alter image quality (e.g. HDR, Turing's DLSS), we will want to find a better way of comparing differences. This is particularly tricky with HDR as screenshots are inapplicable and even taking accurate photographs will most likely be viewed on an SDR screen. With DLSS, there is a built-in reference quality based on 64x supersampling, which in deep learning terms is the 'ground truth'; an intuitive solution would be to use a neural network based method of analyzing quality differences, but that is likely beyond our scope.

The following tech demos and test applications were provided via NVIDIA:

  • Star Wars 'Reflections' Demo (includes real time ray tracing and DLSS support)
  • Final Fantasy XV Official Benchmark (includes DLSS support)
  • Asteroids Demo (features mesh shading and variable LOD)
  • Epic Infiltrator Demo (features DLSS)

The Testbed

Because NVIDIA is not productizing any other reference-quality GeForce RTX 2080 Ti and 2080 card besides the Founders Editions, which are non-reference by specifications, we've gone ahead and emulated the true reference specifications with a 90MHz downclock and lowering the TDP by roughly 10W. This is to keep comparisons standardized and apples-to-apples, as we always look at reference-to-reference results.

In a classic case of Murphy's Law, our usual PSU started malfunctioning around the time of the review, but given the time constraints we couldn't do a 1:1 replacement in time. As it is a digital PSU, we were beginning to use it for PCIe power readings to augment system measurements, but for now we will have to stick power draw at the wall. For the time being, we've swapped it out with another high-quality and high-wattage PSU.

CPU: Intel Core i7-7820X @ 4.3GHz
Motherboard: Gigabyte X299 AORUS Gaming 7 (F9g)
Power Supply: Corsair AX860i
EVGA 1000 G3
Hard Disk: OCZ Toshiba RD400 (1TB)
Memory: G.Skill TridentZ DDR4-3200 4 x 8GB (16-18-18-38)
Case: NZXT Phantom 630 Windowed Edition
Monitor: LG 27UD68P-B
Video Cards: AMD Radeon RX Vega 64 (Air Cooled)
NVIDIA GeForce RTX 2080 Ti Founders Edition
NVIDIA GeForce RTX 2080 Founders Edition

NVIDIA GeForce GTX 1080 Ti Founders Edition
NVIDIA GeForce GTX 1080 Founders Edition
NVIDIA GeForce GTX 980 Ti
NVIDIA GeForce GTX 980
Video Drivers: NVIDIA Release 411.51 Press
AMD Radeon Software Adrenalin Edition 18.9.1
OS: Windows 10 Pro (April 2018 Update)
Spectre/Meltdown Mitigations Yes, both
Meet The GeForce RTX 2080 Ti & RTX 2080 Founders Editions Cards Battlefield 1
POST A COMMENT

338 Comments

View All Comments

  • Dribble - Thursday, September 20, 2018 - link

    The DLSS basically gives you a resolution jump for free (e.g. 4k for 1440p performance) and is really easy to implement. That's going to take off fast and probably means even the 2070 will be faster then the 1080Ti in games that support it. Reply
  • Lolimaster - Saturday, September 22, 2018 - link

    No not free, everyone can see the blurry mess the renamed blur effect is. Reply
  • Inteli - Saturday, September 22, 2018 - link

    TIL that when you stop isolating variables in a benchmark, a lower-end card can be faster than a higher-end card. Reply
  • tamalero - Wednesday, September 19, 2018 - link

    Die size is irrelevant to consumers. They see price vs performance. not how big the silicon is.

    AMD was toasted for having hot slow chips. many times.. so did nvidia.. big and hot means nothing if it doesn't perform as expected for the insane prices they 're asking for.
    Reply
  • Yojimbo - Wednesday, September 19, 2018 - link

    Die size is not irrelevant to consumers because increased die size means increased cost to manufacture. Increased cost to manufacture means a pressure for higher prices. The question is what you get in return for those higher prices.

    People like what they know... what they are used to. If some new AA technique comes along and increases performance significantly but introduces visual artifacts it will be rejected as a step backwards. But if a new technology comes along that has a significant performance cost yet increases visual quality much more significantly than the aforementioned artifacts decrease it, people will also have a tendency to reject it. That is, until they become familiar with the technology... That's where we are with RTX. No one can become familiar with the technology when there are no games that make use of it. So trying to judge the value of the larger die sizes is an abstract thing. In a few months the situation will be different.

    Personally, I think the architecture will be remembered as one of the biggest and most important in the entire history of gaming. There is so much new technology in it that some of it barely anyone is saying much about (where have you heard about texture space shading, for example?). Several of these technologies will have their greatest benefits with VR, and if VR had taken off people would be marveling about this architecture immediately. But I think that VR will eventually take off, and I think several of these technologies will become the standard way of doing things for the next several years. They are new and complicated for developers, though. Only a few developers are prepared to take advantage of the stuff today. It's going to be some time before we really can put the architecture into its proper historical perspective.

    From the point of view of a purchase today, though, it's a bit of an unknown. If you buy a card now and plan to keep it for 4 years, I think you'd be better off getting a 20 series than a 10 series. If you buy it and keep it for 2 years, then it's a bit less clear, but we'll have a better idea of the answer to that question in 6 months, I think.

    I do think, though, that if an architecture with this much new stuff were introduced 20 years ago everybody, including games enthusiast sites like Anandtech, would be going gaga over it. The industry was moving faster then and people were more optimistic. Also the sites didn't try to be so demure. Hmm, gaga as the opposite of demure. Maybe that's why she's called Lady Gaga.
    Reply
  • Santoval - Wednesday, September 19, 2018 - link

    I agree that this might be the most game-changing graphics tech of the last couple of decades, and that the future belongs to ray-tracing, but I also think that precisely due to the general uncertainty and the very high prices Nvidia might suffer one of their biggest sales slumps this generation, if not *the* biggest. They did not handle the launch well : it is absurd to release a new series with zero ray-traced, DLSS supporting or mesh shaded games at launch.

    Their extensive NDAs, lack of information and ban on benchmarks between the Gamescom pre-launch and the actual launch, despite going live with (blind faith based) preorders during that window, was also controversial and highly suspicious. It appears that Nvidia gave graphics cards to game developers very late to avoid leaks, but that resulted in having no RTX supporting games at launch. They thought they could not have it both ways apparently, but choosing that over having RTX supporting games at launch was a very risky gamble.

    Since their sales will almost certainly slump, they should release 7nm based graphics cards (either a true 30xx series or Turing at 7nm, I guess the latter) much sooner, probably in around 6 months. They might have expected a sales slump, which is why they released 2080Ti now. I suppose they will try to avoid it with aggressive marketing and somewhat lower prices lately, but it is not certain they'll succeed.
    Reply
  • eddman - Thursday, September 20, 2018 - link

    Would you've still defended this if it was priced at $1500? How about $2000? Do you always ignore price when new tech is involved?

    The cards, themselves, aren't bad. They are actually very good. It's their pricing.

    These cards, specifically 2080 Ti, are overpriced compared to their direct predecessors. Ray tracing, DLSS, etc. etc. they still do not justify such prices for such FPS gains in regular rasterized games.

    A 2080 Ti might be an ok purchase for $850-900, but certainly not $1200+. Even 8800 GTX with its new cuda cores and new generation of lighting tech launched at the same MSRP as 7800 GTX.

    These cards are surely more expensive to make, but there is no doubt that the biggest factor for these price jumps is that nvidia is basically competing with themselves. Why price them lower when they can easily sell truckloads of pascal cards at their usual prices until the inventory is gone.
    Reply
  • Andrew LB - Thursday, September 20, 2018 - link

    You, like so many others don't get it. nVidia has re-worked their product lines. Didn't you notice how the Ti came out at the same time as the 2080? You might also notice that Titan is now called Titan V (volta) and not GTX Titan. Titan is now in its own family of non-gaming cards and that is reflected in the driver section on their site. They now have titan specific drivers.

    Here, watch this. Jay explains it fairly well.

    https://youtu.be/5XRWATUDS7o?t=6m2s
    Reply
  • eddman - Thursday, September 20, 2018 - link

    You took an opinion and decided it's a fact. It's not. That guy is not the authority on graphics cards.

    There is no official word that titan is now 2080 Ti. Nvidia named that card 2080 Ti, it has a 102 named chip. Nvidia themselves constantly compare it to 1080 Ti, which also has a 102 named chip, therefore it's the successor to 1080 Ti and it's very normal to expect similar pricing.

    Don't worry, there will be a Titan turing, considering that 2080 Ti does not even use the fully enabled chip.

    It's really baffling to see people, paying customers, defending a $1200 price tag. It is as if you like to be charged more.
    Reply
  • eddman - Thursday, September 20, 2018 - link

    $1000, but it's still too high, and you cannot find any card at that price anyway. Reply

Log in

Don't have an account? Sign up now