Quick Look: Vulkan Performance on The Talos Principleby Daniel Williams & Ryan Smith on February 17, 2016 10:00 AM EST
Following yesterday’s hard launch of Vulkan 1.0 – drivers, development tools, and the rest of the works – also released alongside Vulkan was the first game with Vulkan rendering support, The Talos Principle. Developer Croteam has a history of supporting multiple rendering paths with their engines, and the 2014 puzzle-em-up is no different, supporting DirectX 9, DirectX 11, and OpenGL depending on which platform it’s being run on. Now with Vulkan’s release Croteam has gone one step further, implementing early Vulkan support in a beta build of the game.
Since this is the first game with any kind of Vulkan support, we wanted to spend a bit of time looking at what Vulkan performance was like under Windows. Games with full support for Vulkan are still going to be some time off, as even with game dev participation in the standardization process it takes time to write a solid and high efficiency rendering path for these new low-level APIs, but none the less it gives us a chance to at least take a peek at the state of Vulkan on day 1.
To be very clear here this is an early look at Vulkan performance; Croteam admits from the get-go that their current implementation is very early, and is not as fast as their now highly tuned DirectX 11 implementation. Furthermore The Talos Principle is not a title that’s designed to exploit the CPU utilization and draw call improvements that are central to Vulkan (unlike say Star Swarm when we first looked at DX12). So with that in mind, it’s important to set reasonable expectations of what’s to come.
On the driver side of matters, both AMD and NVIDIA released Vulkan drivers yesterday. As is common with new API releases, both drivers are developer betas and either lack features or are based on older branches than current consumer drivers, however the NVIDIA driver has passed Vulkan conformance testing. AMD and NVIDIA will be integrating Vulkan into their release consumer drivers in the future as they improve on driver quality and catch up with the latest driver branches.
Finally, for our testing we’re using our standard GPU testbed running Windows 8.1, in part to showcase Vulkan on a platform that can’t receive DirectX 12. As the release of AMD’s drivers was unexpected – we had already begun preparing for this article earlier in the week – we don’t have results for very many AMD cards, but as this is a quick look it gets the point across.
|CPU:||Intel Core i7-4960X @ 4.2GHz|
|Motherboard:||ASRock Fatal1ty X79 Professional|
|Power Supply:||Corsair AX1200i|
|Hard Disk:||Samsung SSD 840 EVO (750GB)|
|Memory:||G.Skill RipjawZ DDR3-1866 4 x 8GB (9-10-9-26)|
|Case:||NZXT Phantom 630 Windowed Edition|
|Video Cards:||NVIDIA GeForce GTX 980 Ti
NVIDIA GeForce GTX 960
NVIDIA GeForce GTX 760
AMD Radeon R9 Fury X
AMD Radeon R9 285
AMD Radeon R9 370
|Video Drivers:||NVIDIA Release 361.91 (DX11 & OpenGL)
NVIDIA Beta 356.39 (Vulkan)
AMD Radeon Software Crimson 16.1.1 Hotfix (DX11 & OpenGL)
AMD Radeon Software Beta for Vulkan (Vulkan)
|OS:||Windows 8.1 Pro|
The Talos Principle: Performance
Update 02/19: By request, I've also added Fury X numbers to our comparison to showcase high-end AMD performance
We’ve gone ahead and run our full collection of cards with Ultra settings at both 1080p and 720p to showcase a typical gaming workload and a lighter workload that is much more unlikely to be GPU limited. We’ve also gone ahead and run our two most powerful cards, the GeForce GTX 980 Ti and Radeon R9 Fury X, at 1440p to also showcase a more strictly GPU-bound scenario.
As expected from Croteam’s comments, at no point here does Vulkan catch up with DirectX 11. This is still an early rendering path and there’s no reason to expect that in time it won’t get up to the speed of DX11 (or even surpass it), but that’s not the case right now.
The real reason we set about to run these tests was not to compare early Vulkan to DX11, but rather to compare Vulkan to the API it succeed, OpenGL. OpenGL itself isn’t going anywhere – it is the DirectX 11 to Vulkan’s DirectX 12, the API that will remain for non-guru programmers who don’t need the power but need easier access – but as OpenGL suffers from many of the same performance bottlenecks as DX11 (plus some whole new ones from a 24 year legacy), there’s clear room for improvement with Vulkan.
To that end the results are more promising. As compared to The Talos Principle’s OpenGL renderer, the Vulkan renderer is not all that different in performance in clearly GPU-bound scenarios. But once we start looking at CPU-bound scenarios, even in a somewhat lightweight game like The Talos Principle, Vulkan pulls ahead. This is especially evident on the GTX 980 Ti and R9 Fury X at 1080p, and across a few different cards at 720p. This offers our first sign that Vulkan will indeed be capable of bringing its desired CPU performance benefits to games, perhaps even in games where they’re not explicitly pushing the draw calls limits of a system.
These performance results do also highlight some performance issues as well. The two slower AMD cards – both of which have 2GB of VRAM – see some unusual performance regressions. Based on our experience with DX12 and Mantle, it seems likely that on these settings The Talos Principle is approaching full VRAM utilization, leading to the occasional drop in performance. Just as with DX12, developers have near-full control of the GPU, and will need to manage VRAM usage carefully.
Radeon R9 285 Running via Vulkan
As for image quality, the rendering path that Croteam has implemented appears to be every bit as good as their existing paths. Both AMD and NVIDIA cards exhibited great image quality that was comparable to the baseline DX11 rendering path. And admittedly we weren’t expecting any differently, but it means there are no image quality affecting bugs that we’ve picked up on in our testing.
That said, these Vulkan drivers are classified as betas by both AMD and NVIDIA, and this is not a misnomer. We encountered issues with drivers for both parties, particularly in NVIDIA’s case where we couldn’t successfully run a Talos benchmarking session twice without rebooting, otherwise the game would crash. So coupled with the known limitations for these drivers, it goes without saying that these drivers are really only for testing and development purposes, and that AMD and NVIDIA will need to knock out some more bugs before integrating Vulkan support into their release drivers.
Overall with this being the third low-level API release in the past two years (and a rebirth of sorts for Mantle), for our regular readers there aren’t any great surprises to be found with Vulkan as implemented on The Talos Principle. Still, the results do show promise. Khronos has set about creating a new cross-platform low-level API, and this early preview of Vulkan shows that they have achieved their basic goals. Now it will be a matter of seeing what developers can do with the API with more developer time and in conjunction with further driver improvements from AMD, NVIDIA, and the other GPU vendors.
Post Your CommentPlease log in or sign up to comment.
View All Comments
Flunk - Wednesday, February 17, 2016 - linkYou're right about only the largest developers having the resources to implement DX12 or Vulkan, but a lot of smaller companies are using 3rd party engines like Unreal or Unity so if they support DX12 or Vulkan it significantly lowers the bar of entry.
Murloc - Wednesday, February 17, 2016 - linkI guess small teams will use game engines or DX11 then.
Most of the business is captured by the big companies. They'll make proper use of this.
Nintendo Maniac 64 - Wednesday, February 17, 2016 - linkRetroArch already has a Vulkan implementation, and it has a "team" of something like 5 people:
bcronce - Wednesday, February 17, 2016 - linkAll of those "tricks" are undefined and engine specific and can break at any moment. All current 3D graphics APIs are horribly slow unless 1) someone spends a lot of time optimizing and 2) the driver makers make undocumented "tricks" specific to a given engine or game
It is currently impossible to make a high quality fast 3D games without being a AAA programming shop because AMD and Nvidia don't have the time of day to create these hacks to make your game work decently. Instead, small shops have to mimic existing AAA games and hope the drivers mistake them for the optimized game.
This is exactly why you can see large performance increases for specific games with every new driver release. All of those "we made some game 25% faster!" isn't because the game engine got updated, it's because the drivers got a whole bunch of undocumented hacks and shortcuts added.
Senti - Thursday, February 18, 2016 - link^ This. Developing as huge company with dedicated support from drivers and developing as small group are two absolutely different things.
Do you know how small developers feel about all those "driver optimizations" that bring 25% speedup to some stupid overhyped game? The absolutely hate that driver developers instead of fixing their pile of bugs that cause your programs to just plain break instead spend all the time on one-time point hacks.
Harder to develop for low level API? Harder compared to what, to extremely buggy high level API? It probably can't be any worse...
RobATiOyP - Sunday, February 21, 2016 - linkWell said! Mozilla's Servo project developed a GPU render back end, which accelerated things, one key thing is to mimic what games do, to try and stay on relatively well tested code paths, currently GPU acceleration has defaulted to off in Firefox on most HW for this reason.
Probably a mix n match toolkit to aid applications, construct HW dependant drivers, avoiding the middle layer mistake anti-pattern will evolve, putting small developers onto more comfortable ground using widely used code paths & techniques.
RobATiOyP - Sunday, February 21, 2016 - linkWith time, there'll be patterns and probably helper toolkits which lower the investment required to target Vulkan. There's already talk of re-implementing OpenGL with it's state tracking and so on, over the top of Vulkan.
For now performance is the last concern, finding the necessary revisions in 1.0 version of Vulkan specification is the priority.
tipoo - Wednesday, February 17, 2016 - linkI wonder if they're just using a wrapper to bridge HLSL shader programs to Vulkan. That would leave a lot of performance on the table without hand writing new ones. With the small team size and quick turnaround I think this is possible.
Anyways, huge leap over OGL at any rate, and with polish should hopefully be competitive with DX.
przemo_li - Wednesday, February 17, 2016 - linkIt possible.
However more likely is that shaders are generated in pseudo code, and them transpilled to HSLS, GLSL, SPIR-V, etc.
Kontis - Wednesday, February 17, 2016 - linkAs explained by the dev here: http://steamcommunity.com/app/257510/discussions/0...
this is currently a quick wrapper.
This kind of a naive port from a high-level API (like dx 9/11) to DirectX 12 or Vulkan will always result in a degraded performance.
If you want to benchmark Vulkan working in a way it was designed to be used, you have to wait for proper Vulkan renderer implementations.