The DirectX 12 Performance Preview: AMD, NVIDIA, & Star Swarm
by Ryan Smith on February 6, 2015 2:00 PM EST- Posted in
- GPUs
- AMD
- Microsoft
- NVIDIA
- DirectX 12
Star Swarm & The Test
For today’s DirectX 12 preview, Microsoft and Oxide Games have supplied us with a newer version of Oxide’s Star Swarm demo. Originally released in early 2014 as a demonstration of Oxide’s Nitrous engine and the capabilities of Mantle, Star Swarm is a massive space combat demo that is designed to push the limits of high-level APIs and demonstrate the performance advantages of low-level APIs. Due to its use of thousands of units and other effects that generate a high number of draw calls, Star Swarm can push over 100K draw calls, a massive workload that causes high-level APIs to simply crumple.
Because Star Swarm generates so many draw calls, it is essentially a best-case scenario test for low-level APIs, exploiting the fact that high-level APIs can’t effectively spread out the draw call workload over several CPU threads. As a result the performance gains from DirectX 12 in Star Swarm are going to be much greater than most (if not all) video games, but none the less it’s an effective tool to demonstrate the performance capabilities of DirectX 12 and to showcase how it is capable of better distributing work over multiple CPU threads.
It should be noted that while Star Swarm itself is a synthetic benchmark, the underlying Nitrous engine is relevant and is being used in multiple upcoming games. Stardock is using the Nitrous engine for their forthcoming Star Control game, and Oxide is using the engine for their own game, set to be announced at GDC 2015. So although Star Swarm is still a best case scenario, many of its lessons will be applicable to these future games.
As for the benchmark itself, we should also note that Star Swarm is a non-deterministic simulation. The benchmark is based on having two AI fleets fight each other, and as a result the outcome can differ from run to run. The good news is that although it’s not a deterministic benchmark, the benchmark’s RTS mode is reliable enough to keep the run-to-run variation low enough to produce reasonably consistent results. Among individual runs we’ll still see some fluctuations, while the benchmark will reliably demonstrate larger performance trends.
Star Swarm RTS Mode
The Test
For today’s preview Microsoft, NVIDIA, and AMD have provided us with the necessary WDDM 2.0 drivers to enable DirectX 12 under Windows 10. The NVIDIA driver is 349.56 and the AMD driver is 15.200. At this time we do not know when these early WDDM 2.0 drivers will be released to the public, though we would be surprised not to see them released by the time of GDC in early March.
In terms of bugs and other known issues, Microsoft has informed us that there are some known memory and performance regressions in the current WDDM 2.0 path that have since been fixed in interim builds of Windows. In particular the WDDM 2.0 path may see slightly lower performance than the WDDM 1.3 path for older drivers, and there is an issue with memory exhaustion. For this reason Microsoft has suggested that a 3GB card is required to use the Star Swarm DirectX 12 binary, although in our tests we have been able to run it on 2GB cards seemingly without issue. Meanwhile DirectX 11 deferred context support is currently broken in the combination of Star Swarm and NVIDIA's drivers, causing Star Swarm to immediately crash, so these results are with D3D 11 deferred contexts disabled.
For today’s article we are looking at a small range of cards from both AMD and NVIDIA to showcase both performance and compatibility. For NVIDIA we are looking at the GTX 980 (Maxwell 2), GTX 750 Ti (Maxwell 1), and GTX 680 (Kepler). For AMD we are looking at the R9 290X (GCN 1.1), R9 285 (GCN 1.2), and R9 260X (GCN 1.1). As we mentioned earlier support for Fermi and GCN 1.0 cards will be forthcoming in future drivers.
Meanwhile on the CPU front, to showcase the performance scaling of Direct3D we are running the bulk of our tests on our GPU testbed with 3 different settings to roughly emulate high-end Core i7 (6 cores), i5 (4 cores), and i3 (2 cores) processors. Unfortunately we cannot control for our 4960X’s L3 cache size, however that should not be a significant factor in these benchmarks.
| DirectX 12 Preview CPU Configurations (i7-4960X) | |||
| Configuration | Emulating | ||
| 6C/12T @ 4.2GHz | Overclocked Core i7 | ||
| 4C/4T @ 3.8GHz | Core i5-4670K | ||
| 2C/4T @ 3.8GHz | Core i3-4370 | ||
Though not included in this preview, AMD’s recent APUs should slot between the 2 and 4 core options thanks to the design of AMD’s CPU modules.
| CPU: | Intel Core i7-4960X @ 4.2GHz |
| Motherboard: | ASRock Fatal1ty X79 Professional |
| Power Supply: | Corsair AX1200i |
| Hard Disk: | Samsung SSD 840 EVO (750GB) |
| Memory: | G.Skill RipjawZ DDR3-1866 4 x 8GB (9-10-9-26) |
| Case: | NZXT Phantom 630 Windowed Edition |
| Monitor: | Asus PQ321 |
| Video Cards: | AMD Radeon R9 290X AMD Radeon R9 285 AMD Radeon R7 260X NVIDIA GeForce GTX 980 NVIDIA GeForce GTX 750 Ti NVIDIA GeForce GTX 680 |
| Video Drivers: | NVIDIA Release 349.56 Beta AMD Catalyst 15.200 Beta |
| OS: | Windows 10 Technical Preview 2 (Build 9926) |
Finally, while we’re going to take a systematic look at DirectX 12 from both a CPU standpoint and a GPU standpoint, we may as well answer the first question on everyone’s mind: does DirectX 12 work as advertised? The short answer: a resounding yes.
Facebook
Twitter
RSS
245 Comments
View All Comments
inighthawki - Monday, February 9, 2015 - link
>> btw funny how "M$ would need to do huge kernel rework to bring DX12 to Win7/8" while mantle, which does similar thing, is easily capable to be "OS version independent" (sure it is amd specific but still)How do you know that DX12 will not support a number of features that Mantle will not? For example, DX12 is expected to provide the application with manual memory management, a feature not available in Mantle while running on WDDM 1.3 or below.
lordken - Tuesday, February 10, 2015 - link
what I meant is in performance terms. While mantle is able to deliver +/-same performance boost as DX12 but still on old windows kernel.Not saying DX12 wont support something that mantle wont be able to do on old windows kernel. I merely tried to highlight that same performance boost can be achieved on current OS without the need of M$ taunting gamers with Win10 (forced) upgrade for DX12
killeak - Tuesday, February 10, 2015 - link
"btw funny how "M$ would need to do huge kernel rework to bring DX12 to Win7/8" while mantle, which does similar thing, is easily capable to be "OS version independent" (sure it is amd specific but still)"Direct3D has a very different design. While APIs like OpenGL or Mantle are implemented in the drivers, Direct3D is implemented (the runtime) in the OS. That means, that no matter what hardware you have, the code that is executed under the API, is for most part, always the same. Sure, the Driver needs to expose and abstract the hardware (following another API, in this case WDDM 2.0), but the actual implementation is much more slim. Which means is much more solid and reliable.
Now, OpenGL is implemented in the driver, the OS only expose the basic C functions to create the context and the like. A good driver can make OpenGL works as fast, or even more, than D3D, but the reality says that 90% of the time, OpenGL works worse. Not just because of performance, but because each driver for each OS and each GPU has a different implementation, things usually doesn't work as you expected.
After years of working with OpenGL and D3D, the thing that I miss more of D3D when I am coding for OpenGL platforms, is the single runtime. Program once, run everywhere (well on every windows) works on D3D but not on OpenGL, hell is even harder on mobile with OpenGL ES, and the broken drivers of Mali, Qualcomm, etc. Sure, if your app is simple OpenGL works, but for AAA it just doesn't cut...
The true is, IHVs are here to sell hardware, not software, so they invest the minimum time and money on it (most of the time they optimize drivers for big AAA titles and benchmarks). For mobile, where SoCs are replaced every year, is even workse, since drivers never get mature enough. Heck, Mali for example doesn't have devices with the 700 series on the market and they already announced the 800 series, while their OpenGL ES drivers for the 600 are really bad.
Going back to Mantle and Win7/8. In the drivers, you can do what ever you want, so yes, you can make your own API and make it work wherever you want, that is why Mantle can do things low level without WDDM 2.0, it doesn't need to be common or compatible to other drivers/vendors.
Bill McGann - Tuesday, February 10, 2015 - link
Yeah, this is a huge reason why GL is largely ignored by Windows devs. D3D is extremely stable thanks to it largely being implemented by MS, and them having the power to test/certify the vendor's back-ends.GL on the other hand is the wild west, with every vendor doing whatever they like... You even have them stealing MS's terrible 90's browser-war strategies of deliberately supporting broken behavior, hoping devs will use it, so that games will break on other vendor's (strictly compliant) drivers. Any situation where vendors are abusing devs like this is pretty f'ed up.
tobi1449 - Thursday, February 12, 2015 - link
The console & pc aspect isn't going anywhere and was never meant to. AMD formulated their early press releases in a way that some people jumped the hype train before it was even built, but AMD was shut down by Microsoft and Sony pretty quickly about that.Bill McGann - Tuesday, February 10, 2015 - link
FYI mantle is very carefully specified as a vendor-agnostic API, like GL, with extensions for vendor-specific behavior.If AMD even bother launching Mantle after D3D12/GLNext appear, and if it remains AMD-only, it's because nVidia/Intel have chosen not to adopt the spec, not because AMD have deliberately made it AMD-only.
tobi1449 - Thursday, February 12, 2015 - link
a) I can see why there's resistance against adopting a competitors API.b) AFAIK AMD hasn't released anything needed to implement Mantle for other hardware yet. Sure, they've often talked about it and most of the time Mantle is mentioned this pops up, but in reality (if this is still correct) it is as locked down as say G-Sync or PhysX.
Arbie - Tuesday, February 10, 2015 - link
I closely followed graphics board technology and performance for many years. But after a certain point I realized that there are actually very few - count 'em on one hand - games that I even enjoy playing. Three of those start with "Crysis" (and the other two with "Peggle"). The Battlefield series might have the same replay interest; don't know.So unless and until there are really startling ~3x gains for the same $$, my interest in desktop graphics card performance is much more constrained by game quality than by technology. I don't want to run "Borderlands" 50% faster because... I don't want to run it at all. Or any other of the lousy console ports out there.
computertech82 - Wednesday, February 11, 2015 - link
SLIGHT PROBLEM. I think it's safe to say the dx11 vs dx12 was ran on the SAME OS 10. That probably just means dx11 runs crappy on win10, not that dx12 is so much better. I bet it would be different with win7/8 dx11 vs win10 dx12 (meaning very little difference).Notmyusualid - Thursday, February 12, 2015 - link
Good point - hadn't considered it until you mentioned it.Then the comparison should really have been dx11 - Win 7/8, dx12 - Win 10, Mantle - both (if poss).