Other Notes

Before jumping into our results, let’s quickly talk about testing.

For our test we are using the latest version of the Windows 10 technical preview – build 10041 – and the latest drivers from AMD, Intel, and NVIDIA. In fact for testing DirectX 12 these latest packages are the minimum versions that the test supports. Meanwhile 3DMark does of course also run on Windows Vista and later, however on Windows Vista/7/8 only the DirectX 11 and Mantle tests are available since those are the only APIs available.

From a test reliability standpoint the API Overhead Feature Test (or as we’ll call it from now, AOFT) is generally reliable under DirectX 12 and Mantle, however we would like to note that we have found it to be somewhat unreliable under DirectX 11. DirectX 11 scores have varied widely at times, and we’ve seen one configuration flip between 1.4 million draw calls per second and 1.9 million draw calls per second based on indeterminable factors.

Our best guess right now is that the variability comes from the much greater overhead of DirectX 11, and consequently all of the work that the API, video drivers, and OS are all undertaking in the background. Consequently the DirectX 11 results are good enough for what the AOFT has set out to do – showcase just how much incredibly faster DX12 and Mantle are – but it has a much higher degree of variability than our standard tests and should be treated accordingly.

Meanwhile Futuremark for their part is looking to make it clear that this is first and foremost a test to showcase API differences, and is not a hardware test designed to showcase how different components perform.

The purpose of the test is to compare API performance on a single system. It should not be used to compare component performance across different systems. Specifically, this test should not be used to compare graphics cards, since the benefit of reducing API overhead is greatest in situations where the CPU is the limiting factor.

We have of course gone and benchmarked a number of configurations to showcase how they benefit from DirectX 12 and/or Mantle, however as per Futuremark’s guidelines we are not looking to directly compare video cards. Especially since we’re often hitting the throughput limits of the command processor, something a real-world task would not suffer from.

The Test

Moving on, we also want to quickly point out the clearly beta state of the current WDDM 2.0 drivers. Of note, the DX11 results with NVIDIA’s 349.90 driver are notably lower than the results with their WDDM 1.3 driver, showing much greater variability. Meanwhile AMD’s drivers have stability issues, with our dGPU testbed locking up a couple of different times. So these drivers are clearly not at production status.

DirectX 12 Support Status
  Current Status Supported At Launch
AMD GCN 1.2 (285) Working Yes
AMD GCN 1.1 (290/260 Series) Working Yes
AMD GCN 1.0 (7000/200 Series) Working Yes
NVIDIA Maxwell 2 (900 Series) Working Yes
NVIDIA Maxwell 1 (750 Series) Working Yes
NVIDIA Kepler (600/700 Series) Working Yes
NVIDIA Fermi (400/500 Series) Not Active Yes
Intel Gen 7.5 (Haswell) Working Yes
Intel Gen 8 (Broadwell) Working Yes

And on that note, it should be noted that the OS and drivers are all still in development. So performance results are subject to change as Windows 10 and the WDDM 2.0 drivers get closer to finalization.

One bit of good news is that DirectX 12 support on AMD GCN 1.0 cards is up and running here, as opposed to the issues we ran into last month with Star Swarm. So other than NVIDIA’s Fermi cards, which aren’t turned on in beta drivers, we have the ability to test all of the major x86-paired GPU architectures that support DirectX 12.

For our actual testing, we’ve broken down our testing for dGPUs and for iGPUs. Given the vast performance difference between the two and the fact that the CPU and GPU are bound together in the latter, this helps to better control for relative performance.

On the dGPU side we are largely reusing our Star Swarm test configuration, meaning we’re testing the full range of working DX12-capable GPU architectures across a range of CPU configurations.

DirectX 12 Preview dGPU Testing CPU Configurations (i7-4960X)
Configuration Emulating
6C/12T @ 4.2GHz Overclocked Core i7
4C/4T @ 3.8GHz Core i5-4670K
2C/4T @ 3.8GHz Core i3-4370

Meanwhile on the iGPU side we have a range of Haswell and Kaveri processors from Intel and AMD respectively.

CPU: Intel Core i7-4960X @ 4.2GHz
Motherboard: ASRock Fatal1ty X79 Professional
Power Supply: Corsair AX1200i
Hard Disk: Samsung SSD 840 EVO (750GB)
Memory: G.Skill RipjawZ DDR3-1866 4 x 8GB (9-10-9-26)
Case: NZXT Phantom 630 Windowed Edition
Monitor: Asus PQ321
Video Cards: AMD Radeon R9 290X
AMD Radeon R9 285
AMD Radeon HD 7970
NVIDIA GeForce GTX 980
NVIDIA GeForce GTX 750 Ti
NVIDIA GeForce GTX 680
Video Drivers: NVIDIA Release 349.90 Beta
AMD Catalyst 15.200.1012.2 Beta
OS: Windows 10 Technical Preview (Build 10041)

 

CPU: AMD A10-7850K
AMD A10-7700K
AMD A8-7600
AMD A6-7400L
Intel Core i7-4790K
Intel Core i5-4690
Intel Core i3-4360
Intel Core i3-4130T
Pentium G3258
Motherboard: GIGABYTE F2A88X-UP4 for AMD
ASUS Maximus VII Impact for Intel LGA-1150
Zotac ZBOX EI750 Plus for Intel BGA
Power Supply: Rosewill Silent Night 500W Platinum
Hard Disk: OCZ Vertex 3 256GB OS SSD
Memory: G.Skill 2x4GB DDR3-2133 9-11-10 for AMD
G.Skill 2x4GB DDR3-1866 9-10-9 at 1600 for Intel
Video Cards: AMD APU Integrated
Intel CPU Integrated
Video Drivers: AMD Catalyst 15.200.1012.2 Beta
Intel Driver Version 10.18.15.4124
OS: Windows 10 Technical Preview (Build 10041)
3DMark API Overhead Feature Test Discrete GPU Testing
POST A COMMENT

113 Comments

View All Comments

  • nrexz - Friday, March 27, 2015 - link

    How much can they do with it really? Games will still be developed to the limits of the consoles not pc's.

    Also, I'm not sure if I should be impressed or sad that Forbes published this yesterday.
    Reply
  • nathanddrews - Friday, March 27, 2015 - link

    That might be an oversimplification. If anything, this could result in console ports NOT running like crap. What's the biggest complaint about ports? That the game is tailored "to the metal" of a console, making port to such a variety of PCs more difficult to develop.

    Think about it - when designing games around the Xbone/PS4, they tailor the games for eight cores and are not restricted by DX11 call limits or RAM - only the GPU power, but then when they port to PC, those optimized engines have to sludge through the DX11 pipeline before tapping into the GPU. With that restrictive pipeline removed (and GPUs shipping more RAM), those game engines can operate more efficiently in multicore PCs.

    It's not a cure-all (low-res textures), but I think this could be the start of a revolution in which ports stop sucking.
    Reply
  • Flunk - Friday, March 27, 2015 - link

    The current consoles both have 8GB of RAM, all of which is GPU-addressable, so texture resolution shouldn't really be a problem.

    Also, the Xbox One is built around DX11 so this will be helpful for that. Frankly DirectX 12 will be helpful because it will make Xbox One -> PC ports easier so hopefully we'll either see more of them or see better ports.

    It's not really a big worry anyway, this is quite likely the last console generation anyway.
    Reply
  • happycamperjack - Friday, March 27, 2015 - link

    Only about 5 to 5.5GB of RAM is consoles are usable. The rest are reserved by OS. Reply
  • Laststop311 - Saturday, March 28, 2015 - link

    I think its actually 3-3.5GB reserved for the system so 4.5-5GB available to the GPU Reply
  • nathanddrews - Friday, March 27, 2015 - link

    My comment about low-res textures has to do with the fact that Xbone/PS4 don't always use the same high-res texture packs available to PC users and that DX12 won't help with that in either scenario.

    The API Xbone runs is far removed from its PC counterpart. It's a heavily modified "Direct3D 11.X" that is built exclusively for Xbone, which removes the overhead that Windows DX11 has to deal with. In PC terms, it's effectively a superset of DX11.2 features running with DX12 efficiency.

    "Microsoft, though, claims that the Direct3D 11.X for Xbox One features significant enhancements in the area of runtime overhead, which results in a very streamlined, 'close to metal' level of runtime performance."
    Reply
  • DERSS - Saturday, March 28, 2015 - link

    "Close to metal"?

    Maybe they meant "Close to silicon"? Or they meant to compare with Apple's Metal for iOS?
    Reply
  • deruberhanyok - Saturday, March 28, 2015 - link

    It's just an expression.

    Way back before Apple had "Metal", ATI had "Close to Metal" (http://en.wikipedia.org/wiki/Close_to_Metal), and even earlier than that, S3 had their own API, also called Metal.

    It just means being able to code with as little overhead as possible. The idea is to have very little between the application and the hardware running it, to get as close to the maximum possible performance as you can.
    Reply
  • Navvie - Tuesday, March 31, 2015 - link

    The term goes back to the C64 and Amiga demo scenes. Programming in assembler without an API and literally "hitting the metal".

    Silicon is a metalloid element, so "hitting the metal" doesn't really need correcting.
    Reply
  • Kidster3001 - Wednesday, April 01, 2015 - link

    The 'metal' comes from an old saying: 'bare metal'. It's still used in the compute industry when referring to special testing that bypasses OS and driver layers, talking to the silicon directly. Reply

Log in

Don't have an account? Sign up now