Frame Time Consistency & Recordings

Last, but not least, we wanted to also look at frame time consistency across Star Swarm, our two vendors, and the various APIs available to them. Next to CPU efficiency gains, one of the other touted benefits of low-level APIs like DirectX 12 is the ability for developers to better control frame time pacing due to the fact that the API and driver are doing fewer things under the hood and behind an application’s back. Inefficient memory management operations, resource allocation, and shader compiling in particular can result in unexpected and undesirable momentary drops in performance. However, while low-level APIs can improve on this aspect, it doesn’t necessarily mean high-level APIs are bad at it. So it is an important distinction between bad/good and good/better.

On a technical note, these frame times are measured within (and logged by) Star Swarm itself. So these are not “FCAT” results that are measuring the end of the pipeline, nor is that possible right now due to the lack of an overlay option for DirectX 12.

Starting with the GTX 980, we can immediately see why we can’t always write-off high-level APIs. Benchmark non-determinism aside, both DirectX 11 and DirectX 12 produce consistent frame times; one is just much, much faster than the other. Both on paper and subjectively in practice, Star Swarm has little trouble maintaining consistent frame times on the GTX 980. Even if DirectX 11 is slow, it is at least consistent.

The story is much the same for the R9 290X. DirectX 11 and DirectX 12 both produce consistent results, with neither API experiencing frame time swings. Meanwhile Mantle falls into the same category as DirectX 12, producing similarly consistent performance and frame times.

Ultimately it’s clear from these results that if DirectX 12 is going to lead to any major differences in frame time consistency, Star Swarm is not the best showcase for it. With DirectX 11 already producing consistent results, DirectX 12 has little to improve on.

Finally, along with our frame time consistency graphs, we have also recorded videos of shorter run-throughs on both the GeForce GTX 980 and Radeon R9 290X. With YouTube now supporting 60fps, these videos are frame-accurate representations of what we see when we run the Star Swarm benchmark, showing first-hand the overall frame time consistency among all configurations, and of course the massive difference in performance.

Mid Quality Performance First Thoughts
Comments Locked

245 Comments

View All Comments

  • jeffkibuule - Saturday, February 7, 2015 - link

    Right, it's a secret to the public, not so much to the engineers.
  • Jumangi - Friday, February 6, 2015 - link

    MS is a much bigger company so the resources they have give a big edge. I also suspect while AMD publicly still supports Mantle they probably aren't doing much for the future as they are smart enough, or I hope they are smart enough, to realize DX 12 makes Mantle irrelevant.
  • toffty - Friday, February 6, 2015 - link

    I wouldn't say Mantle is irrelevant since Directx12 is Windows 10 and Xbox One only. Mantle is for Window < 10, linux and OSX. It will be competing against OpenGL in that space, true, but if it's easier to port applications made with Mantle to Xbox One, Mantle will have a leg up on OpenGL. Mantle also needs to becomes open and nVidia supports it too.
  • Penti - Saturday, February 7, 2015 - link

    There is no mantle for anything else than Windows. PS4 uses it's own API's, Nintendo uses it's own API's, X1 doesn't straight up run the same runtime or API as DX on Windows and already have low-level features, D3D11.X is a superset of D3D/DX11. Games are normally DX/HLSL to begin with, so you don't need to mess around with shaders that much and converting formats. Converting GLSL to HLSL is practical though. Many engines also has their own shader languages and systems making stuff like Mantle's shading language similarities irrelevant. Most will design for something else as their first path than Mantle.
  • Jumangi - Saturday, February 7, 2015 - link

    Fanboys can get mad but Linux and OSX are irrelevant for mainstream gaming. So yes Mantle has no real future and will be forgotten about in a couple of years.
  • bloodypulp - Sunday, February 8, 2015 - link

    Keep on dreaming, Nvidia fangirl. Mantle is coming for SteamOS(Linux). Count on it.
  • yannigr2 - Friday, February 6, 2015 - link

    Mantle was the catalyst to bring DX12 closer. No reason for AMD to spend much resources on Mantle now. Job is done.
  • Notmyusualid - Friday, February 6, 2015 - link

    More or less my thinking too. Best to keep us all on the same page, so to speak.

    Respect to AMD, but let us all join the DX12 train together....
  • AnnonymousCoward - Saturday, February 7, 2015 - link

    Not necessarily. How do you know it wasn't MS's idea even? Look at Khato's post above.
  • tipoo - Friday, February 6, 2015 - link

    What's the support status of Intels chips which are going to get it? I think all Iris models will, and a few of the higher end HD Graphics series parts will. Are they currently supported?

    It would be interesting to see if this could push the Iris Pro 5200 any further. Though this is more for the flip situation of a weak CPU with a strong GPU, rather than this strong CPU with a modest GPU.

Log in

Don't have an account? Sign up now