Frame Time Consistency & Recordings

Last, but not least, we wanted to also look at frame time consistency across Star Swarm, our two vendors, and the various APIs available to them. Next to CPU efficiency gains, one of the other touted benefits of low-level APIs like DirectX 12 is the ability for developers to better control frame time pacing due to the fact that the API and driver are doing fewer things under the hood and behind an application’s back. Inefficient memory management operations, resource allocation, and shader compiling in particular can result in unexpected and undesirable momentary drops in performance. However, while low-level APIs can improve on this aspect, it doesn’t necessarily mean high-level APIs are bad at it. So it is an important distinction between bad/good and good/better.

On a technical note, these frame times are measured within (and logged by) Star Swarm itself. So these are not “FCAT” results that are measuring the end of the pipeline, nor is that possible right now due to the lack of an overlay option for DirectX 12.

Starting with the GTX 980, we can immediately see why we can’t always write-off high-level APIs. Benchmark non-determinism aside, both DirectX 11 and DirectX 12 produce consistent frame times; one is just much, much faster than the other. Both on paper and subjectively in practice, Star Swarm has little trouble maintaining consistent frame times on the GTX 980. Even if DirectX 11 is slow, it is at least consistent.

The story is much the same for the R9 290X. DirectX 11 and DirectX 12 both produce consistent results, with neither API experiencing frame time swings. Meanwhile Mantle falls into the same category as DirectX 12, producing similarly consistent performance and frame times.

Ultimately it’s clear from these results that if DirectX 12 is going to lead to any major differences in frame time consistency, Star Swarm is not the best showcase for it. With DirectX 11 already producing consistent results, DirectX 12 has little to improve on.

Finally, along with our frame time consistency graphs, we have also recorded videos of shorter run-throughs on both the GeForce GTX 980 and Radeon R9 290X. With YouTube now supporting 60fps, these videos are frame-accurate representations of what we see when we run the Star Swarm benchmark, showing first-hand the overall frame time consistency among all configurations, and of course the massive difference in performance.

Mid Quality Performance First Thoughts
Comments Locked

245 Comments

View All Comments

  • Stuka87 - Friday, February 6, 2015 - link

    Very promising from the looks of it. Should help out a lot with some of these newer games.

    One thing I wish you would have done was to show kernel times on the CPU graphs. Would be nice to know how much of that CPU was userspace, and how much was kernel.
  • unacomn - Friday, February 6, 2015 - link

    Very promising results for future games.
    I would love to see a test with AMD CPUs as well, curious if the architecture of the current AM3+ and FM2 CPUs benefits much in terms of performance from this.
  • Stuka87 - Friday, February 6, 2015 - link

    For Mantle they benefit more, as they are a larger bottleneck. With my old AMD system (965BE @4GHz with an HD7950) BF4 was unplayable with DirectX11, but with Mantle it ran pretty decent.
  • mikato - Friday, February 6, 2015 - link

    It seems really strange to read that you couldn't run BF4 with those specs. I play the newest Call of Duty games with my 965BE (not overclocked) and my HD6950 unlocked to 6970. Settings are turned down a bit with my 1920x1200 resolution, but it runs well enough to play as twitchy fast as I need to be. I am in the market for an upgrade soon though so I can turn the settings back up.
  • shing3232 - Friday, February 6, 2015 - link

    BF4 are way more demanding than COD because there are too much object in MP especially when you playing a 64 player server.
  • XFR - Friday, February 6, 2015 - link

    Are these articles edited or proofread?
  • JarredWalton - Friday, February 6, 2015 - link

    Sometimes. I did make a reading/editing pass after this was posted, so if there are still errors let us know.
  • Jtaylor1986 - Friday, February 6, 2015 - link

    My only question is how is Mantle still in beta at this stage of the game? Microsoft seems to have developed DirectX 12 much more quickly than AMD has developed Mantle
  • jeffkibuule - Friday, February 6, 2015 - link

    It's more likely they were unknowingly developed side-by-side, but DirectX 12 was unveiled much later. Certainly if there were new DX12-only features that require special hardware, some AMD engineers would have wondered why Microsoft was requesting XYZ when developing the Xbox One SoC vs. PS4 (not that those two groups or general engineering teams would have been able to compare notes).
  • Khato - Friday, February 6, 2015 - link

    FYI - public announcement comes a long, long time after the initial specification. More importantly, directx is more of a collaboration between the players involved (Microsoft, Intel, NVIDIA, AMD) any more, not something that Microsoft develops in secret and only shares once it's finalized. Which is to say that AMD has known about the direction for directx 12 for over 2 years now.

Log in

Don't have an account? Sign up now