Frame Time Consistency & Recordings

Last, but not least, we wanted to also look at frame time consistency across Star Swarm, our two vendors, and the various APIs available to them. Next to CPU efficiency gains, one of the other touted benefits of low-level APIs like DirectX 12 is the ability for developers to better control frame time pacing due to the fact that the API and driver are doing fewer things under the hood and behind an application’s back. Inefficient memory management operations, resource allocation, and shader compiling in particular can result in unexpected and undesirable momentary drops in performance. However, while low-level APIs can improve on this aspect, it doesn’t necessarily mean high-level APIs are bad at it. So it is an important distinction between bad/good and good/better.

On a technical note, these frame times are measured within (and logged by) Star Swarm itself. So these are not “FCAT” results that are measuring the end of the pipeline, nor is that possible right now due to the lack of an overlay option for DirectX 12.

Starting with the GTX 980, we can immediately see why we can’t always write-off high-level APIs. Benchmark non-determinism aside, both DirectX 11 and DirectX 12 produce consistent frame times; one is just much, much faster than the other. Both on paper and subjectively in practice, Star Swarm has little trouble maintaining consistent frame times on the GTX 980. Even if DirectX 11 is slow, it is at least consistent.

The story is much the same for the R9 290X. DirectX 11 and DirectX 12 both produce consistent results, with neither API experiencing frame time swings. Meanwhile Mantle falls into the same category as DirectX 12, producing similarly consistent performance and frame times.

Ultimately it’s clear from these results that if DirectX 12 is going to lead to any major differences in frame time consistency, Star Swarm is not the best showcase for it. With DirectX 11 already producing consistent results, DirectX 12 has little to improve on.

Finally, along with our frame time consistency graphs, we have also recorded videos of shorter run-throughs on both the GeForce GTX 980 and Radeon R9 290X. With YouTube now supporting 60fps, these videos are frame-accurate representations of what we see when we run the Star Swarm benchmark, showing first-hand the overall frame time consistency among all configurations, and of course the massive difference in performance.

Mid Quality Performance First Thoughts
Comments Locked

245 Comments

View All Comments

  • Archetype - Saturday, August 1, 2015 - link

    They did not optimize the hell out of it for NVidia. They just added DX12 support. Originally it was just for DX11 on any card and Mantle supported by AMD. It is not a game - Its a tech demo.

    The 980 obviously has significant horse power. I am just unsure why they used a 290x and not the current flagship. But maybe the 290x is still supposed to be more powerful.
  • Freosan - Sunday, August 2, 2015 - link

    Archetype. When this was published, the 290x WAS the current AMD flagship.
  • Azix - Monday, August 3, 2015 - link

    flagship was 295x2
  • Archetype - Saturday, August 1, 2015 - link

    They explained it quite clearly. Starswarm was written specifically to have zounds of draw calls and as such a high level API (layers and layers of API between the code that draws the scenes) will not be able to deal with it well. That was the whole point of Mantle and now DX12. To remove some of those layers of API and give software more direct access to underlying drivers and through them the hardware. You really need to get more informed.
  • Flunk - Friday, February 6, 2015 - link

    That might annoy me more if they weren't giving everyone on Windows 7 and 8 an upgrade to 10 for nothing. I suppose not having to backport this to Windows 8 (as was originally announced) is probably saving a fair amount.
  • dakishimesan - Friday, February 6, 2015 - link

    Sorry for my laziness, I was actually just pointing out a spelling mistake; I think it should say because DirectX 12… Etc.

    But for what it's worth I agree with you, with windows 10 being a free upgrade, and only enthusiast really caring about direct X 12 on their current high performance gear, I have no problem with Microsoft doing it this way.
  • Christopher1 - Monday, February 16, 2015 - link

    Exactly. I understood why people were reticent to upgrade to Windows 7 and 8 because it was a paid upgrade, but with Windows 10 being a free upgrade if you have Windows 7 or 8? No reason not to upgrade.
  • yuhong - Friday, February 6, 2015 - link

    Yea, the old DirectX redists are long dead.
  • Wwhat - Saturday, February 7, 2015 - link

    I'm highly suspicious about windows 10.
    There is a reason why they give it for free, as the saying goes "If You're Not Paying for It; You're the Product", which is generally true these days if it comes from a commercial outfit.
  • damianrobertjones - Saturday, February 7, 2015 - link

    The more that people use the Windows store the more profit MS makes. Simple.

Log in

Don't have an account? Sign up now