Frame Time Consistency & Recordings

Last, but not least, we wanted to also look at frame time consistency across Star Swarm, our two vendors, and the various APIs available to them. Next to CPU efficiency gains, one of the other touted benefits of low-level APIs like DirectX 12 is the ability for developers to better control frame time pacing due to the fact that the API and driver are doing fewer things under the hood and behind an application’s back. Inefficient memory management operations, resource allocation, and shader compiling in particular can result in unexpected and undesirable momentary drops in performance. However, while low-level APIs can improve on this aspect, it doesn’t necessarily mean high-level APIs are bad at it. So it is an important distinction between bad/good and good/better.

On a technical note, these frame times are measured within (and logged by) Star Swarm itself. So these are not “FCAT” results that are measuring the end of the pipeline, nor is that possible right now due to the lack of an overlay option for DirectX 12.

Starting with the GTX 980, we can immediately see why we can’t always write-off high-level APIs. Benchmark non-determinism aside, both DirectX 11 and DirectX 12 produce consistent frame times; one is just much, much faster than the other. Both on paper and subjectively in practice, Star Swarm has little trouble maintaining consistent frame times on the GTX 980. Even if DirectX 11 is slow, it is at least consistent.

The story is much the same for the R9 290X. DirectX 11 and DirectX 12 both produce consistent results, with neither API experiencing frame time swings. Meanwhile Mantle falls into the same category as DirectX 12, producing similarly consistent performance and frame times.

Ultimately it’s clear from these results that if DirectX 12 is going to lead to any major differences in frame time consistency, Star Swarm is not the best showcase for it. With DirectX 11 already producing consistent results, DirectX 12 has little to improve on.

Finally, along with our frame time consistency graphs, we have also recorded videos of shorter run-throughs on both the GeForce GTX 980 and Radeon R9 290X. With YouTube now supporting 60fps, these videos are frame-accurate representations of what we see when we run the Star Swarm benchmark, showing first-hand the overall frame time consistency among all configurations, and of course the massive difference in performance.

Mid Quality Performance First Thoughts
Comments Locked

245 Comments

View All Comments

  • OrphanageExplosion - Sunday, February 8, 2015 - link

    On a tiny minority of titles.
  • bloodypulp - Sunday, February 8, 2015 - link

    Battlefield 4
    Battlefield Hardline
    Thief
    Star Citizen
    Plants vs. Zombies: Garden Warfare
    Civilization: Beyond Earth
    Dragon Age: Inquisition
    Mirror's Edge 2
    Sniper Elite 3
    ... and growing every day.
  • bloodypulp - Sunday, February 8, 2015 - link

    Who needs to wait for DX12? Mantle is running great for me right now. :)
  • sireangelus - Sunday, February 8, 2015 - link

    would you do one quick test using an 8core fx?
  • johnny_boy - Sunday, February 8, 2015 - link

    Would have loved to see this, and some lower end CPUs even.
  • editorsorgtfo - Sunday, February 8, 2015 - link



    What about threaded CPUs ? for example 1 core 2 threads old pentium CPUs and 2 cores 4 threads i3 CPUs ? can you still count that them as 2 cores and 4 cores ?

    I wanna ask this on the anandtech comment section but I don't have an account there XD
  • boe - Sunday, February 8, 2015 - link

    What I care about are great graphics. It is a shame there is no Crytek 4 engine to show off what DX12 could do. MS should have hired the original crytek developers to create some showpiece game.
  • Gigaplex - Monday, February 9, 2015 - link

    The API won't really change what you can do compared to DX11 other than reduce some system requirements. The feature levels are what provides new eye candy, and this preview doesn't cover that aspect. Wait until it hits retail, you'll probably see some fancy tech demos.
  • Thermalzeal - Sunday, February 8, 2015 - link

    I have one big question to ask.

    Since Direct X12 is resulting in significant performance gains, what is the potential for these improvements to translate over to the Xbox One? While I'm sure the Xbox One already has some of these bare metal improvements, due to the focus of the device...is it possible that DX12 will make the Xbox One more powerful than the PS4?
  • Ryan Smith - Sunday, February 8, 2015 - link

    "Since Direct X12 is resulting in significant performance gains, what is the potential for these improvements to translate over to the Xbox One?"

    Only Microsoft really knows the answer to that one. But I would be shocked beyond belief if the XB1's D3D 11.X API didn't already implement many of these optimizations. It is after all a fixed console, where low-level APIs have been a mainstay since day one.

    "is it possible that DX12 will make the Xbox One more powerful than the PS4?"

    In a word, no. The best case scenario for Microsoft is that Sony implements their own low-level API (if they haven't already) and we're back at square one. APIs can't make up for hardware differences when both parties have the means and influence to create what would be similar APIs.

Log in

Don't have an account? Sign up now