Frame Time Consistency & Recordings

Last, but not least, we wanted to also look at frame time consistency across Star Swarm, our two vendors, and the various APIs available to them. Next to CPU efficiency gains, one of the other touted benefits of low-level APIs like DirectX 12 is the ability for developers to better control frame time pacing due to the fact that the API and driver are doing fewer things under the hood and behind an application’s back. Inefficient memory management operations, resource allocation, and shader compiling in particular can result in unexpected and undesirable momentary drops in performance. However, while low-level APIs can improve on this aspect, it doesn’t necessarily mean high-level APIs are bad at it. So it is an important distinction between bad/good and good/better.

On a technical note, these frame times are measured within (and logged by) Star Swarm itself. So these are not “FCAT” results that are measuring the end of the pipeline, nor is that possible right now due to the lack of an overlay option for DirectX 12.

Starting with the GTX 980, we can immediately see why we can’t always write-off high-level APIs. Benchmark non-determinism aside, both DirectX 11 and DirectX 12 produce consistent frame times; one is just much, much faster than the other. Both on paper and subjectively in practice, Star Swarm has little trouble maintaining consistent frame times on the GTX 980. Even if DirectX 11 is slow, it is at least consistent.

The story is much the same for the R9 290X. DirectX 11 and DirectX 12 both produce consistent results, with neither API experiencing frame time swings. Meanwhile Mantle falls into the same category as DirectX 12, producing similarly consistent performance and frame times.

Ultimately it’s clear from these results that if DirectX 12 is going to lead to any major differences in frame time consistency, Star Swarm is not the best showcase for it. With DirectX 11 already producing consistent results, DirectX 12 has little to improve on.

Finally, along with our frame time consistency graphs, we have also recorded videos of shorter run-throughs on both the GeForce GTX 980 and Radeon R9 290X. With YouTube now supporting 60fps, these videos are frame-accurate representations of what we see when we run the Star Swarm benchmark, showing first-hand the overall frame time consistency among all configurations, and of course the massive difference in performance.

Mid Quality Performance First Thoughts
Comments Locked

245 Comments

View All Comments

  • nikon133 - Sunday, February 8, 2015 - link

    Windows Mobile was very strong smartphone/PDA OS, back in the pre iOS/Android days. I don't remember how it compared to Symbian, but I do remember that it overtook Palm (marketshare wise) at some point.

    Tablets, true... unless OP considers PDAs an early tablets.
  • sr1030nx - Sunday, February 8, 2015 - link

    Win10 (and 8.1 to a very limited extent) uses universal apps, meaning you write an app once and it works across PC, tablet and phones.
    Also means you only need to buy an app once and you get it everywhere.
  • Christopher1 - Sunday, February 8, 2015 - link

    The only person who is wrong here is you, hwangeruk. There are numerous people who have had that the whole "Free upgrade to Windows 10 from 7 and 8!" is directly due to Microsoft wanting to encourage people to not stay on old code and move to new, safer and faster code.
  • pixelstuff - Sunday, February 8, 2015 - link

    Windows 8 is not old code by any stretch. I think hwangeruk was right. Microsoft wants all Windows 8 users and especially Windows 7 users to upgrade so they can all run the unified apps that will also run on Windows Phone 10 devices. Microsoft eventually wants all developers writing Windows 10 apps even for things like Photoshop and Illustrator. To make that feasible Microsoft really needs to upgrade the majority of their user base.
  • domboy - Monday, February 9, 2015 - link

    But even if they do get everybody to upgrade to 10, that still doesn't guarantee developers are going to switch to the store model, especially the big ones as they'd have to start giving Microsoft a cut of their profits. Same for game developers... Steam is pretty well established at this point. I will probably upgrade since it's free, but really only so I don't get locked out of DirectX 12....
  • Naqoyqatsi - Monday, February 9, 2015 - link

    "In that scenario, you are the product."

    No, you are the laborer.
  • Frenetic Pony - Sunday, February 8, 2015 - link

    I suppose you only use iOS and OSX then as Chrome OS, Linux, and Android are also free?
  • SparkySamza - Tuesday, February 10, 2015 - link

    as the saying i hear still now and when i was a child " there is no such thing as a free lunch" i know microsoft has done something, from gathering data via the operating system or even maybe filling it with micro transactions or even forcing people to use xbox everything even though pc people want to be as far away from xbox as they can.
  • Cygni - Friday, February 6, 2015 - link

    This is a great way to ensure that developers are forced to continue coding for DX11, or just switch to Mantle. Windows 7 is going to continue to be the dominant OS long into the future.
  • Viewgamer - Friday, February 6, 2015 - link

    I'm sorry but does no one find it suspicious that a supposedly "165W" GTX 980 consumes only 14W less than the "290W" R9 290X in DX12 ? and 19W less in DX11.

    No matter how you slice it, it's abundantly clear that the 165W power figure from Nvidia is pure PR fabrication. Just like the fabricated specs for the GTX 970.

Log in

Don't have an account? Sign up now