Our First FCAT & The Test

First announced back at the end of March, FCAT has been something of a bewildering experience for us. NVIDIA has actually done a great job on the software, but between picky games, flaky DVI cables, and dead SSDs (we killed an Intel enterprise grade SSD 910 with FCAT) things have not gone quite to plan, pushing back our intended use of FCAT more than once. In any case, with most of the kinks worked out we’re ready to start integrating it into our major GPU reviews.

For the time being we’re putting FCAT on beta status, as we intend to try out a few different methods of presenting data to find something that’s meaningful, useful, and legible. To that end we’d love to get your feedback in our comments section so that we can further iterate on our presentation and data collection.

We’ve decided to go with two metrics for our first run with FCAT. The first metric is rather simple: 95th percentile frametimes. For years we’ve done minimum framerates (when practical), which are similar in concept, so this allows us to collect similar stats at the end of the rendering pipeline while hopefully avoiding some of the quirkiness that comes from looking at minimum framerates within games themselves. The 95th percentile frametime is quite simply the amount of time it takes to render the slowest 5% of frames. If a game or video card is introducing significant one-off stuttering by taking too long to render some frames, this will show us.

This is primarily meant to capture single-GPU issues, but in practice with AMD having fixed the bulk of their single-GPU issues months ago, we don’t actually expect much. None the less it’s a good way of showing that nothing interesting is happening in those situations.

Our second metric is primarily focused on multi-GPU setups, and is an attempt to quantize the wild frametime variations seen at times with multi-GPU setups, which show up as telltale zigzag lines in frametime graphs.

In this metric, which for the moment we’re calling Delta Percentages, we’re collecting the deltas (differences) between frametimes, averaging that out, and then running the delta average against the average frametime of the entire run. The end result of this process is that we can measure whether sequential frames are rendering in roughly the same amount of time, while controlling for performance differences by looking at the data relative to the average frametime (rather than as absolute time).

In general, a properly behaving single-GPU card should have a delta average of under 3%, with the specific value depending in part on how variable the workload is throughout any given game benchmark. 3% may sound small, but since we’re talking about an average it means it’s weighed against the entire run. The higher the percentage the more unevenly frames are arriving, and exceeding 3% is about where we expect players with good eyes to start noticing a difference. Alternatively in a perfectly frame metered situation, such as v-sync enabled with a setup that can always hit 60fps, then this would be a flat 0%, representing the pinnacle of smoothness.

Moving on, we’ll be running FCAT against 6 of our 10 games for the time being: Sleeping Dogs, Hitman: Absolution, Total War: Shogun 2, Battlefield 3, Bioshock, and Crysis 3. The rest of our games are either highly inconsistent or generally fussy, introducing too much variance into our FCAT results.

Finally, due to the amount of additional time it takes to put together FCAT results, we’re going to primarily publish FCAT results with major product launches and major driver updates. Due to how frame metering works, the only time frame consistency significantly changes is either with the introduction of new architectures/GPUs, or with the introduction of significant driver changes, so those are the scenarios we’ll be focusing on.

The Test

NVIDIA’s launch drivers for the GTX 780 are 320.18, drivers that are essentially identical to the public 320.14 drivers released last week.

CPU: Intel Core i7-3960X @ 4.3GHz
Motherboard: EVGA X79 SLI
Power Supply: Antec True Power Quattro 1200
Hard Disk: Samsung 470 (256GB)
Memory: G.Skill Ripjaws DDR3-1867 4 x 4GB (8-10-9-26)
Case: Thermaltake Spedo Advance
Monitor: Samsung 305T
Video Cards: AMD Radeon HD 7970 GHz Edition
AMD Radeon HD 7990
NVIDIA GeForce GTX 580
NVIDIA GeForce GTX 680
NVIDIA GeForce GTX 690
NVIDIA GeForce GTX 780
NVIDIA GeForce GTX Titan
Video Drivers: NVIDIA ForceWare 320.14
NVIDIA ForceWare 320.18
AMD Catalyst 13.5 Beta 2
OS: Windows 8 Pro

 

Software, Cont: ShadowPlay and "Reason Flags" DiRT: Showdown
Comments Locked

155 Comments

View All Comments

  • ambientblue - Thursday, August 8, 2013 - link

    you are a sucker if you are willing to pay so much for twice the vram and 10% performance over the 780... if you got your titans before the 780 was released then sure its a massive performance boost over 680s but that's because the 680s should have been cheaper and named 660, and titan should have cost the amount the 680 was going for. You wont be so satisfied when the GTX 880 comes out and obliterates your titan at half the cost. THen again with that kind of money youll probably just buy 3 of those.
  • B3an - Thursday, May 23, 2013 - link

    I'd REALLY like to see more than just 3GB on high end cards. It's not acceptable. With the upcoming consoles having 8GB (with atleast 5GB+ usable for games) then even by the end of this year we may start seeing current high-end PC GPU's struggling due to lack of graphics RAM. These console games will have super high res textures, and when ported to PC, 3GB graphics RAM will not cut it at high res. I also have 2560x1600 monitors, and theres no way X1/PS4 games are going to run at this res with just 3GB. Yet the whole point of a high-end card is for this type of res as it's wasted on 1080p crap.

    Not enough graphics RAM was also a problem years ago on high-end GPU's. I remember having a 7950 G2X with only 512MB (1GB total but 512MB for each GPU) and it would get completely crippled (single digit FPS) from running games at 2560x1600 or even 1080p. Once you hit the RAM limit things literally become a slideshow. I can see this happening again just a few months from now, but to pretty much EVERYONE who doesn't have a Titan with 6GB.

    So i'm basically warning people thinking of buy a high-end card at this point - you seriously need to keep in mind that just a few months from now it could be struggling due to lack of graphics RAM. Either way, don't expect your purchase to last long, the RAM issue will definitely be a problem in the not too distant future (give it 18 months max).
  • Vayra - Thursday, May 23, 2013 - link

    How can you be worried about the console developments, and especially when it comes to VRAM of all things, when even the next-gen consoles are now looking to be no more than 'on-par' with todays' PC performance in games. I mean, the PS4 is just a glorified midrange GPU in all respects and so is the X1 even though they treat things a bit differently, not using GDDR5. Even the 'awesome' Killzone and CoD Ghost trailers show dozens of super-low-res textures and areas completely greyed out so as not to consume performance. All we get with the new consoles is that finally, 2011's 'current-gen' DX11 tech is coming to the console @ 1080p. But both machines will be running on that 8GB as their TOTAL RAM, and will be using it for all their tasks. Do you really think any game is going to eat up 5 Gigs of VRAM on 1080p? Even Crysis 3 on PC does not do that on its highest settings (it just peaks at/over 3gb I believe?) at 1440p.

    Currently the only reason to own a gpu or system with over 2 GB of VRAM is because you play at ultra settings at a reso over 1080p. For 1080p, which is what 'this-gen' consoles are being built for (sadly...) 2 GB is still sufficient and 3 GB is providing headroom.

    Hey, and last but not least, Nvidia has to give us at least ONE reason to still buy those hopelessly priced Titans off them, right?

    Also, aftermarket versions of the 780 will of course be able to feature more VRAM as we have seen with previous generations on both camps. I'm 100% certain we will be seeing 4 GB versions soon.
  • B3an - Friday, May 24, 2013 - link

    The power of a consoles GPU has nothing to do with it. Obviously these consoles will not match a high-end PC, but why would they have to in order to use more VRAM?! Nothing is stopping a mid-range or even a low-end PC GPU from using 4GB VRAM if it wanted to. Same with consoles. And obviously they will not use all 8GB for games (as i pointed out) but we're probably looking at atleast 4 - 5GB going towards games. The Xbox One for example is meant to use up to 3GB for the OS and other stuff, the remaining 5GB is totally available to games (or it's looking like that). Both the X1 and PS4 also have unified memory, meaning the GPU can use as much as it wants that isn't available to the OS.

    Crysis 3 is a bad example because this game is designed with ancient 8 year old console hardware in mind so it's crippled from the start even if it looks better on PC. When we start seeing X1/PS4 ports to PC the VRAM usage will definitely jump up because textures WILL be higher res and other things WILL be more complex (level design, physics, enemy A.I and so on). Infact the original Crysis actually has bigger open areas and better physics (explosions, mowing down trees) than Crysis 3 because it was totally designed for PC at the time. This stuff was removed in Crysis 3 because they had to make it play exactly the same across all platforms.

    I really think games will eat up 4+GB of VRAM within the next 18 months, especially at 2560x1600 and higher, and atleast use over 3GB at 1080p. The consoles have been holding PC's back for a very very long time. Even console ports made for ancient console hardware with 512MB VRAM can already use over 2GB on the PC version with enough AA + AF at 2560x1600. So thats just 1GB VRAM left on a 3GB card, and 1GB is easily gone by just doubling texture resolution.
  • Akrovah - Thursday, May 23, 2013 - link

    You're forgetting that on these new consoles that 8GB is TOTAL system memory, not just the video RAM. While on a PC you have the 3GB of VRAM here plus the main system memory (probably around 8 Gigs beign pretty stnadard at thsi point).

    I can guarantee you the consoles are not using that entire amount, or even the 5+ availabe for games, as VRAM. And this part is just me talking out of my bum, but I doubt many games on these consoles will use more than 2GB of teh unified memory for VRAM.

    Also I don;t think the res has much to do with the video memory any more. Some quick math and even if the game is tripple buffering a resolution of 2560x1600 only needs about 35 Megs of storage. Unless my math is wrong
    2560x1600 = 4,096,000 pixels at 24 bits each = 98,304,000 bits to store a single completed frame.
    divide by 8 = 12,288,000 bytes /1024 = 12,000 KiB / 1024 = 11.72 MiB per frame.

    Somehow I don't think modern graphic card's video memory has anythign to do with screen resolution and mostly is used by the texture data.
  • inighthawki - Thursday, May 23, 2013 - link

    Most back buffer swap chains are created with 32-bit formats, and even if they are not, chances are the hardware would convert this internally to a 32-bit format for performance to account for texture swizzling and alignment costs. Even so, a 2560x1600x32bpp back buffer would be 16MB, so you're looking at 32 or 48MB for double and triple buffering, respectively.

    But you are right, the vast majority of video memory usage will come from high resolution textures. A typical HD texture is already larger than a back buffer (2048x2048 is slightly larger than 2560x1600) and depending on the game engine may have a number of mip levels also loaded, so you can increase the costs by about 33%. (I say all of this assuming we are not using any form of texture compression just for the sake of example).

    I also hope anyone who buys a video card with large amounts of ram is also running 64-bit Windows :), otherwise their games can't maximize the use of the card's video memory.
  • Akrovah - Friday, May 24, 2013 - link

    I was under the impression that on a 32 bit rendering pipeline the upper 8 bits were used for transparancy calulation, but that it was then filtered down to 24 bits when actually written to the buffer because that's how displays take information.

    But then I just made that up in my own mind because I don't actually know how or when the 32-bit render - 24-bit display conversion takes place.

    But assuming I was wrong and what you say is correct (a likely scenario in this case) my previous point still stands.
  • jonjonjonj - Thursday, May 23, 2013 - link

    i wouldn't be worried. the lowend cpu and apu in consoles wont be pushing anything. the consoles are already outdated and they haven't even been launched. the consoles have 8GB TOTAL memory not 8GB of vram.
  • B3an - Friday, May 24, 2013 - link

    Again, the power of these consoles has absolutely nothing to do with how much VRAM they can use. If a low-end PC GPU existed with 4GB VRAM, it can easily use all that 4GB if it wanted to.

    And it's unified memory in these consoles. It all acts as VRAM. ALL of the 8GB is available to the GPU and games that isn't used by the OS (which is apparently 3GB on the Xbox One for OS/other tasks, leaving 5GB to games).
  • Akrovah - Friday, May 24, 2013 - link

    No, it doesn't all act as VRAM. You still have your data storage objects like all your variables (of which a game can have thousands) AI objects, pathfinding data, all the corodiantes for everything in the current level/map/whatever. Basically the entire state of the game that is operating behind the scenes. This is not insignifigant.

    All the non OS used RAM is available to the games yes, but games are storing a hell of alot more data than what is typically stored in video RAM. Hence PC games that need 2 GB of RAM also oly require 512 Megs of VRAM.

Log in

Don't have an account? Sign up now