Our First FCAT & The Test

First announced back at the end of March, FCAT has been something of a bewildering experience for us. NVIDIA has actually done a great job on the software, but between picky games, flaky DVI cables, and dead SSDs (we killed an Intel enterprise grade SSD 910 with FCAT) things have not gone quite to plan, pushing back our intended use of FCAT more than once. In any case, with most of the kinks worked out we’re ready to start integrating it into our major GPU reviews.

For the time being we’re putting FCAT on beta status, as we intend to try out a few different methods of presenting data to find something that’s meaningful, useful, and legible. To that end we’d love to get your feedback in our comments section so that we can further iterate on our presentation and data collection.

We’ve decided to go with two metrics for our first run with FCAT. The first metric is rather simple: 95th percentile frametimes. For years we’ve done minimum framerates (when practical), which are similar in concept, so this allows us to collect similar stats at the end of the rendering pipeline while hopefully avoiding some of the quirkiness that comes from looking at minimum framerates within games themselves. The 95th percentile frametime is quite simply the amount of time it takes to render the slowest 5% of frames. If a game or video card is introducing significant one-off stuttering by taking too long to render some frames, this will show us.

This is primarily meant to capture single-GPU issues, but in practice with AMD having fixed the bulk of their single-GPU issues months ago, we don’t actually expect much. None the less it’s a good way of showing that nothing interesting is happening in those situations.

Our second metric is primarily focused on multi-GPU setups, and is an attempt to quantize the wild frametime variations seen at times with multi-GPU setups, which show up as telltale zigzag lines in frametime graphs.

In this metric, which for the moment we’re calling Delta Percentages, we’re collecting the deltas (differences) between frametimes, averaging that out, and then running the delta average against the average frametime of the entire run. The end result of this process is that we can measure whether sequential frames are rendering in roughly the same amount of time, while controlling for performance differences by looking at the data relative to the average frametime (rather than as absolute time).

In general, a properly behaving single-GPU card should have a delta average of under 3%, with the specific value depending in part on how variable the workload is throughout any given game benchmark. 3% may sound small, but since we’re talking about an average it means it’s weighed against the entire run. The higher the percentage the more unevenly frames are arriving, and exceeding 3% is about where we expect players with good eyes to start noticing a difference. Alternatively in a perfectly frame metered situation, such as v-sync enabled with a setup that can always hit 60fps, then this would be a flat 0%, representing the pinnacle of smoothness.

Moving on, we’ll be running FCAT against 6 of our 10 games for the time being: Sleeping Dogs, Hitman: Absolution, Total War: Shogun 2, Battlefield 3, Bioshock, and Crysis 3. The rest of our games are either highly inconsistent or generally fussy, introducing too much variance into our FCAT results.

Finally, due to the amount of additional time it takes to put together FCAT results, we’re going to primarily publish FCAT results with major product launches and major driver updates. Due to how frame metering works, the only time frame consistency significantly changes is either with the introduction of new architectures/GPUs, or with the introduction of significant driver changes, so those are the scenarios we’ll be focusing on.

The Test

NVIDIA’s launch drivers for the GTX 780 are 320.18, drivers that are essentially identical to the public 320.14 drivers released last week.

CPU: Intel Core i7-3960X @ 4.3GHz
Motherboard: EVGA X79 SLI
Power Supply: Antec True Power Quattro 1200
Hard Disk: Samsung 470 (256GB)
Memory: G.Skill Ripjaws DDR3-1867 4 x 4GB (8-10-9-26)
Case: Thermaltake Spedo Advance
Monitor: Samsung 305T
Video Cards: AMD Radeon HD 7970 GHz Edition
AMD Radeon HD 7990
NVIDIA GeForce GTX 580
NVIDIA GeForce GTX 680
NVIDIA GeForce GTX 690
NVIDIA GeForce GTX 780
NVIDIA GeForce GTX Titan
Video Drivers: NVIDIA ForceWare 320.14
NVIDIA ForceWare 320.18
AMD Catalyst 13.5 Beta 2
OS: Windows 8 Pro

 

Software, Cont: ShadowPlay and "Reason Flags" DiRT: Showdown
Comments Locked

155 Comments

View All Comments

  • varad - Thursday, May 23, 2013 - link

    You do realize that a GPU like Titan has almost 5 times the number of transistors compared to Intel's biggest Core i7 CPU, right? There are 7.1 billion transistors in Titan vs 1.4 billion in Core i7 3770k. So, it means they cannot match the price of "a good CPU" unless they decide to become a non-profit organization :)
  • AssBall - Thursday, May 23, 2013 - link

    Well if all you needed was a single titan to run your is, computations, games, and nothing else, then no problem.
  • krutou - Sunday, May 26, 2013 - link

    Two problems with your logic

    22 nm fabrication is more expensive (price per transistor)

    CPUs are more difficult to design
  • An00bis - Friday, May 31, 2013 - link

    it's not like you can just shove your hand in a jar full of transistors and just slap it on a chip and consider it a cpu, a cpu is required to do a gpu's task (integrated gpu) AND be good at everything a gpu can't do, which is... well lots of things actually. A gpu is much simpler, hence why the manufacturing + designing cost is probably more expensive than a big ass card that has to include memory+a pcb+a gpu
  • chizow - Thursday, May 23, 2013 - link

    Great card, but a year late. This is what GTX 600 series should've been but we all know how that went.

    I think Nvidia made some pretty big mistakes with how they handled the entire Kepler generation after Tahiti's launch price debacle. I know their financial statements and stockholders don't agree but they've managed to piss off their core consumers at every performance segment.

    Titan owners have to feel absolutely gutted at this point having paid $1000 for a part that is only ~10-15% faster than the GTX 780. End result of this generation is we are effectively paying 50-100% more for the same class of card than previous generations. While the 780 is a great card and a relatively good value compared to Titan, we're still paying $650 for what is effectively Kepler's version of the GTX 470.
  • Crisium - Thursday, May 23, 2013 - link

    People who bought a Titan knew what they were getting into. If you have regrets, you were in no position to buy a $1000 GPU to begin with and made a grievous financial error.

    $650 isn't horrible for this price, but you are still paying the Nvidia Tax.
  • chizow - Thursday, May 23, 2013 - link

    I don't think so, if you polled GTX Titan owners if they would've paid $1000 knowing 2-3 months later there would be a part that performed similarly at 35% less price, I think you would hear most of them would've waited to buy not 1, but 2 for just a bit more. Or instead of buying 2 Titans, buying 3x780s.

    Also, it really has nothing to do with being in a financial position or not, it's funny when Titan released I made the comment anyone interested in Titan would be better served by simply investing that money into Nvidia stock, letting that money grow on Titan's fat margins, and then buying 2x780s when they released. All according to plan, for my initial investment of 1 Titan I can buy 2x780s.

    But I won't. Nvidia blew it this generation, I'll wait for Maxwell.
  • IanCutress - Thursday, May 23, 2013 - link

    Titan was a compute card with optional gaming, rather than a gaming card with optional FP64 compute. That's why the price difference exists. If you bought a Titan card for Gaming, then you would/should have been smart enough to know that a similar card without compute was around the corner.
  • chizow - Thursday, May 23, 2013 - link

    Unfortunately, that was never how *GTX* Titan was marketed, straight from the horses mouth:
    "With the DNA of the world’s fastest supercomputer and the soul of NVIDIA® Kepler™ architecture, GeForce® GTX TITAN GPU is a revolution in PC gaming performance."

    Not to mention the fact Titan is a horrible compute card and value outside of CUDA workloads, and even there it suffers as a serious compute card due to the lack of ECC. It's an overpriced gaming card, plain and simple.

    At the time, it was still uncertain whether or not Nvidia would launch more SKUs based on GK110 ASIC, but informed consumers knew Nvidia had to do something with all the chips that didn't make the TDP cut as Tesla parts.
  • mayankleoboy1 - Thursday, May 23, 2013 - link

    Really ? Apart from a few apps, Titan is poor compared to a 7970. It has bad OpenGL performance, which 90% of industry renderfarms use.
    Titan is really an overpriced gaming card.

Log in

Don't have an account? Sign up now