FCAT: The Evolution of Frame Interval Benchmarking, Part 1
by Ryan Smith on March 27, 2013 9:00 AM ESTMore To Come
While we were unable to complete our work with FCAT ahead of NVIDIA’s embargo, we wanted to provide an article that at least gives a brief overview of FCAT, as FCAT is in many ways itself part two of a process we started yesterday with our article and analysis of stuttering on AMD cards.
FCAT, we believe, is the next evolution of frame interval benchmarking. Where FRAPS' coarse nature does not suffice, FCAT provides a clear picture of what’s happening at the end of the rendering pipeline, giving us for the first time an automated, quantitative look at frame intervals, stuttering, and more. To be clear it is by no means a perfect tool, but as we have taken the time to lay out yesterday and today, compared to the beginning of the rendering pipeline, it is the end of the rendering pipeline that is more meaningful both for quantitative analysis, and ultimately for the users.
Speaking more directly however, FCAT is quite simply the frame interval analysis tool we have long wanted. It is the tool that will enable us to analyze stuttering, micro-stuttering, and more, in a manner consistent with our benchmarking methods and core beliefs in the scientific method. It’s exceedingly rare that we say this, but we haven’t been this excited by a new benchmarking tool in a very long time.
Wrapping things up, we will be following up this article next week with part 2 in our look at FCAT. In part 2 we will go into further detail about how to analyze the results FCAT generates, and what we’re finding across a range of video cards and games, both in single-GPU and multi-GPU configurations. So until then, stay tuned.
Facebook
Twitter
RSS
88 Comments
View All Comments
ARealAnand - Tuesday, April 9, 2013 - link
I'm not sure you realize how development of silicon based semiconductor product works. This is not on the scale of a planting season where you put down some seed an later that year you harvest your crop. You start off with the specification phase of the product, you move to development of the hdl and verification of everything and then you go to nre and silicon samples. This is a multi-year process. AMD may well have known about this issue just as long as nvidia but silicon products don't go from specification to product overnight. That is why vendors offer driver/firmware/microcode updates. As to taking a pot shot at AMD laying off R&D people, it's called a business decision. Sometimes you need to let people go so that the rest of the employees can remain employed. Otherwise you can end up facing bankruptcy and massive layoffs. I don't know if it's the right move at this time or not, and I'm no financial analyst. Currently nvidia has no gpu chips directly that share a piece of silicon with the cpu, unlike AMD and intel. The next generation of consoles seem to have gone with AMD. I haven't heard of any big chipsets from nvidia. Anyway, sorry for the long response. I believe AMD, Intel, and nvidia all have strong points and areas where they can improve. This seems like it might be largely a driver issue and I'll admit that AMD seem to have had issues with drivers. Your post just seemed like an easy shot against AMD. I am not affiliated with AMD, nvidia, or Intel and the views expressed in this post are my own opinions and not those of any current or previous employer.arbiter9605 - Wednesday, April 17, 2013 - link
From what I heard/read somewhere, Nvidia new of such a problem back in their 8000 series card(8800gtx/gts/gt). So its something they knew about for a long time. Nvidia has hardware built in to their cards to keep the cards in sync frame wise where as AMD doesn't. Some time of 2 years for AMD to fix it properly as well as a software driver fix will most likely cost some fps.name99 - Thursday, March 28, 2013 - link
I'd like an AnandTech investigation of a slightly different issue:How fast does Apple update the screen on iPads and iPhones when displaying movies? Specifically, when displaying movies, do they switch to updating the screen only every 24th of a second?
The reason I think this is an interesting question is that, in my experience, movies displayed on iPad show none of the stuttering when panning that is so obvious on both TVs and computers, stuttering which is, as far as I can tell, generated by the 3:2 pullup. (I don't have a 120Hz TV, so I can't comment on how well they deal with this.
So we have the visual suggestion that this is what Apple is doing, along with the obvious point that it would presumably save power to only refresh the screen at 24Hz (though the power savings may be negligible).
I must admit I would find very interesting an investigation of this (perhaps by similar techniques to what are being used here, like a movie consisting of a color coded sequence of frames, and time-stamped video capture; though you'd probably have to use an external video camera.)
And this is not just an Apple specific issue; it would be interesting to know if Android and MS likewise are capable of displaying stutter-free movies on their mobile devices (unlike on the desktop where, sure, you have far less control, but for fsck's sake --- can't you at least do the job right in full screen mode?)
mayankleoboy1 - Thursday, March 28, 2013 - link
So where does VirtuPro and VirtuHyperformance etc. come in this picture ?wingless - Friday, March 29, 2013 - link
Great question. Enough Z77 mobos alone support this to warrant investigation.cactusdog - Thursday, March 28, 2013 - link
Wow, so this issue of stuttering has been talked about amongst users for at least a decade, then Scott Wasson from techreport decides to run a test of latency just weeks before Nvidia releases their super duper new tool to test latency?......and Nvidia have been working on it for 2 years? What a coincidence!! ...and NVidia cards perform better in this regard? Double coincidence!! So does that mean Nvidia is a benevolent company who wants to help AMD fix their stuttering issues so AMD can sell more cards?? Wow they must be saints!! We can add this to Nvidia's long list of open, honest and transparent business practices.cobalt42 - Thursday, March 28, 2013 - link
Not sure if joking or troll.... Scott's latency tests started in their August 23, 2012 article entitled "Inside the second: Gaming performance with today's CPUs". That's not "just weeks" ago. Second, if NVIDIA thinks Scott's FRAPS tests are so awesome for them, why would they bother to release a tool that measures at an entirely different point in the pipeline? Your conspiracy theory is not only factually wrong, it doesn't even make a good conspiracy theory.JPForums - Thursday, March 28, 2013 - link
I also agree with you and Spoelie. Simulation step stutter is also an issue that should be covered. It would be really nice to get simulation timestamps directly from the output of the simulator and match them up to their corresponding output frames. However, this would probably require collaboration with game developers that you probably won't get. Until then, using a tool that works at the output of the renderer (like FRAPS) and can associate simulation steps with output frames would be nice.That said, there is really little that AMD or Nvidia can do to fix issues in the application other than trick it into working correctly. These results would be more useful for game developers developing new engines. Also keep in mind, simulation steps are tied loosely (through queues) to GPU's processing capabilities (unless the bottleneck happens to be before the command is dispatched to D3D). Simulation steps should be roughly equal to frame times on average. If the GPU processing were completely consistent, then the latency between the simulation step and output would be fixed and the output would appear smooth. It is variations in frame times that cause variations in simulation steps. On average, the variations of each should be roughly equal. The worse case stutter, which should be something like double the frame time variation (when simulator is compensating it the opposite direction as the frame time variation), is what we need to look out for. That said, variation in frame time is generally smaller than frame time itself. I would suggest that simulation step stuttering is a smaller problem than frame time stuttering and becomes smaller as frame times get shorter. Point of interest, Nvidia's Frame metering may actually increase simulation step stuttering.
Unwinder - Thursday, March 28, 2013 - link
I've just added FCAT overlay rendering support to OSD server of MSI Afterburner and EVGA Precision. Still need some time to discuss exact RGB color sequence with NVIDIA, then I guess we'll release it to public.Ryan Smith - Friday, March 29, 2013 - link
Hi Unwinder;Thank you for the update and for adding support for this. I think it's a great relief all-around having someone besides NVIDIA providing the overlay.
It's too big to post in our comments, but if you need the colors hit me up. They're not secret. I can send you the list of precise colors in hex form.