FCAT: The Evolution of Frame Interval Benchmarking, Part 1
by Ryan Smith on March 27, 2013 9:00 AM ESTMore To Come
While we were unable to complete our work with FCAT ahead of NVIDIA’s embargo, we wanted to provide an article that at least gives a brief overview of FCAT, as FCAT is in many ways itself part two of a process we started yesterday with our article and analysis of stuttering on AMD cards.
FCAT, we believe, is the next evolution of frame interval benchmarking. Where FRAPS' coarse nature does not suffice, FCAT provides a clear picture of what’s happening at the end of the rendering pipeline, giving us for the first time an automated, quantitative look at frame intervals, stuttering, and more. To be clear it is by no means a perfect tool, but as we have taken the time to lay out yesterday and today, compared to the beginning of the rendering pipeline, it is the end of the rendering pipeline that is more meaningful both for quantitative analysis, and ultimately for the users.
Speaking more directly however, FCAT is quite simply the frame interval analysis tool we have long wanted. It is the tool that will enable us to analyze stuttering, micro-stuttering, and more, in a manner consistent with our benchmarking methods and core beliefs in the scientific method. It’s exceedingly rare that we say this, but we haven’t been this excited by a new benchmarking tool in a very long time.
Wrapping things up, we will be following up this article next week with part 2 in our look at FCAT. In part 2 we will go into further detail about how to analyze the results FCAT generates, and what we’re finding across a range of video cards and games, both in single-GPU and multi-GPU configurations. So until then, stay tuned.
Facebook
Twitter
RSS
88 Comments
View All Comments
wingless - Wednesday, March 27, 2013 - link
Is this NVIDIA developed tool unbiased towards AMD?Ryan Smith - Wednesday, March 27, 2013 - link
So that's a very good question, and one I didn't have time to adequately delve into for this article.The short answer is that on a technical level, there's no real possibility to fix the results. The extractor tool is only looking at a recorded video and has no idea what generated it. The overlay tool meanwhile is hooking into the pipeline at the application/Present level, similar to FRAPS. So it doesn't know anything about the hardware either.
On a practical level we've been over this with a fine-toothed comb. So have PCPerspective and others. What FCAT generates and what FCAT sees is consistent between all video cards. There's nothing strange going on.
That said, even NVIDIA is aware of the potential conflict of interest, which is why their goal is to eventually wash their hands of FCAT. The overlay can easily be implemented by someone else - ideally the FRAPS guys - and the extractor wouldn't be too hard to recreate (and the possibility is open of releasing the source for that). Someone had to do all the work to put this together though, and there really isn't anyone in a better position than a GPU vendor like AMD or NVIDIA. But now that they've laid down the template, independent parties can easily recreate all of this.
DanNeely - Wednesday, March 27, 2013 - link
IMO If their intent was to avoid maintaining the tool long term to avoid conflict of interest accusations they should've planned on making it open source from the start. Forcing 3rd parties to recreate something they always intended as a throwaway would be spiteful; and a pledge of opensourcing it would help mitigate accusations even if they held the code closed until it was completed.Ryan Smith - Wednesday, March 27, 2013 - link
Unfortunately the overlay cannot be open sourced. There are patent/licensing issues there with some of the code they're using. The only way for that to be compliant is to recreate it from scratch, hence the very specific interest in having the FRAPS guys do it.xdrol - Wednesday, March 27, 2013 - link
Seriously? That's lame.. What patented code does one need for drawing a single rectangle?Gigaplex - Wednesday, March 27, 2013 - link
Just what the heck are they doing in the overlay code that has patent/licensing issues? Recreating it from scratch won't avoid any patent issues, and it's simple enough to implement that you don't need to cross license copyrighted code from a 3rd party.zanon - Friday, March 29, 2013 - link
Ryan, Gigaplex is right and you're misunderstanding something here. Recreating code from scratch (*without* looking at the original, in other words clean-room reverse engineering) will get around any copyright issues. However, the whole reason software patents are so pernicious and wrong is that they protect the mere idea, not any implementation. Someone who came up with this completely independently and did it all themselves would still be violating any software patents on the idea of it.If it actually is patented then the only way around it is for someone in a non-software patent country to do it or for anonymous people to just blow the law off and do it anyway. Also, software patents don't preclude open source either, they're semi-orthogonal depending on the license, but OSS is primarily about copyright (though the GPL 3 tries to expand it a bit). Nvidia may have separate licensing issues though that'd be unique to them.
ARealAnand - Tuesday, April 9, 2013 - link
please see comment below for my attempt at a reply to this comment. Sorry for the comment spam.wingless - Wednesday, March 27, 2013 - link
Thank you for the informative reply. I look forward to your use of FCAT (it needs a new name: FrapCAT).Taracta - Wednesday, March 27, 2013 - link
I think there is a need to go over what frame drops and partial frames (runts) are as explained in article [LINK]http://www.anandtech.com/show/2794[/LINK]