AMD Comments on GPU Stuttering, Offers Driver Roadmap & Perspective on Benchmarking
by Ryan Smith on March 26, 2013 2:28 AM ESTFinal Words
Bringing things to a close, when AMD first came to us about this, we had mixed feelings about what they were proposing. Typically, manufacturers only take issue with benchmarking methodologies when they have some deficiency on their part. Surprisingly enough, AMD's stance was equal parts recognition of where they had issues and offering their input on the right way to do things.
Meanwhile, we’ve wanted to do an article like this for some time now; specifically a dissection of FRAPS so that we could better explain why we have not been using FRAPS for investigating stuttering. To our surprise, we found AMD repeating many of the positions we held, and more importantly they were offering up further facts and additional data that we didn’t have access to that would help support our own position. So despite whatever AMD’s intentions were, this worked out well for us.
Ultimately AMD’s message has been one of information, explanation, and admission of oversight. AMD has been clear with us from the start that the primary reason they even ended up in this situation is because they weren’t doing sufficient competitive analysis, and that they have revised their driver development process so that they now do this analysis to prevent future problems. The fact that NVIDIA seemed to have figured all of this out much earlier was a point of frustration for AMD. The company likely left non-negligable amounts of performance on the table over the years, which could've definitely helped in close races.
At the same time they’ve been hard at work on fixing existing stuttering problems, with many of those fixes being delivered while fixes for more DX10+ games are right around the corner.
At the same time however AMD’s message is not just one about stuttering, but also one about benchmark and analysis methods with FRAPS. FRAPS, despite its limitations, has clearly exposed problems with AMD’s drivers that resulted in stuttering that AMD needed to fix. Meanwhile measuring frame intervals with FRAPS has become an increasingly common technique in reviews, only to really become popular at the same time as when AMD has finally fixed many of these issues.
AMD’s concern – and one that NVIDIA has shared with them in the past – is that measuring the rendering pipeline at the beginning of the pipeline like FRAPS goes about it does not accurately represent what the end user is seeing, due to the various buffers in the Pipeline and how the Present mechanism works. While FRAPS was good enough to pick up on the major stuttering issues in AMD’s drivers, as these issues get resolved it’s far too coarse a tool to pick up on finer issues, and in fact what FRAPS is now seeing is decoupled from what the user is seeing due to the presence of the context queue and other buffers. All of these, for the record, are points we agreed with AMD on, even before our meeting.
The end result of all of this is that change is in the air. Just as how quickly as Scott Wasson and others changed the nature of GPU reviewing by using FRAPS to measure frametimes, things must change again for GPU reviewers. If FRAPS is no longer an adequate tool to measure stuttering and frame intervals – as both AMD and NVIDIA insist – then new methods and new tools must be created to measure those factors at the end of the rendering pipeline, where the results would match what the end user is seeing. Though on the subject of tools, AMD for their part is favoring double-blind trials as the ultimate method of detecting stuttering. They’re fundamentally right since the perceptibility of stuttering depends on the person, but admittedly this is also the least objective/qualitative way of evaluating stuttering.
In any case, just as how change is in the air for GPU reviews, AMD has had to engage in their own changes too. They have changed how they develop their drivers, how they do competitive analysis, and how they look at stuttering and frame intervals in games. And they’re not done. They’re already working on changing how they do frame pacing for multi-GPU setups, and come July we’re going to have the chance to see the results of AMD’s latest efforts there.
For us this is what we hope to be the start of our own changes. There are tools in development that meet our criteria for better measuring frame intervals, and hopefully in the not too distant future we’ll be able to discuss those tools to a much greater degree, and to use those tools to go about measuring frame intervals in the manner we’ve always wanted to. But that is a story for another day, so until then you’ll have to stay tuned to find out.
Facebook
Twitter
RSS
103 Comments
View All Comments
JPForums - Tuesday, March 26, 2013 - link
Their stance wasn't "Everyone else stutters so why should we bother with it." It was closer to "Everyone else stutters and we have no more control over it that they do ... Wait, you're saying they don't stutter as bad as we do ... and fixing our stutter would've actually helped our performance. Aw, nuts." I agree with Spoelie, it was ineptitude, not ill-will.extide - Wednesday, March 27, 2013 - link
You make absolutely no sense.Galidou - Saturday, March 30, 2013 - link
Lots of people seem to think AMD does so much mistakes that everytime it happens the world speaks about it more intensely because they actually admit it instead of trying to HIDE things, they even let the sites like anandtech make reviews on their problem. Right now, at home, I have two very comparable systems based on overclocked SNB. One with a 7950, and the other with a gtx 660 ti. Both play games extremely well but I have more trouble with my 660 ti. I get lots of ''Display adapter has stopped responding'' while surfing the internet, when waking up from long idle states and when playing League of Legends. I switched drivers, did clean install and I can't get rid of it totally. No my card is not overclocked and I can make it furmark all day long without a problem.I decided to play on my girlfriend's computer(which has the 7950) when I play league of legends because you just can't be interrupted in this kind of game. It even froze in LoL a couple times, at first I thought it was my SSD but after reading the dump files, found out it was the 660 ti. But hey, Nvidia is perfect(we just don't see them speaking of their problems that's it)... 4 months of it now, thanks alot... I wrote on nvidia forums did my research did everything they told me. Good thing it's only with LoL, internet and long idles, everything else runs flawlessly.
HisDivineOrder - Tuesday, March 26, 2013 - link
Actually, I think you should reread the article. It's true that fixing these stuttering issues has given them some frame rate improvements, but the article seemed relatively clear on the point that they were focused on frame rates and not frame latency or frame interval or whatever they're choosing to call it today.They had all their focus on one aspect of driver development and that actually cost them in that area because they weren't considering a more well-rounded approach.
I'll grant you AMD was pretty inept though. This has been problem they've had for years and it's taken them this long to suss it out...
piroroadkill - Tuesday, March 26, 2013 - link
Hey, what about PC Perspective: http://www.pcper.com/reviews/Graphics-Cards/Frame-...Looks to me as being the best way to measure it, as it uses a DVI capture card to actually capture what the user sees, forgoing any overhead software may have..
Rick83 - Tuesday, March 26, 2013 - link
Yes, that is the correct way to measure frame time.KikassAssassin - Tuesday, March 26, 2013 - link
Yeah, their method looks like the best of both worlds. It gets around the limitations of FRAPS without the complexity and difficulty of using GPUView.Anand Lal Shimpi - Tuesday, March 26, 2013 - link
The ideal method would be something that gives us timestamps at both ends of the pipeline, but that's a tall order. The PCPer method is very interesting indeed... ;)Mopar63 - Tuesday, March 26, 2013 - link
First very informative article. The issue at hand is that this so called concern is based on an individuals perception. Remember we are not talking about a stuttering that was so bad as to be noticeable to all gamers. Scott basically had to make a video specifically designed to point out the issue for others to see it originally.Because there is no real way to quantify the personal experience we have an issue in the fact that we now have a measurement craze that is being treated as fact when it is based in the end on subjection for the final result.
Having access to various levels of AMD and NVidia based machines I can tell you that my gaming experience across them has been pretty uniform in most cases. The cases when I had a bad experience, probably a wash as they are on both platforms.
I think the biggest issue is we sometimes get to caught up in the technology. We let benchmarks and measurement programs dictate to us what we will get the most enjoyment from with our gaming experience. A game is not the frame rates but the play that matters. While frame rates might play a roll it is not the measurement of them that makes that part of the fun.
At the end of the day the single best test of a video card is not a benchmark suite or tool to measure frame rendering time. The best tool is to play the games you want and see if you get the game experience you desire. Turn off the benchmark and turn on the game, that is the ONLY true test of what is best.
HisDivineOrder - Tuesday, March 26, 2013 - link
Speak for yourself. I've noticed this problem between nVidia vs AMD for years. For many years, gamers have said that nVidia cards are "smoother." People didn't listen because they didn't want to hear the truth or because they were likely stuck with one high end from AMD and a low end or medium end from nVidia.But comparing equivalent cards, I can tell you my experience has always led inescapably to the "feeling" that the nVidia card is smoother at the same or even slightly lower frame rate.
This just proves what I "felt" was the case was in fact really the case. If you didn't see it, then that's a fail on the part of your visual acuity or perhaps you had a bias you wanted to see, so you saw less than everything present.
But the stutter was always there. Now even AMD admits it.