AMD Comments on GPU Stuttering, Offers Driver Roadmap & Perspective on Benchmarking
by Ryan Smith on March 26, 2013 2:28 AM ESTFinal Words
Bringing things to a close, when AMD first came to us about this, we had mixed feelings about what they were proposing. Typically, manufacturers only take issue with benchmarking methodologies when they have some deficiency on their part. Surprisingly enough, AMD's stance was equal parts recognition of where they had issues and offering their input on the right way to do things.
Meanwhile, we’ve wanted to do an article like this for some time now; specifically a dissection of FRAPS so that we could better explain why we have not been using FRAPS for investigating stuttering. To our surprise, we found AMD repeating many of the positions we held, and more importantly they were offering up further facts and additional data that we didn’t have access to that would help support our own position. So despite whatever AMD’s intentions were, this worked out well for us.
Ultimately AMD’s message has been one of information, explanation, and admission of oversight. AMD has been clear with us from the start that the primary reason they even ended up in this situation is because they weren’t doing sufficient competitive analysis, and that they have revised their driver development process so that they now do this analysis to prevent future problems. The fact that NVIDIA seemed to have figured all of this out much earlier was a point of frustration for AMD. The company likely left non-negligable amounts of performance on the table over the years, which could've definitely helped in close races.
At the same time they’ve been hard at work on fixing existing stuttering problems, with many of those fixes being delivered while fixes for more DX10+ games are right around the corner.
At the same time however AMD’s message is not just one about stuttering, but also one about benchmark and analysis methods with FRAPS. FRAPS, despite its limitations, has clearly exposed problems with AMD’s drivers that resulted in stuttering that AMD needed to fix. Meanwhile measuring frame intervals with FRAPS has become an increasingly common technique in reviews, only to really become popular at the same time as when AMD has finally fixed many of these issues.
AMD’s concern – and one that NVIDIA has shared with them in the past – is that measuring the rendering pipeline at the beginning of the pipeline like FRAPS goes about it does not accurately represent what the end user is seeing, due to the various buffers in the Pipeline and how the Present mechanism works. While FRAPS was good enough to pick up on the major stuttering issues in AMD’s drivers, as these issues get resolved it’s far too coarse a tool to pick up on finer issues, and in fact what FRAPS is now seeing is decoupled from what the user is seeing due to the presence of the context queue and other buffers. All of these, for the record, are points we agreed with AMD on, even before our meeting.
The end result of all of this is that change is in the air. Just as how quickly as Scott Wasson and others changed the nature of GPU reviewing by using FRAPS to measure frametimes, things must change again for GPU reviewers. If FRAPS is no longer an adequate tool to measure stuttering and frame intervals – as both AMD and NVIDIA insist – then new methods and new tools must be created to measure those factors at the end of the rendering pipeline, where the results would match what the end user is seeing. Though on the subject of tools, AMD for their part is favoring double-blind trials as the ultimate method of detecting stuttering. They’re fundamentally right since the perceptibility of stuttering depends on the person, but admittedly this is also the least objective/qualitative way of evaluating stuttering.
In any case, just as how change is in the air for GPU reviews, AMD has had to engage in their own changes too. They have changed how they develop their drivers, how they do competitive analysis, and how they look at stuttering and frame intervals in games. And they’re not done. They’re already working on changing how they do frame pacing for multi-GPU setups, and come July we’re going to have the chance to see the results of AMD’s latest efforts there.
For us this is what we hope to be the start of our own changes. There are tools in development that meet our criteria for better measuring frame intervals, and hopefully in the not too distant future we’ll be able to discuss those tools to a much greater degree, and to use those tools to go about measuring frame intervals in the manner we’ve always wanted to. But that is a story for another day, so until then you’ll have to stay tuned to find out.
Facebook
Twitter
RSS
103 Comments
View All Comments
Galidou - Saturday, March 30, 2013 - link
Nope, never, I remember Nvidia back in the days of the 6800 GT that caused INFINITE stuttering(worse I've ever seen) with Nforce 3 or was it nforce 4 motherboard that I had. Only thing I could do to fix it was to underclock the video card, go back to older drivers. That made me lose 30-40% performance.They never ever fixed the problem or admitted it, EVER. I had to change video card after 6 months of trying everything. Nvidia forums were full of it not even an answer from them that they were fixing that issue. Some were able to fix this by disabling AGP fastwrites or other tricks but others had no choice doing what I did and lose the performance...
HisDivineOrder - Tuesday, March 26, 2013 - link
It's great that AMD admitted to a problem, but wow what a big problem to have totally missed. I guess they were so busy laying off engineers and R&D they didn't keep ahead of the game.haplo602 - Tuesday, March 26, 2013 - link
all nice and fine, but now please get your arse moving and do something for OpenGL performance AMD !!!kzinti1 - Tuesday, March 26, 2013 - link
If Windows is a major problem with stuttering, then why can't they develop a user-switchable "gaming mode" to make the OS prioritize the resources of the OS in favor of the games and their rendering processes?HisDivineOrder - Tuesday, March 26, 2013 - link
Microsoft is the company that might work something like that out. Unfortunately, Microsoft is also one of the companies that wants you to go buy a console. So I don't think they're going to facilitate what you suggest.I also suspect it's not as simple as what you suggest since it'd require game support, low level changes, etc. But ultimately, it doesn't matter how easy or hard it is because MS won't do it. They have no reason to.
If they cared about PC gaming in the slightest, I think they'd have ported Halo 3, ODST, Halo Reach, Halo 4, Gears of War 2, Gears of War 3, or Fable 2 to PC. Face it. MS gave up on PC gaming. Steam is what kept it going and Steam is what will carry it forward.
And the Steam Box may do exactly what you're suggesting.
mikato - Wednesday, March 27, 2013 - link
I'm pretty sure they care a bit because gaming is the only reason many people still use Windows.mgambrell - Wednesday, March 27, 2013 - link
methinks you place too much confidence in their acumen. As an exercise, find one thing microsoft has done lately which can be spun as plausibly in service of windows gamers.Dribble - Tuesday, March 26, 2013 - link
Fundamentally AMD failed because instead of making a driver to play games well, they make one that's there to give the highest fps at the expense of everything else. They were the first for example they customize the driver for every game - which makes the driver an order of magnitude more complex and introduced a lot more bugs to everything for a few % more performance.They did this because they care about the bottom line numbers shown in reviews more then actually playing the game well. Only now a reviewer has focused on stuttering are they focusing on it. It's not the only problem either - runt frames was also exposed by another tool which if anything is a cheat to exploit fraps - but AMD haven't got as far as discussing that yet.
This is a problem - AMD should be making drivers to play games well, not to look good in reviews. Journalists shouldn't be the ones having to do AMD's driver QA. I can't believe AMD didn't know about the stuttering - it's obvious even with a slow cam, they just didn't think it was important because it didn't effect their sales because journalists weren't reporting on it.
Spoelie - Tuesday, March 26, 2013 - link
Read the article again, your assumptions are wrong.Fixing the stuttering provided an increase in averaged framerates (in cases up to 13%), so it would've made them look a lot better even in traditional reviews not reporting on stuttering. And that's a huge delta for a small software change.
If anything, you could blame them for ineptitude, but there's no ill-will here.
Dribble - Tuesday, March 26, 2013 - link
The increase in fps was a surprise to them. The article suggests that if they had known it would increase fps they would have done it ages ago. Fact is there was stuttering, they knew about it but ignored it - the "well we assumed everyone else stuttered too" excuse isn't great. Clearly it was fixable, and a side effect was it even increased fps, but they were so fixated on fps charts in reviews that it was never deemed important enough to look at until the reviews started castigating them for it.If they had actually been trying to make the card as good as possible for gamers to play with they would have fixed it years ago as stuttering really matters to people trying to play the games.