AMD Frame Pacing Explored: Catalyst 13.8 Brings Consistency to Crossfire
by Ryan Smith on August 1, 2013 2:00 PM ESTThe Test
For the purposes of our testing we’ll be looking at the 6 games we’ve adopted for use with FCAT due to their proven reliability. These are Total War: Shogun 2, HItman: Absolution, Sleeping Dogs, Battlefield 3, Bioshock Infinite, and Crysis 3. All of our results unless otherwise noted are using Catalyst 13.8b1 for the AMD cards, and NVIDIA’s 326.19 beta drivers for the GeForce cards.
Our metric of choice for measuring frame times and frame pacing is a metric we’re calling Delta Percentages. With delta percentages we’re collecting the deltas (differences) between frame times, averaging that out, and then dividing delta average by the average frame time of the entire run. The end result of this process is that we can measure whether sequential frames are rendering in roughly the same amount of time, while controlling for performance differences by looking at the data relative to the average frame time (rather than as absolute time). This gives us the average frame-to-frame time difference as a percentage.
In general, a properly behaving single-GPU card should have a delta average of under 3%, with the specific value depending in part on how variable the workload is throughout any given game benchmark. 3% may sound small, but since we’re talking about an average it means it’s weighed against the entire run, as the higher the percentage the more unevenly frames are arriving. For a multi-GPU setup we’d ideally like to see the delta percentages be equal to our single-GPU setups, but this is for the most part unreasonable. There is no hard number for what is or isn’t right here, but based on play testing we’d say 15%-20% is a reasonable threshold for acceptable variance, with anything under 10% being very good for a multi-GPU setup.
Finally, in our testing we did encounter an issue with Catalyst 13.8 that required we make some slight adjustments to FCAT to compensate for this bug, so we need to make note of this. For reasons we can’t sufficiently explain at this time but has been confirmed by AMD, in some cases in Crossfire mode AMD’s latest drivers are periodically drawing small slices of old frame buffers at the top of the screen. The gameplay impact is minimal-to-nonexistent, but this problem throws off FCAT badly.
To quickly demonstrate the problem, below we have two consecutive frames from one of our Battlefield 3 runs. The correct FCAT color order here is dark blue, green, light blue, and olive. The frames corresponding to dark blue and green occur on frame one, and light blue and olive on frame two. Yet looking at frame two, we see a small 6 pixel high stripe of dark blue at the very top of the image. At this point the dark blue frame should have already been discarded, as the cards have moved on to the green and later light blue frames. Instead we’re getting a very small slice of a frame that is essentially 2 frames old.
The gameplay impact from this is trivial to none; the issue never exceeds a 6 pixel slice, only occurs at the top of the frame (which is generally skybox territory), and is periodic to the point where it occurs at most a few times per minute. And based on our experience this primarily occurs when a buffer swap should be occurring during or right after the start of a new refresh cycle, which is why it’s so periodic.
However the larger issue is that FCAT detects this as a frame drop, believing that over a dozen frames have been dropped. This isn’t actually possible of course – the context queue isn’t large enough to hold that many frames – and analysis shows that it’s actually part of the old frame as we’ve explained earlier. As such we’ve had to modify FCAT to ignore this issue so that it doesn’t find these slices and count them as dropped frames. The issue is real enough (this isn’t a capture error) and AMD will be fixing it, but it’s not evidence of a dropped frame as the stock implementation of FCAT would assume.
Ultimately our best guess here is that AMD is somehow mistiming their buffer swaps, as the 2 frame old aspect of this correlates nicely to the fact that the dark blue and light blue frames would both be generated by the same GPU in a two-GPU setup.
| CPU: | Intel Core i7-3960X @ 4.3GHz |
| Motherboard: | EVGA X79 SLI |
| Power Supply: | Antec True Power Quattro 1200 |
| Hard Disk: | Samsung 470 (256GB) |
| Memory: | G.Skill Ripjaws DDR3-1867 4 x 4GB (8-10-9-26) |
| Case: | Thermaltake Spedo Advance |
| Monitor: | Samsung 305T |
| Video Cards: |
AMD Radeon HD 6990 AMD Radeon HD 7970GE AMD Radeon HD 7990 NVIDIA GeForce GTX 590 NVIDIA GeForce GTX 680 NVIDIA GeForce GTX 690 |
| Video Drivers: |
NVIDIA ForceWare 326.19 AMD Catalyst 13.5 Beta 2 AMD Catalyst 13.6 Beta 2 AMD Catalyst 13.8 Beta 1 |
| OS: | Windows 8 Pro |
Facebook
Twitter
RSS
102 Comments
View All Comments
chizow - Saturday, August 3, 2013 - link
Written on the walls at AMD Driver HQ I'm sure, quickly referenced when questioned about microstutter being worst on CF.krutou - Friday, August 2, 2013 - link
BUT, BUT, BUT, RADEON PRO?!?LordOfTheBoired - Saturday, August 3, 2013 - link
Took them by the hand? Looks more to me like "waited until people cared, then released a benchmark to prove they didn't have the problem, while offering no constructive commentary."And the hell of it is... AMD's stance makes sense. This IS a market that hates VSync because ZOMG LAG. "The market" has made it ABUNDANTLY clear that they have no interest whatsoever in technologies that improve the visual experience at the (real or imagined) cost of responsiveness.
But apparently that's only true when there's the visual equivalent of a record skip on your screen and not when it's a subtle frame rate fluctuation. The former is a good thing because it means you "aren't lagged", the latter is a horrible thing because it means you "aren't lagged".
chizow - Wednesday, August 7, 2013 - link
@LordOfTheBoired - this is the type of indignant attitude that got AMD and their fans in this predicament to begin with.mikato - Wednesday, August 7, 2013 - link
Well one result of Nvidia releasing the software was to allow tech journalists to shine a bright spotlight on a problem of their competitor's products. Your "holding AMD by the hand" idea is pretty amusing.novastar78 - Wednesday, August 7, 2013 - link
What's sad is that AMD has ripped ATi to shreds to the point where they are spread pretty thin. Just getting a working product out the door is a task. You are being waaay too hard on them.They were using traditional methods to test and frankly it's not so unimaginable that they were caught by this. Granted, maybe it should have been caught sooner, but to demonize them or put them down for it seems a bit harsh.
They know now that it's a big problem are definitely committed to fixing it. You can clearly see that they are trying to stabilize the company and there is lots of turmoil. The moves they are making and the people being brought on board are a good sign. There is definitely a process change that needs to happen but when the tree is being shaken so many times too many apples can fall.
Give it some time, I think we will see great things form them over the next few years.
Wreckage - Thursday, August 1, 2013 - link
It did not help that certain people were claiming that AMD does not have a stutter problem.DanNeely - Thursday, August 1, 2013 - link
The problem is that AMD didn't think they had a stutter problem; or more precisely, until 3rd parties shoved the reality in their face they assumed (without testing) that they and nVidia had equal amounts of the problem.I suspect at one point in the past they were right; and that the genesis of nVidia's "years in the making" tool to measure the problem dates back to when they discovered it was a problem internally and began working on fixes so that they could announce the same tool at the same time that their drivers had a negligible impact from it. It'd be interesting to see what would happen if someone tested SLI card/driver configurations from a few years ago to see how well nVidia did at the time.
BrightCandle - Friday, August 2, 2013 - link
NVidia has had this fixed for a lot longer than that. The 680's were noticeably smoother on their release day compared to the 7970 crossfire. NVidia has claimed they have been fixing this since the 8800 and there is no reason not to believe them as HardOCP and other review sites have been noting the difference for years.HisDivineOrder - Friday, August 2, 2013 - link
Exactly. This has been a problem for as long as there has been Crossfire. I think a lot of people are so shocked by this fact (because some of them owned Crossfire and just dealt with it and didn't realize they were getting shafted) they can't accept it.Yes, you got reamed. Yes, for years, you were using Crossfire and suffering from microstutter when you needn't have to. Yes, AMD took you for a ride. It's so infuriating it drives people not to want to believe it because to believe it would be so horrible as to suddenly be intolerable.
Do what most people do. Just stick with nVidia since you know they do proper testing. Boycott AMD for a five year period and come back once you're sure AMD's got their act together.