AMD Frame Pacing Explored: Catalyst 13.8 Brings Consistency to Crossfire
by Ryan Smith on August 1, 2013 2:00 PM ESTThe Test
For the purposes of our testing we’ll be looking at the 6 games we’ve adopted for use with FCAT due to their proven reliability. These are Total War: Shogun 2, HItman: Absolution, Sleeping Dogs, Battlefield 3, Bioshock Infinite, and Crysis 3. All of our results unless otherwise noted are using Catalyst 13.8b1 for the AMD cards, and NVIDIA’s 326.19 beta drivers for the GeForce cards.
Our metric of choice for measuring frame times and frame pacing is a metric we’re calling Delta Percentages. With delta percentages we’re collecting the deltas (differences) between frame times, averaging that out, and then dividing delta average by the average frame time of the entire run. The end result of this process is that we can measure whether sequential frames are rendering in roughly the same amount of time, while controlling for performance differences by looking at the data relative to the average frame time (rather than as absolute time). This gives us the average frame-to-frame time difference as a percentage.
In general, a properly behaving single-GPU card should have a delta average of under 3%, with the specific value depending in part on how variable the workload is throughout any given game benchmark. 3% may sound small, but since we’re talking about an average it means it’s weighed against the entire run, as the higher the percentage the more unevenly frames are arriving. For a multi-GPU setup we’d ideally like to see the delta percentages be equal to our single-GPU setups, but this is for the most part unreasonable. There is no hard number for what is or isn’t right here, but based on play testing we’d say 15%-20% is a reasonable threshold for acceptable variance, with anything under 10% being very good for a multi-GPU setup.
Finally, in our testing we did encounter an issue with Catalyst 13.8 that required we make some slight adjustments to FCAT to compensate for this bug, so we need to make note of this. For reasons we can’t sufficiently explain at this time but has been confirmed by AMD, in some cases in Crossfire mode AMD’s latest drivers are periodically drawing small slices of old frame buffers at the top of the screen. The gameplay impact is minimal-to-nonexistent, but this problem throws off FCAT badly.
To quickly demonstrate the problem, below we have two consecutive frames from one of our Battlefield 3 runs. The correct FCAT color order here is dark blue, green, light blue, and olive. The frames corresponding to dark blue and green occur on frame one, and light blue and olive on frame two. Yet looking at frame two, we see a small 6 pixel high stripe of dark blue at the very top of the image. At this point the dark blue frame should have already been discarded, as the cards have moved on to the green and later light blue frames. Instead we’re getting a very small slice of a frame that is essentially 2 frames old.
The gameplay impact from this is trivial to none; the issue never exceeds a 6 pixel slice, only occurs at the top of the frame (which is generally skybox territory), and is periodic to the point where it occurs at most a few times per minute. And based on our experience this primarily occurs when a buffer swap should be occurring during or right after the start of a new refresh cycle, which is why it’s so periodic.
However the larger issue is that FCAT detects this as a frame drop, believing that over a dozen frames have been dropped. This isn’t actually possible of course – the context queue isn’t large enough to hold that many frames – and analysis shows that it’s actually part of the old frame as we’ve explained earlier. As such we’ve had to modify FCAT to ignore this issue so that it doesn’t find these slices and count them as dropped frames. The issue is real enough (this isn’t a capture error) and AMD will be fixing it, but it’s not evidence of a dropped frame as the stock implementation of FCAT would assume.
Ultimately our best guess here is that AMD is somehow mistiming their buffer swaps, as the 2 frame old aspect of this correlates nicely to the fact that the dark blue and light blue frames would both be generated by the same GPU in a two-GPU setup.
| CPU: | Intel Core i7-3960X @ 4.3GHz |
| Motherboard: | EVGA X79 SLI |
| Power Supply: | Antec True Power Quattro 1200 |
| Hard Disk: | Samsung 470 (256GB) |
| Memory: | G.Skill Ripjaws DDR3-1867 4 x 4GB (8-10-9-26) |
| Case: | Thermaltake Spedo Advance |
| Monitor: | Samsung 305T |
| Video Cards: |
AMD Radeon HD 6990 AMD Radeon HD 7970GE AMD Radeon HD 7990 NVIDIA GeForce GTX 590 NVIDIA GeForce GTX 680 NVIDIA GeForce GTX 690 |
| Video Drivers: |
NVIDIA ForceWare 326.19 AMD Catalyst 13.5 Beta 2 AMD Catalyst 13.6 Beta 2 AMD Catalyst 13.8 Beta 1 |
| OS: | Windows 8 Pro |
Facebook
Twitter
RSS
102 Comments
View All Comments
chizow - Friday, August 2, 2013 - link
No it makes perfect sense. Use a camera if you have to capture how the frames are updated with Vsync Off normally. You will see the "tear" or refresh line differs from frame to frame, with either the top or bottom being refreshed relative to the previous frame. The key distinctions vs runtframes are that the next frame updates where the last one left off and there is significant differentiation between the frames, giving the sense of motion, with tearing.Unlike with runtframes, every other frame is a runt or partial followed by a more fully formed frame, so you basically get the effect of 2 similar frames followed by an updated frame due to the runt. This results in the irregular cadence known commonly as microstutter.
Please see the videos here at PCPer to better understand, all videos were done with Vsync Off, and as you can clearly see, Vsync Off with runts and without are clearly not the same thing.
http://www.pcper.com/reviews/Graphics-Cards/Frame-...
krutou - Friday, August 2, 2013 - link
Runts are due to screen tearing...What isn't shown in most reviews because it is difficult to evaluate objective, but the most apparent to the average user is animation smoothness. PC Perspective had a whole slew of videos comparing CF to SLI over most 7000 and 600 series cards and the difference in smoothness between the two solutions was obvious, even though there was no screen tearing.
vegemeister - Monday, August 5, 2013 - link
Agreed. It seems to me that this whole kerfuffle is two camps debating what kind of garbage output they'd rather see. The only reasonable way to use frame time logs in a review is discussing the percent of frames that take "too long" at a variety of refresh rates (144Hz, 120Hz, 85Hz, 60Hz, 48Hz, 30Hz, and 20Hz would be a good sampling). Perhaps you would also scale to the refresh rate, to examine how frequently a user would likely see dropped frames.Omoronovo - Thursday, August 1, 2013 - link
Fantastic article Ryan.I did however have a query regarding the comments made regarding the odd issue of FCAT recording high dropped frames and the changes you implemented to prevent this error skewing results. Did you re-run any of the tests on the nVidia results to see if the changes you made to fcat had any noticeable change in the results generated from nVidia cards? Particularly regarding runt frames; as you mention yourself, AMD has reduced runt frames to zero in many cases which is an impressive achievement if accurate, and I would hope by redoing the tests on nVidia data that it proves to remain true (rather than skewing nVidia results as well, which would indicate that the FCAT changes to fix one problem would introduce variancy in the comparion outside the changes made in the driver).
Like usual, fantastic article from Anandtech, and you in particular Ryan. Hope to read more articles from you soon.
Ryan Smith - Thursday, August 1, 2013 - link
Indeed I did, and it made no difference on NVIDIA cards. The FCAT miscounts only occur on AMD cards with the frame buffer issue as outlined in the article.airmantharp - Thursday, August 1, 2013 - link
Thanks for the great work Ryan-I'm still waiting for TR's results to put this all into perspective (always need multiple datapoints), but this bodes very well for those looking at 4k setups in the future. Before this driver was proven, AMD wasn't even in the running, regardless of price.
TheJian - Friday, August 2, 2013 - link
The statements here are hilarious (expect nothing less from Ryan these days). He frames it as a 7990 problem, but in reality it was every pair of AMD cards you could throw at a PC. ALL MULTI-GPU situations on AMD are affected and have been for at least a year and a half (always?). 2x7950, 2x7970 etc etc...And 7990 wasn't a rough launch. It was awful. We are still waiting for Ryan's FCAT articles. I expect we'll start to see more now, just as I said before many times, AFTER AMD FIXES their problems. Here we are...LOL. No bias at anandtech though...ROFL. Still missing FCAT part 2 article from march. IT was promised in 2 weeks, never happened. Tons of FCAT results left out of most articles due to drivers changing, game problems, patches etc...blah blah...PCper does it for every article and had NONE of these issues. Hmm...Funny that...
This leaves us with 2 options:
1. You're either too dumb to use the tool and should just lose your job (say it ain't so).
2. You're hiding the truth from your users and should have your traffic reduced further as Alexa shows it's off half or so since the 660TI article in sept last year when Ryan said silly things like 'a lot of people are buying Catleap 1440p monitors from ebay from some dude in korea'...ROFL. Umm, no they are not. Still not many today since even in your own 1440p article (again with ridiculous conclusions like buy an AMD A8-5600 for any single gpu card - no Buy Intel!) shows the steam hardware survey which shows less than 1.25% of us use above 1920x1200 and most of those use TWO video cards...LOL.
No driver launch should be important for a company...ROFL. It's extremely important we don't HAVE TO TALK about your crappy drivers because they didn't take a year or two to make them work. Right? The important thing is that the product is launched WORKING out of the box as advertised.
"What’s not being addressed in the first driver are the Direct3D 9 and OpenGL rendering paths, along with Eyefinity in any scenario."
So it's still not working for some people in multiple scenarios (all xp users right? They all use DX9). You act as though high FPS=stutter free. NOT SO. Just because you're hitting 60fps in rage or 90fps etc, that does not mean the customer isn't affected. The problem is SEEING the stutter, not if the game is fast enough when it ISN'T stuttering.
"and is periodic to the point where it occurs at most a few times per minute."
So a few times per minute you have to deal with this...Get back to work AMD. NV doesn't do this.
To summarize, AMD is still stuttering a lot at 20% (assuming you picked 15-20 so they could squeak inside for most games...LOL). But reality is they only made it UNDER 20 for 2 games of the 6 right? BF3 and Crysis3, otherwise they are STILL over even your worst end at 20% (nowhere near 15 on the low-end of that range right?). Also note the card runs 5-10% slower now.
http://www.pcper.com/reviews/Graphics-Cards/Frame-...
Better off reading the article at PCper above. AMD said Ryan Shrout was wrong 1/2 dozen times...LOL. That's changed now :) Also as they show Skyrim is affected with or without mods even with the new driver. But the point on this is you don't have to mod it to see stutter.
Pcper also shows it's been affecting 7970CF (and all other CF users) for ages up to now. Finally they get some relief. However TRIPLE card owners still suffer the same problems:
"there is a lot of work AMD can do to improve the experience for users tripling their investment."
"I will be curious if its even possible for AMD to fix the frame pacing issues with CrossFire and Eyefinity this generation.
Even worse though is that with all the excitement over 4K displays and in particular the ASUS PQ321Q 4K 60 Hz monitor I recently reviewed, the Eyefinity limitation will pop up again in this niche case too. "
So due to bandwidth he doesn't even think they'll be able to fix this..OUCH AMD. An even bigger OUCH to AMD buyers. He notes in the same paragraph AMD didn't want to discuss it (shocking). WOW.
"Not only does this driver validate everything we have worked on for the last two years but the fact that AMD has decided to enable the frame pacing fix by default emphasizes that fact even more."
Funny anandtech didn't start saying anything until March and remained mum on FCAT stuff basically until the fix just as I said they would do.
http://hardocp.com/article/2013/08/01/amd_catalyst...
Hardocp isn't as kind to AMD either IMHO.
"Far Cry 3 was the most inconsistent and choppy. Even with Frame Pacing turned on, it was stuttery feeling, but this lagginess was reduced slightly with Frame Pacing."
Still having issues.
"This is definitely a foot forward in the right direction from AMD though, but work still needs to be done."
Let me know when AMD gets done with all these PHASES so users actually get a REAL DONE driver that should have been in the box or downloadable all along. Working products people, is the reason NV owns 65% of the discrete gpu market today despite all the free games AMD gives away to get you to buy & wait for their drivers to work (which also happens to be why they make ZERO on their gpu division). No mention of AMD's quarterly report again here...Too depressing I guess now. I expect Ryan to cover their Q report again if consoles make them some cash...LOL. But he'll continue to refuse to cover NV's quarters because all that shows is they are giving AMD a whipping in each one.
Some hardocp conclusions:
"It also has to be said that NVIDIA has been doing this for years. Though we don't know the exact name for what it does in its drivers, it is known that NVIDIA is doing something to smooth out frame times and make SLI feel good while gaming. There is a reason that we have been saying for years how NVIDIA SLI felt smoother. AMD is just now playing catch up, and it is quite late in the game in our opinion to be playing this kind of catch up. In our opinion this type of technology should have employed years ago. If AMD is serious about improving the gameplay experience, this is one area that needs a lot of attention."
"In our personal gaming observations, we still observe that NVIDIA SLI is smoother and more consistent than AMD CrossFire with Frame Pacing."
"AMD has improved its own consistency. We still feel that it lags behind whatever NVIDIA is doing."
Years later AMD finally getting in the game. Jeez. I'll also remind you ~38% of the public is STILL running windows XP so all of these are forced to DirectX 9 paths. So all of them should by Nvidia cards correct? That sort of blows your whole DirectX9 exclusive (as if nobody would run there) titles out of the water. They don't have to be exclusive if the person running the game can't run dx10 or dx11 (at that point it's DIRECTX 9 exclusive for an XP user right?). Get it? Of course you do Ryan, but it sounded better for AMD saying 'um, well, this never happens so who cares about phase two, everyone is solved today'...I digress...
boozed - Friday, August 2, 2013 - link
What happened in your life to cause you to be so bitter and mean?TheJian - Tuesday, August 6, 2013 - link
No argument with the data though ;) It's comic the first things people do when there is no argument against the data is a personal attack. So stating the facts means you're bitter and mean to you?https://en.wikipedia.org/wiki/Ad_hominem
Please review the chart from Graham, then come back with something at the top of his pyramid instead of the bottom :)
DeviousOrange - Friday, August 2, 2013 - link
TL;DR maybe someone will give you the F&*#s you wanted in posting this crap but im allegic to fanboy troll.