A Look At Triple-GPU Performance And Multi-GPU Scaling, Part 1
by Ryan Smith on April 3, 2011 7:00 AM ESTFitting Three Video Cards in an ATX Case
I thought we’d flip our normal GPU review style on its head by starting with Power, Temperature, and Noise first. NVIDIA and AMD have both long recommended against placing high-end video cards directly next to each other, in favor of additional spacing between video cards. Indeed this is a requirement for their latest dual-GPU cards, as both the GTX 590 and 6990 draw relatively massive amounts of air using a fan mounted at the center of the card and exhaust roughly half their air inside of the case. Their reference style single-GPU cards on the other hand are fully exhausting with fans mounted towards the rear of the card. Thus multi-GPU configurations with the cards next to each other is supposed to be possible, though not ideal.
There’s a reason I want to bring this up first, and a picture is worth a thousand words.
While AMD and NVIDIA’s designs share a lot in common – a rear-mounted blower fan pushes air over a vapor chamber cooler – the shrouds and other external equipment are quite different. It’s not until we see a picture that we can appreciate just how different they are.
With the Radeon HD 6000 series, AMD’s reference designs took on a very boxy design. The cards fully live up to the idea of a “black box”; they’re enclosed on all sides with a boxy cooler and a black metal backplate. As a GPU reviewer I happen to like this design as the GPUs are easy to stack/store, and the backplate covers what would normally be the only exposed electronics on the card. The issue with this boxy design is that AMD is taking full advantage of the PCIe specification, leading to the 6900 series being the full width allowed.
NVIDIA on the other hand has always had some kind of curve in their design, normally resulting in a slightly recessed shroud around the blower intake. For the GTX 580 and GTX 570 they took a further step in recessing the shroud around this area, leading to the distinct wedge shape. At the same time NVIDIA does not use a backplate, saving precious millimeters of space. The end result of this is that even when packed like sardines, the GTX 580 and GTX 570 blowers have some space reserved for air intake.
The Radeon HD 6970 does not, and this is our problem. The picture of the 6970 in triple-CF really paints the picture, as the middle card is directly pressed up against the top card. Because these cards are so large and heavy the rear ends tend to shift and dip some when installed against a vertical motherboard – in fact this is why we can normally get away with a dense dual-CF setup since the bottom card dips a bit more – but in a triple-CF configuration the end result is that one of the cards will end up getting up-close and personal with another one.
Without outside intervention this isn’t usable. We hit 99C on the middle card in Crysis when we initially installed the three cards, and Crysis isn’t the hardest thing we run. For the purposes of our test we ultimately resorted to wedging some space between the cards with wads of paper, but this isn’t a viable long-term solution.
Unfortunately long-term alternatives are few if you want to give a triple-GPU setup more space. Our testbed uses an Asus Rampage II Extreme, which features three PCIe slots mixed among a total of 6 slots; the way it’s laid out makes it impossible to have our triple-GPU configuration setup in any other manner. Even something like the ASRock P67 Extreme4 can’t escape the fact that the ATX spec only has room for 7 slots and that when manufacturers actually use the 7th and topmost slot that it’s a short PCIe x1 slot. In short you won’t find an ATX motherboard that can fit three video cards and at the same time gives each one a slot’s worth of breathing room. For that you have to use a larger than ATX form factor.
So what’s the point of all of this rambling? With AMD’s current shroud design it’s just not practical to do triple-CF on air on an ATX motherboard. If you want to play with three AMD boards you need to think outside of the box: either use water cooling or use a larger motherboard.
Facebook
Twitter
RSS
97 Comments
View All Comments
Ryan Smith - Monday, April 4, 2011 - link
There are 2 reasons for that:1) We can't immediately get another 6990. I know it seems odd that we'd have trouble getting anything, but vendors are generally uninterested in sampling cards that are reference, which is why we're so grateful to Zotac and PowerColor for the reference 580/6970.
2) We actually can't run a second 6990 with our existing testbed. The Rampage II Extreme only has x16 slots at positions 2 and 4; position 6 is x8. The spacing needs for a 6990CF setup require 2 empty slots, meaning we'd have to install it in position 6. Worse yet is that position 6 is abutted by our Antec 1200W PSU - this isn't a problem with single-GPU cards as the blowers are well clear of the PSU, but a center-mounted fan like the 6990 would get choked just as if there was another card immediately next to it.
We will be rebuilding our testbed for SNB and using a mobo with better spacing, but that's not going to happen right away. The point being that we're not ignoring the 590/6990 multiple card configurations, it's just not something we're in a position to test right now.
piroroadkill - Monday, April 4, 2011 - link
As long as it's in the works, that's alright. Seems like you have your reasons for it being the way it is.Rukur - Monday, April 4, 2011 - link
This whole technology is stupid with monitors.Why don't you stitch together 3 projectors for a seamless canvas to play a game ?
SlyNine - Monday, April 4, 2011 - link
"This whole technology is stupid with monitors." Do you suppose neural interfaces will be her soon. kick ass.Rukur - Monday, April 4, 2011 - link
Can you read more than one sentence ?monkeyshambler - Monday, April 4, 2011 - link
Interesting stuff, but for a 3 card SLI / crossfire what I'd really want to see is what the framerates are when every setting on the card is maxed.e.g. 24x AA 16x AF, high quality settings selected in the driver control panels etc.
supplement this with whats the performance on triple SLI with 3 1920*1080 monitors @ 4x AA
As lets face it if your going to spend this sort of money (and likely a watercooling rig too as theirs no way three cards are tolerable otherwise) you want to have a genuine show of why you should invest.
The current resolutions just will never stretch the cards or enable them to differentiate significantly from a standard SLI setup.
Hope we can see some of the above in a future article....
Rukur - Monday, April 4, 2011 - link
I tend to agree. How is maxing everything any worse then half inch monitor bezels all over your play area.The whole idea of eye infinity is stupid unless we all look through widows with 1 inch gaps while racing extreme cars.
How about some projectors stitched together for real people to actually try.
erple2 - Tuesday, April 5, 2011 - link
Wasn't there an analysis a while back comparing 1x, 2x, 4x, 8x and 16x AA? I thought that the conclusion to that was that there's no discernible difference between 8x and 16x AA, and the differences between 4x and 8x were only visible in careful examination of static images. Under normal play, you couldn't actually tell any difference between them.Maybe I'm just remembering wrong.
Also, I think that Ryan mentioned why they haven't yet done the triple monitor tests yet (lack of hardware).
DanNeely - Tuesday, April 5, 2011 - link
That's generally correct. Toms Hardware has run PCI restriction tests roughly once per GPU generation. The only game that ever really suffered at x4 bandwidth was MS flight simulator.PCIe bandwidth can impact some compute tasks. Einstien@home runs about 30% faster on a 460 in an 16x slot vs an 8x.
fepple - Monday, April 4, 2011 - link
With my two 5870s I have a wierd problem in crossfire. I have two screens a 24''' LCD and a 37'' LED TV. When in crossfire if I play video on the second screen it gets some odd artifacts of black(ish) horizontal lines across the bottom of the screen. Only solution i've found is to not have the cards in crossfire and plug the TV/Screen into different cards for watching stuff.Annoying, any thoughts?