PCI Express 3.0: More Bandwidth For Compute

It may seem like it’s still fairly new, but PCI Express 2 is actually a relatively old addition to motherboards and video cards. AMD first added support for it with the Radeon HD 3870 back in 2008 so it’s been nearly 4 years since video cards made the jump. At the same time PCI Express 3.0 has been in the works for some time now and although it hasn’t been 4 years it feels like it has been much longer. PCIe 3.0 motherboards only finally became available last month with the launch of the Sandy Bridge-E platform and now the first PCIe 3.0 video cards are becoming available with Tahiti.

But at first glance it may not seem like PCIe 3.0 is all that important. Additional PCIe bandwidth has proven to be generally unnecessary when it comes to gaming, as single-GPU cards typically only benefit by a couple percent (if at all) when moving from PCIe 2.1 x8 to x16. There will of course come a time where games need more PCIe bandwidth, but right now PCIe 2.1 x16 (8GB/sec) handles the task with room to spare.

So why is PCIe 3.0 important then? It’s not the games, it’s the computing. GPUs have a great deal of internal memory bandwidth (264GB/sec; more with cache) but shuffling data between the GPU and the CPU is a high latency, heavily bottlenecked process that tops out at 8GB/sec under PCIe 2.1. And since GPUs are still specialized devices that excel at parallel code execution, a lot of workloads exist that will need to constantly move data between the GPU and the CPU to maximize parallel and serial code execution. As it stands today GPUs are really only best suited for workloads that involve sending work to the GPU and keeping it there; heterogeneous computing is a luxury there isn’t bandwidth for.

The long term solution of course is to bring the CPU and the GPU together, which is what Fusion does. CPU/GPU bandwidth just in Llano is over 20GB/sec, and latency is greatly reduced due to the CPU and GPU being on the same die. But this doesn’t preclude the fact that AMD also wants to bring some of these same benefits to discrete GPUs, which is where PCI e 3.0 comes in.

With PCIe 3.0 transport bandwidth is again being doubled, from 500MB/sec per lane bidirectional to 1GB/sec per lane bidirectional, which for an x16 device means doubling the available bandwidth from 8GB/sec to 16GB/sec. This is accomplished by increasing the frequency of the underlying bus itself from 5 GT/sec to 8 GT/sec, while decreasing overhead from 20% (8b/10b encoding) to 1% through the use of a highly efficient 128b/130b encoding scheme. Meanwhile latency doesn’t change – it’s largely a product of physics and physical distances – but merely doubling the bandwidth can greatly improve performance for bandwidth-hungry compute applications.

As with any other specialized change like this the benefit is going to heavily depend on the application being used, however AMD is confident that there are applications that will completely saturate PCIe 3.0 (and thensome), and it’s easy to imagine why.

Even among our limited selection compute benchmarks we found something that directly benefitted from PCIe 3.0. AESEncryptDecrypt, a sample application from AMD’s APP SDK, demonstrates AES encryption performance by running it on square image files.  Throwing it a large 8K x 8K image not only creates a lot of work for the GPU, but a lot of PCIe traffic too. In our case simply enabling PCIe 3.0 improved performance by 9%, from 324ms down to 297ms.

Ultimately having more bandwidth is not only going to improve compute performance for AMD, but will give the company a critical edge over NVIDIA for the time being. Kepler will no doubt ship with PCIe 3.0, but that’s months down the line. In the meantime users and organizations with high bandwidth compute workloads have Tahiti.

Video & Movies: The Video Codec Engine, UVD3, & Steady Video 2.0 Managing Idle Power: Introducing ZeroCore Power
Comments Locked

292 Comments

View All Comments

  • RussianSensation - Saturday, January 14, 2012 - link

    BF3 is not a 2012 game.......

    Also, most of us have been gaming on our older cards. Who in the world who has a previous high-end card is going to drop $600 for BF3 alone? No thanks.
  • SSIV - Saturday, February 18, 2012 - link

    Since there's a new driver out for there cards we can now regard these results with a grain of salt. Revise the benchmarks!
  • DaOGGuru - Thursday, March 1, 2012 - link

    I don't know why people keep forgeting about the 560ti 2win. Yes I said 2win = 2 560ti processors on one card. It still kills the 7970 numbers in BF3 by 20Fps. and is same price. It also beats the 580 and is cheaper. It's a single card with 50amp min. draw and it will smoke anything except 590 and the 6990...

    http://www.guru3d.com/article/evga-geforce-gtx-560...
  • CeriseCogburn - Thursday, March 8, 2012 - link

    Oh, right, well this isn't an nvidia card review, so we won't hear from 50 posts about how some CF (would be SLI of course in this case) combo will whip the crap out of it in performance and price...
    You know ?
    That's how it goes...
    Usually the articel itself rages on about how some amd CF combo is really so much good and better and blah blah blah.... then the rpice perf, then the results - on and on and on ....
    ---
    The angry ankle biters are swarmed up on the under red dog radeon side...
    --
    So you made a very good point, I'm just sorry it took 29 pages of reading to get to it, in it's glorious singularity.... you shouldn't strike out in independent thought like that it's dangerous.... not allowed unless the card being reviewed is an nvidia !!!!
  • DaOGGuru - Thursday, March 1, 2012 - link

    oops... forgot to say look at previous post links BF3 rating for the 560ti 2win and compare to this charts 7970 fps. The 2win is pumping out @20 more FPS and is $50.00 - $100.00 cheaper than the 7970... lame.. ATi is still behind Nvidia but proud of it! lol They are just now catching up to Nvidia's tessellations and oh and AFTER they changed to a "cuda core copy" architecture and posting it as big news... Evga's older 560ti 2win still dusts it by 20FPS.. lame.
  • DaOGGuru - Thursday, March 1, 2012 - link

    sorry 10FPS not 20.. it's late.
  • DaOGGuru - Thursday, March 1, 2012 - link

    I don't get what's the hub-bub about the 7970.. sure it's the fastest single cpu;BUT, for $50.00-$100.00 less you can get the 560Ti 2win (dual cpu) that smokes the 7970 and the 2win PCB does have an SLI bridge and is cabapable of doing SLI to a second card but it's currently locked by Nvidia (see paragraph 3).

    Also, the 2win draws a min of only 50amps (way less than most sli configurations) 1. has a considerably lower noise dba, 2. runs cooler and with less power than almost all the high end cards and 3. will run 3 montiors in Nvidia 2D and 3D surround off a single card! 4.Will kill the GTX 580 by @33-23% (depending on review) 5. Will beat the 590 in some sample testing for TDP. And finally 6. will kill the 7970 by 10-20FPS in BF3 including by 10FPS in 1920x1200 4AA-16AF Ultra high mode. So, why have people forgotten the 2win? It's a singlecard, multi-GPU, full 3D/2D surround without a second card in SLI, $500.00USD beast !

    OH and for those that say you can't SLI with a second 2win.... http://www.guru3d.com/article/evga-geforce-gtx-560... (this review states on conclusion page) > quote " you will have noticed there is a SLI connector on the PCB. Unfortunately you can not add a second card to go for quad-SLI mode. It's not a hardware limitation, yet a limitation set by NVIDIA, the GTX 560 Ti series is only allowed in 2-way SLI mode, which this card already is."

    ... So actually, the card is cabale 2card SLI but Nvidia for some (gosh aweful reason) won't let the dog off the chain. Probably because it will absolutely kill the need for a GTX580, 570, 560 Ti SLI configuration for ever!

    Resources: (pay attention to BF3 FPS and compare to 7970 FPS in this article.)
    http://www.anandtech.com/show/5048/evgas-geforce-g...
    http://www.guru3d.com/article/evga-geforce-gtx-560...
    Peace...
  • DaOGGuru - Thursday, March 1, 2012 - link

    I don't get what's the hub-bub about the 7970.. sure it's the fastest single CPU; BUT, for $50.00-$100.00 less you can get the 560Ti 2win (dualCPU) that smokes the 7970 and the 2win PCB does have an SLI bridge and is capable of doing SLI to a second card but it's currently locked by Nvidia (see paragraph 3).

    Also, the 2win draws a min of only 50amps (way less than most sli configurations) 1. Has a considerably lower noise DBA, 2. runs cooler and with less power than almost all the high end cards and 3. Will run 3 monitors in Nvidia 2D and 3D surround off a single card! 4.Will kill the GTX 580 by @33-23% (depending on review) 5. Will beat the 590 in some sample testing for TDP. And finally 6. will kill the 7970 by 10-20FPS in BF3 including by 10FPS in 1920x1200 4AA-16AF Ultra high mode. So, why have people forgotten the 2win? It's a single card, multi-GPU, full 3D/2D surround without a second card in SLI, $500.00USD beast !

    OH and for those that say you can't SLI with a second 2win.... http://www.guru3d.com/article/evga-geforce-gtx-560... (this review states on conclusion page) > quote " you will have noticed there is a SLI connector on the PCB. Unfortunately you cannot add a second card to go for quad-SLI mode. It's not a hardware limitation, yet a limitation set by NVIDIA, the GTX 560 Ti series is only allowed in 2-way SLI mode, which this card already is."

    ... So actually, the card is capable 2card SLI but Nvidia for some (gosh awful reason) won't let the dog off the chain. Probably because it will absolutely kill the need for a GTX580, 570, 560 Ti SLI configuration forever!

    Resources: (pay attention to BF3 FPS and compare to 7970 FPS in this article.)
    http://www.anandtech.com/show/5048/evgas-geforce-g...
    http://www.guru3d.com/article/evga-geforce-gtx-560...
    Peace...
  • CeriseCogburn - Thursday, March 8, 2012 - link

    Ummm.... I read you, I see your frustration with all the posts - just refer to my one above there - you really should not be dissing the new amd like that - they like are 1st and uhh... nvidia is evil... so no comparisons like that are allowed when the fanboy side content is like 100 to 1....
    Now next nvidia card review you will notice a hundred posts on how this or that CF beats the nvidia in price perf and overall perf, etc, and it will be memorized and screamed far and wide...
    Just like... your point "doesn't count", okay ?
    It's best to ignore you GREEN fanboy types... ( yes even if you point out gigantic savings, or rather especially when you do...)
    Thanks for waiting till page 30 - a wise choice.
  • CeriseCogburn - Sunday, March 11, 2012 - link

    Southern Islands is a whole generation late. AMD promised us this SI in the last generation 6000 series. Then right before that prior release, they told us they had changed everything and 6000 was not Southern Islands anymore. LOL
    Talk about late - it's what two years late ?
    Maybe it's three years....
    In every case here, Nvidia beat them to the core architecture by two years. Now amd is merely late to the party crashing copycats....
    That's late son, that's not original, that's not innovative, that's not superior, it's tag a long tu loo little sister style.

Log in

Don't have an account? Sign up now