With only sixteen PCIe 3.0 lanes available on a Z77 motherboard paired with an Ivy Bridge CPU, when we get to three or four-way GPU solutions these GPUs are itching to get more bandwidth.  The Z77 specification limits us to three GPUs anyway, at x8/x4/x4.  For some extra cost on the motherboard, we can add in a PLX PEX 8747 chip that effectively increases our PCIe 3.0 lane count, giving 32 PCIe 3.0 lanes overall.  Today we discuss this technology, and look at four motherboards on sale today that utilize this PLX chip - the Gigabyte G1.Sniper 3, the ASRock Z77 Extreme9, the ECS Z77H2-AX and the EVGA Z77 FTW.

Multi-GPU on Z77

Aiming for users for 3-way and 4-way GPU setups is aiming for a very niche sub-section of an enthusiast crowd.  Here we are pinpointing those that have the hard cash to create a very nice system, capable of powering several large monitors at high resolution for fast paced and detailed gaming.  Many of these setups are bespoke, specifically designed in modded cases, and a good number use water-cooling to get around the generation of heat.  Let us not forget the power draw, with multiple GPU setups requiring significant power supplies or even a dual power supply solution.  It gets even worse if everything is overclocked as well.

These users often keep up to date with the latest and greatest hardware.  No processor, no graphics card and no chipset is too new for them.  If any of our enthusiasts here were asked about which platform would be best for a multi-GPU setup, we would respond with X79 and Sandy Bridge-E, should money be no object.  Even for high throughput users, such as scientific simulators or video editing - Sandy Bridge-E holds all the performance.  There is one thing Sandy Bridge-E lacks though - PCIe bandwidth.

With the majority of X79 stuck in PCIe 2.0 land, we start to hit bottlenecks transferring data between memory and GPU.  Our limit is 8 GB/s for a full sixteen lane PCIe 2.0 port.  If we jump into Z77 and Ivy Bridge, we have PCIe 3.0.  For a full sixteen lane PCIe 3.0 port, we have double the bandwidth at 16 GB/s.

The other issue is lanes - X79 and SBE have 40 PCIe 2.0 lanes to distribute, giving x16/x8/x8/x8 in a four-way GPU scenario.  Even with double the bandwidth per lane of PCIe 3.0, Z77 and Ivy Bridge can only play around with sixteen lanes in total, as shown by the chipset diagram:

The Z77 specification states that these lanes can be split in any of the following ways:

PCIe Layout Comparison Chart
Configuration GPUs GPU 1 GPU 2 GPU 3 GPU 4
1 1 16x PCIe 3.0 - - -
2 2 8x PCIe 3.0 8x PCIe 3.0 - -
3 2 16x PCIe 3.0 4x PCIe 2.0 - -
4 3 8x PCIe 3.0 4x PCIe 3.0 4x PCIe 3.0 -
5 3 8x PCIe 3.0 8x PCIe 3.0 4x PCIe 2.0 -
6 4 8x PCIe 3.0 4x PCIe 3.0 4x PCIe 3.0 4x PCIe 2.0

For many of these setups, motherboard manufacturers can also choose to direct four of the PCIe 2.0 lanes from the chipset to also aid with CrossFire scenarios (SLI is not certified in this configuration).  The downside of this comes from the limited bandwidth of the PCIe 2.0 lanes, which have half the bandwidth of the PCIe 3.0 lanes, but there is also added latency of navigating data through the chipset rather than across the PCIe bus.

For mini-ITX solutions, we will see the one GPU scenario.  For many microATX solutions, one of the dual GPU scenarios will dominate.  In full ATX mode, manufacturers have all the configurations listed above to choose from.  Luckily, here at AnandTech, we have reviewed the majority of the configurations (1, 2, 4 and 5) in the table above in order to provide comparison points.

Back in the days of X58