A Quick Primer on ILP

NVIDIA throws ILP (instruction level parallelism) out the window while AMD tackles it head on.

ILP is parallelism that can be extracted from a single instruction stream. For instance, if i have a lot of math that isn't dependent on previous instructions, it is perfectly reasonable to execute all this math in parallel.

For this example on my imaginary architecture, instruction format is:

LineNumber INSTRUCTION dest-reg, source-reg-1, source-reg-2

This is compiled code for adding 8 numbers together. (i.e. A = B + C + D + E + F + G + H + I;)

1 ADD r2,r0,r1
2 ADD r5,r3,r4
3 ADD r8,r6,r7
4 ADD r11,r9,r10
5 ADD r12,r2,r5
6 ADD r13,r8,r11
7 ADD r14,r12,r13
8 [some totally independent instruction]
...

Lines 1,2,3 and 4 could all be executed in parallel if hardware is available to handle it. Line 5 must wait for lines 1 and 2, line 6 must wait for lines 3 and 4, and line 7 can't execute until all other computation is finished. Line 8 can execute at any point hardware is available.

For the above example, in two wide hardware we can get optimal throughput (and we ignore or assume full speed handling of read-after-write hazards, but that's a whole other issue). If we are looking at AMD's 5 wide hardware, we can't achieve optimal throughput unless the following code offers much more opportunity to extract ILP. Here's why:

From the above block, we can immediately execute 5 operations at once: lines 1,2,3,4 and 8. Next, we can only execute two operations together: lines 5 and 6 (three execution units go unused). Finally, we must execute instruction 7 all by itself leaving 4 execution units unused.

The limitations of extracting ILP are on the program itself (the mix of independent and dependent instructions), the hardware resources (how much can you do at once from the same instruction stream), the compiler (how well does the compiler organize basic blocks into something the hardware can best extract ILP from) and the scheduler (the hardware that takes independent instructions and schedules them to run simultaneously).

Extracting ILP is one of the most heavily researched areas of computing and was the primary focuses of CPU design until the advent of multicore hardware. But it is still an incredibly tough problem to solve and the benefits vary based on the program being executed.

The instruction stream above is sent to an AMD and NVIDIA SP. In the best case scenario, the instruction stream going into AMD's SP should be 1/5th the length of the one going into NVIDIA's SP (as in, AMD should be executing 5 ops per SP vs. 1 per SP for NVIDIA) but as you can see in this exampe, the instruction stream is around half the height of the one in the NVIDIA column. The more ILP AMD can extract from the instruction stream, the better its hardware will do.

AMD's RV770 (And R6xx based hardware) needs to schedule 5 operations per thread every every clock to get the most out of their hardware. This certainly requires a bit of fancy compiler work and internal hardware scheduling, which NVIDIA doesn't need to bother with. We'll explain why in a second.

Instruction Issue Limitations and ILP vs TLP Extraction

Since a great deal of graphics code manipulates vectors like vertex positions (x,y,c,w) or colors (r,g,b,a), lots of things happen in parallel anyway. This is a fine and logical aspect of graphics to exploit, but when it comes down to it the point of extracting parallelism is simply to maximize utilization of hardware (after all, everything in a scene needs to be rendered before it can be drawn) and hide latency. Of course, building a GPU is not all about extracting parallelism, as AMD and NVIDIA both need to worry about things like performance per square millimeter, performance per watt, and suitability to the code that will be running on it.

NVIDIA relies entirely on TLP (thread level parallelism) while AMD exploits both TLP and ILP. Extracting TLP is much much easier than ILP, as the only time you need to worry about any inter-thread conflicts is when sharing data (which happens much less frequently than does dependent code within a single thread). In a graphics architecture, with the necessity of running millions of threads per frame, there are plenty of threads with which to fill the execution units of the hardware, and thus exploiting TLP to fill the width of the hardware is all NVIDIA needs to do to get good utilization.

There are ways in which AMD's architecture offers benefits though. Because AMD doesn't have to context switch wavefronts every chance it gets and is able to extract ILP, it can be less sensitive to the number of active threads running than NVIDIA hardware (however both do require a very large number of threads to be active to hide latency). For NVIDIA we know that to properly hide latency, we must issue 6 warps per SM on G80 (we are not sure of the number for GT200 right now), which would result in a requirement for over 3k threads to be running at a time in order to keep things busy. We don't have similar details from AMD, but if shader programs are sufficiently long and don't stall, AMD can serially execute code from a single program (which NVIDIA cannot do without reducing its throughput by its instruction latency). While AMD hardware can certainly handle a huge number of threads in flight at one time and having multiple threads running will help hide latency, the flexibility to do more efficient work on serial code could be an advantage in some situations.

ILP is completely ignored in NVIDIA's architecture, because only one operation per thread is performed at a time: there is no way to exploit ILP on a scalar single-issue (per context) architecture. Since all operations need to be completed anyway, using TLP to hide instruction and memory latency and to fill available execution units is a much less cumbersome way to go. We are all but guaranteed massive amounts of TLP when executing graphics code (there can be many thousand vertecies and millions of pixels to process per frame, and with many frames per second, that's a ton of threads available for execution). This makes the lack of attention to serial execution and ILP with a stark focus on TLP not a crazy idea, but definitely divergent.

Just from the angle of extracting parallelism, we see NVIDIA's architecture as the more elegant solution. How can we say that? The ratio of realizable to peak theoretical performance. Sure, Radeon HD 4870 has 1.2 TFLOPS of compute potential (800 execution units * 2 flops/unit (for a multiply-add) * 750MHz), but in the vast majority of cases we'll look at, NVIDIA's GeForce GTX 280 with 933.12 GFLOPS ((240 SPs * 2 flops/unit (for multiply-add) + 60 SFUs * 4 flops/unit (when doing 4 scalar muls paired with MADs run on SPs)) * 1296MHz) is the top performer.

But that doesn't mean NVIDIA's architecture is necessarily "better" than AMD's architecture. There are a lot of factors that go into making something better, not the least of which is real world performance and value. But before we get to that, there is another important point to consider. Efficiency.

Derek Gets Technical Again: Of Warps, Wavefronts and SPMD AMD's RV770 vs. NVIDIA's GT200: Which one is More Efficient?
POST A COMMENT

215 Comments

View All Comments

  • calumhm - Friday, September 11, 2009 - link

    i mean, ATI invented the unified pixel/shader architechture, or so i believe, and this generation they've got dx10.1 level hardware something like 7 months and counting before Nvidia have any.

    Also i once read an article about SLI and Crossfire, about how SLI has only one rendering type, scissors. (meaning the screen is divided in two) Whereas ATI have scissors, tiled, (so that the more demanding areas of the screen are better divided amongst the cards) and others.
    Also, you can combine any HD series ATI card with any HD series card! thats way better than having to say, buy another 7800 for SLI because just one isn't doing it anymore, even though the 9800 series are out. With CFire you could add a 4870 to your old 3870!

    Im currently with Nvidia (a 9600gt (found one for £60!)) but am frequently impressed by ATI.

    -IS- ATI the smart customer's choice?
    Reply
  • heaneyforestrntpe68 - Thursday, October 21, 2021 - link

    well I am looking forward to a single card setup. SLI or CF is beyond the reach of my pockets. :P https://bit.ly/2Z7E1jr Reply
  • billywigga - Friday, August 29, 2008 - link

    im pretty shore the 4870 is low profile ive been looking everywhere for a low profile graphics card and i think i foung a high ends one unlike the geforce 8400 and the 8600 those arenot very good and they dont look good either but where do i buy the 4870 Reply
  • Hrel - Thursday, August 21, 2008 - link

    Why has there been no article comparing the 8800GT to the 9800GT? Is it just a rebrand, are there noticeable performance differences. It's 9 series, I assume it has hybrid power, but I don't know. Anandtech, PLEASE! Do an article on this. Reply
  • billywigga - Friday, August 29, 2008 - link

    bang for the buck id get the 9800 because its newer also all the diffrence is it has more intagrated ram your saving a lot if you just get more ram to your computer. Reply
  • firewolfsm - Friday, August 1, 2008 - link

    I'm trying to do the same benchmark for Crysis for my 4850 as I have a similar system and a fresh vista install. Just wondering what kind of driver settings you used. Reply
  • spikeysting - Saturday, July 19, 2008 - link

    I just got it for $179 at Frys. Such a good deal. Reply
  • Yangorang - Tuesday, July 8, 2008 - link

    Anyone tried this mod?
    http://www.hardforum.com/showthread.php?t=1319658">http://www.hardforum.com/showthread.php?t=1319658
    Reply
  • jALLAD - Friday, July 4, 2008 - link

    I would for sure go for the 4870. (I am a Quake Wars fan boy u see :P)
    But I was unsure how these would perform on Linux. Is the driver support reliable? Right now I use a 7950 NVIDIA and their support on Linux is almost shite. I was wondering whether its better or worse...

    Anyone ?
    Reply
  • KriegenSchlagen - Monday, July 7, 2008 - link

    If you are a Quake Wars fan, aren't the scores higher in 3-4 GeForce SLI configs vs. Crossfire mode 4870? Reply

Log in

Don't have an account? Sign up now