MultiGPU Update: Finding the True Halo with 4-way
by Derek Wilson on February 28, 2009 11:45 PM EST- Posted in
- GPUs
Who Scales ... And Timing
In previous articles, we took a look at some stats on how many tests scale more or less than a certain threshold. Well, it gets a little trickier here to make sense of everything, so rather than pick points ourselves, we decided to list a bunch of them. In the chart below, we've listed the number of tests that fail to scale better than the percentage listed at the top of the column. Lots of tests fail to scale at what we would call a reasonable percentage, but people looking at these parts have a different definition of reasonable.
The maximum scaling percent is 100% just like with scaling from 1 to 2 GPU. But fewer games scale past 2 GPUs, and of those that do, fewer scale as near linearly past 2 GPUs. And to top that off, many tests that do scale at all scale right into a system limitation. A good chunk of games fail to scale past 5%, and fully 13 out of 18 tests fail to scale beyond 50% in each case we tested.
| <2.5 | <5 | <10 | <15 | <20 | <25 | <33.3 | <50 | |
| NVIDIA GeForce GTX 295 Quad SLI | 5 | 5 | 6 | 7 | 8 | 9 | 10 | 13 |
| NVIDIA GeForce 9800 GX2 Quad SLI | 7 | 7 | 8 | 8 | 8 | 8 | 8 | 13 |
| ATI Radeon HD 4870 1GB Quad CrossFire | 5 | 5 | 5 | 7 | 9 | 11 | 12 | 13 |
| ATI Radeon HD 4850 Quad CrossFire | 6 | 7 | 9 | 10 | 10 | 10 | 12 | 13 |
Looking at the lower end we can see a bunch of tests fail to scale at all. Also, at 33%, many fewer situations scale at this rate than when moving from 1 to 2 GPUs. Clearly 4-way multiGPU solutions are not designed with anything but maximum performance in mind. Scaling isn't as important as the fact that these solutions can provide some degree of higher performance in some situations.
We would also like to note that when paying ridiculous amounts of money for not quite as ridiculous performance gains, the robustness of the solution is of very high importance. No one wants to pay over $1000 and get a solution that sometimes provides good scaling and sometimes degrades performance. Neither AMD nor NVIDIA are immune to this, but we would like to see this issue tackled in more earnest beyond simply noting that SLI and CrossFire can be disabled if trouble arises.
NVIDIA does have an advantage at this level though. We would love to see AMD get their driver act together and consistently have drivers that provide good scaling and performance in newly release AAA titles on launch day. We would also love to see them refine their driver development model in order to make sure that improvements released as hotfixes always make it into the very next WHQL driver released (which is currently not the case). Everywhere else, this is merely a slight annoyance that people may take or leave. At the highest of the high end, however, a delay in getting good scaling or the need to use less recent drivers that contain more recent fixes (and juggling which is which) can prove more than just a trifle. For such a high price, NVIDIA delivers a better experience on this count.
Additionally, until OpenCL matures, CUDA is a better GPU computing alternative to what AMD offers, and PhysX can provide additional flexibility now that more titles are beginning to adopt it. Actually, this is the space in which we currently see the most value in CUDA and PhysX, as those in the market for equipment this high end will be more interested in these niche features that don't have broad enough support or large enough current impact for us to heartily recommend them as a must have for everyone.
Technophiles (like myself) that are willing to put this kind of money into hardware often get excited about the hardware on a more than practical level. The technology itself, rather than the experience it delivers, can often be a source of enjoyment for the end user. I know I like playing with PhysX and CUDA in spite of the fact that these technologies still need broader support to compel the average gamer.
Performance, itself, cannot be ignored, and is indeed of the highest importance when it comes to the highest end configurations. We will include the value graphs, but we expect that the line closest to the top of the performance charts are the key factor in decision making when it comes to Quad GPU options. The troubles that come with maintaining a 4 GPU configuration are not worth it if the system doesn't provide a consistently top of the line experience.
Facebook
Twitter
RSS
44 Comments
View All Comments
Razorbladehaze - Thursday, March 5, 2009 - link
No, I am definitely not wrong. You have obviously never taken any courses in research statistic, or methods. No matter what this dictionary is that you pull this from experimental procedures remain the same and these, for FPS, are OBJECTIVE.Your logic is OK regarding programing for specific configurations (not really accurate, but that is besides the overall point here), what is important here is your reasoning is completely wrong. You apply it to one set of programs but not to another and futher deny that such logic you posited even exists within one program set. WTH.
Yea come back after completing your courses in research methods and experimental procedures.
Furthermore you may want to buff up on your courses in logic and reasoning.
To quote the soup NAZI (from Seinfeld):
NO SOUP FOR YOU!!!!
Razorbladehaze - Thursday, March 5, 2009 - link
Ok after thinking briefly i decided to post again.Basically what you need to understand for the whole testing aspect, is basically anything that can be operationally defined can be tested objectively.
Furthermore my ideas on your thoughts relating to 3d mark and game benchmarks.
3d mark has what many would say is good reliability in testing terms such as test-retest. but so would any good benchmark for games, especially built-in game benchmarks.
What is most important is validity, that the outcome actually measures, what it is supposed to measure.
3d mark scores are a conglomerate score and actually relate to nothing other than that final score. These measures are taken from data such as a CPU score, FPS score, and other data correct??
To quote you "Every scientific experiment strives to remove variables from the testing process..."
umm hello, how does 3d mark determine its outcome score again, yeah multiple variables, that's right MULTIPLE VARIABLES.
Ok so going back to what is important is Validity: that which is being measured is actually being measured.
A final outcome score in 3d mark is valid only for comparisons to other 3d marks scores. NO IF, AND's, OR BUTTS!!!!
FPS - in each game is a valid OBJECTIVE data outcome score, as it is the operational definition of GPU rendering used here. Running multiple benchmarking runs in each game provides reliability. Measuring FPS across multiple games with a similar game engine lends concurrent validity to the experiments. This type of testing also allows convergent and divergent validity to be addressed here when using multiple game titles with different engines or similar offshoots.
So in laymen's terms for the world of experimental procedures 3d mark can suck my balls
Hrel - Saturday, March 21, 2009 - link
yes 3D mark can only be compared to 3D mark, I already said that, thanks for repeating what a more intelligent person already said as that seems to be what you do; classes, so you DO let other people tell you how and what to think. Anyway, 3D mark rates the graphics card, and scores can be compared across a wide range of cards without worrying about variables like with games, where one game is better than another but only on some GPU's. 3D mark actually has a score for JUST the GPU, but I guess you've never used it.I already said the only way to get an accurate representation of GPU performance is to test multiple games and 3D mark, it would best if you could just test every game out there, but no one has that kind of time.
so yeah, game testing, even when using the games "in game" benchmark still only measures performance in THAT game, so it means nothing to people who don't play that game. As it can't be used to determine performance in any other game; where as 3D Mark can. One game test could say that the HD4870 is faster and subsequently a more powerful card than the GTX285 even though we all know it's not just because that game runs better on AMD hardware. That's all I was saying and all I was trying to get across, I really wasn't looking for a pissing contest with an anonamyss internet user.
A test in one game, can't give you an accurate idea of how a GPU will perform in other games; 3D Mark, when compared to the scores of other cards, and used in conjunction with FPS scores, CAN!!!!
Razorbladehaze - Thursday, March 5, 2009 - link
Oh yeah and in reviewing what I wrote, I glossed over subjective data. and will further punish 3d mark.most common subjective data sets are: Ranking data, Rating data, interview data, categorical data.
The first two; Ranking and rating is really what the outcome score of 3d mark.
It is a rating on their own scale and it allows them to rank systems by that score.
So with this final bit of info answer if 3d mark outcomes are really subjective or objective(what does it measure besides its own ranking and rating scales).
npp - Sunday, March 1, 2009 - link
It's just common, healthy sense that tells me - something is terribly wrong here. I can't imagine anyone in his right mind spending at least $880 for a (non-professional) graphics solution which draws more than 700W under load, and delivers almost no additional performance, compared to far cheaper alternatives! (If you look closely at the charts you'll see that some games are perfectly playable on even single-GPU solutions). Ok, there are some gains, but they are so minimal that the performance/price and performance/power ratios are simply disturbingly low. I compared the situation once to a quad-core CPU that delivers at most, say, 2,2x speedup over single-cores - with dual cores delivering 1,9. Who would buy such a CPU?Of course, I'm not trying to say people should buying such things; it's the vendors who need to do some hard work on that poor scaling. I would never consider buying 4 GPUs if they deliver anything under 3,5x speedup. It sounds crazy right now, but who knows, there may be interesting things to come.
SiliconDoc - Wednesday, March 18, 2009 - link
" (If you look closely at the charts you'll see that some games are perfectly playable on even single-GPU solutions). "CORRECTION : All games are perfectly playable on plenty more than just a few single gpu solutions.
YES - that's the real truth.
mastrdrver - Sunday, March 1, 2009 - link
It would have been since to see some AA scaling. It was made apparent one the first page that scaling was going to be under performing when at 4 gpus from either 2 or 3. I think just about everyone saw that coming, especially with max of 4x AA.With this much power, it would have been interesting to see what happened to scaling when AA gets turned up. Run the normal 4xAA for all the games. Then, start cranking up the AA and see if that might make the the move from 2 to 4 gpus look better.
I know there are those who, and I'll agree to an extent, say that there isn't a difference from 4x to something like 12x or 24x edge detect (and whatever nVidia uses). Even the move from 4x to 8x is rather small in image improvement. Still though, if your going to drop that kind of money into a set of cards, why not put it to use? While I'm not sure about the 12x to 24x move on edge detect, I do seem to notice things here and there by moving from 4x to 8x and even a few things in 12x AAED. I've seen a hand full of sites look at doing the higher AA levels, but they always do it with 1, or at most 2, gpus when the game is already stressing them at 4x. Why not take a look at these higher levels of AA when there is an abundance of gpu power that seems to be idle.
Overall, it would be an interesting look into something that I'm not sure I've seen any site look at in any kind of depth when getting to the power level of 4, or even 3, gpus.
So are the games not scaling with more gpu power because games/drivers are not optimized for 3 or 4 gpus, or are they not scaling because AA levels are left at "lower" levels? (Outside, of course, being CPU limited because of the abundance of gpu power.)
xxtypersxx - Sunday, March 1, 2009 - link
I have noticed a recent trend where people post comments accusing Anandtech of favoring Intel/Nvidia. Now I absolutely can't stand bias and stopped reading sites like Tom's long ago because of it, but Anandtech has never seemed even remotely unfair. I think the issue is people have noticed a lot of praise for Intel and Nvidia over the past couple of years, but the reason is they have had consistently better performance. When the 4xxx series launched, Anandtech had nothing but praise for it and it's superior value and even criticized Nvidia's stagnant technology and high prices in many articles. So basically what I'm trying to point out is it isn't Anandtech that has been biased against AMD/ATI, its the real world performance that has been skewed and they are simply reporting on it.SiliconDoc - Wednesday, March 18, 2009 - link
Oh good golly. Reviewers are expected to be unbiased, but they almost never are. Unbiased is a cold, boring read for most fans.Anand used to have a very severe bias in favor of intel many years ago.
Now you mentioned Tom's was biased and you left Tom's and are now here, showing of course YOUR BIAS, because if you dare to notice bias here, then you've got to leave HERE, and go somewhere else.
Believe me, if you love ati, you'll stay for the card reviews - that have a huge bias for ati, in every worded paragraph.
I am curious though how you percieved Tom's bias. I bet he didn't favor the red cards ? Was that it ?
I'm very interested in that.
Would you be so kind as to say what bias you found there ?
deputc26 - Sunday, March 1, 2009 - link
better change that to "CrossfireX"