Launching the #CPUOverload Project: Testing Every x86 Desktop Processor since 2010
by Dr. Ian Cutress on July 20, 2020 1:30 PM EST#CPUOverload: What is Realistic?
Truth be told, the concept of a project to benchmark almost 700-900 processors has been rattling around in my head for a few years. I actually wrote the first segment of this article way back in 2016. However, over the course of 2016 and 2017, building new testing suites has taken longer, priorities changed, and the project didn’t so much as get shelved as somewhat pushed down the order on a semi-permanent basis until there was an ideal opening. Those of you who have followed the site may have noticed my responsibilities increase over time, darting 200k miles a year around the world. It can be difficult to keep a large project buoyant without constant attention.
Between 2016 and today, we’ve still be churning though the tests on the hardware, and updating our benchmark database with as many chips as we can find, even if it wasn’t under a governed project. The most recent version of our CPU2019 Bench has 272 CPUs with data recorded on up to 246 benchmark data points for each, just to showcase perhaps what one person can do in a given year. However, the focus of Bench being a specific project wasn’t necessarily a primary target of the site. With the launch of our Bench2020 suite, with a wider variety of tests and analysis, we’re going to put this into action. That’s not to say I have more time than normal (I might have to propose what we can do about getting an intern), but with the recent pandemic keeping me on the ground, it does give a chance to take stock about what users are really after.
With #CPUOverload, the goal is to do more than before, and highlight the testing we do. This is why I’ve spent the best part of 25-30 pages talking about benchmark sustainability, usefulness, automation, and why every benchmark is relevant to some of our user base. Over the last decade, as a hardware tester providing results online for free, one obvious change in the requests from our readers has been to include specific benchmarks that target them, rather than generic ones related to their field. That’s part of what this project is, combined with testing at scale.
Users also want to find their exact CPU, and compare it to an exact CPU potential upgrade – a different model, at least in today’s naming conventions, might have different features. So getting exactly what you want to compare is always going to be better – being able to see how your Intel Core i5-2380P in that Dell OEM system you have had for 7 years compares to a newer Ryzen 7 2700E or Xeon E-2274G is all part of what makes this project exciting. That essence of scale, and trying to test as many different CPU variants as possible, is going to be a vital part of this project.
Obviously the best place to start with a project like this is two-fold: popular processors and modern processors. These get the most attention, and so covering the key parts from Coffee Lake, Kaby Lake, Ryzen and HEDT are going to be high on our list to start. The hardware that we’re also testing for review also gets a priority, so that’s why you might start seeing some Zhaoxin or Xeon/EPYC data enter Bench very soon. One funny element is that if you were to start listing what might be ‘high importance processors’, it very easily come back with a list of between 25-100 SKUs, with various i9/i7/i5/i3 and R7/R5/R3/APU as well as Intel/AMD HEDT and halo parts in there – that’s already 10 segments! Some users might want us to focus on the cheap Xeon parts coming out of China too. Obviously whatever our users want to see be tested, we want to hear about it.
As part of this project, we are also expecting to look at some retrospective performance. Future articles might include ‘how well does Ivy Bridge i5 perform today’, or given AMD and Intel’s tendency to compare five year products to each other, we are looking to do that too, in both short and longer form articles.
When I first approached AMD and Intel’s consumer processor divisions about this project, wondering how much interest there would be for it, both came back to me with positive responses. They filled in a few of my hardware gaps, but cautioned that even as internal PR teams, they won’t have access to most chips, especially the older ones. This means that as we process through the hardware, we might start reaching out to other partners in order to fill in the gaps.
Is testing 900 CPUs ultimately realistic? Based on the hardware I have today, if I had access to Narnia, I could provide data for about 350 of the CPUs. In reality, with our new suite, each CPU takes 20-30 hours to test on the CPU benchmarks, and another 10 hours for the gaming tests. Going for 50-100 CPUs/month might be a tough ask, but let’s see how we get on. We have these dozen or so CPUs in the graphs here to start.
Of course, comments are always welcome. If there’s a CPU, old or new, you want to see tested, then please drop a comment below. It will help how I arrange which test beds get priority.
Facebook
Twitter
RSS
110 Comments
View All Comments
Smell This - Monday, July 20, 2020 - link
;- )
Oxford Guy - Monday, July 20, 2020 - link
"If there’s a CPU, old or new, you want to see tested, then please drop a comment below."• i7-3820. This one is especially interesting because it had roughly the same number of transistors as Piledriver on roughly the same node (Intel 32nm vs. GF 32 nm).
• 5775C
• 5675C (which outperformed and matched the 5775C in some games due to thermal throttling)
• 5775C with TDP bypassed or increased if this is possible, to avoid the aforementioned throttling
• I would really really like you to add Deserts of Kharak to your games test suite. It is the only game I know of that showed Piledriver beating Intel's chips. That unusual performance suggests that it was possible to get more performance out of Piledriver if developers targeted that CPU for optimization and/or the game's engine somehow simply suited it particularly.
• 8320E or 8370E at 4.7 GHz (non-turbo) with 2133 CAS 9-11-10 RAM, the most optimal Piledriver setup. The 9590 was not the most performant of the FX line, likely because of the turbo. A straight overclock coupled with tuned RAM (not 1600 CAS 10 nonsense) makes a difference. 4.7 GHz is a realistic speed achievable by a large AIO or small loop. If you want air cooling only then drop to 4.5 Ghz but keep the fast RAM. The point of testing this is to see what people were able to get in the real world from the AMD alternative for all the years they had to wait for Zen. Since we were stuck with Piledriver as the most performant Intel alternative for so so many years it's worth including for historical context. The "E" models don't have to be used but their lower leakage makes higher clocks less stressful on cooling than a 9000 series. 4.7 GHz was obtainable on a cheap motherboard like the Gigabyte UD3P, with strong airflow to the VRM sink.
• VIA's highest-performance model. If it won't work with Windows 10 then run the tests on it with 8.1. The thing is, though... VIA released an update fairly recently that should make it compatible with Windows 10. I saw Youtube footage of it gaming, in fact, with a discrete card. It really would be a refreshing thing to see VIA included, even though it's such a bit player.
• Lynnfield at 3 GHz.
• i7-9700K, of course.
Oxford Guy - Monday, July 20, 2020 - link
Regarding Deserts of Kharak... It may be that it took advantage of the extra cores. That would make it noteworthy also as an early example of a game that scaled to 8 threads.Oxford Guy - Monday, July 20, 2020 - link
Also, the Chinese X86 CPU, the one based on Zen 1, would be very nice to have included.Oxford Guy - Monday, July 20, 2020 - link
VIA CPUs tested with games as recently as 2019 (there was another video of the quad core but I didn't find it today with a quick search):https://www.youtube.com/watch?v=JPvKwqSMo-k
https://www.youtube.com/watch?v=Da0BkEW459E
The Zhaoxin KaiXian KX-U6880A would be nice to see included, not just the Chinese Zen 1 derivative.
Oxford Guy - Monday, July 20, 2020 - link
"due to thermal throttling"TDP throttling, to be more accurate. I suppose it could throttle due to current demand rather than temp.
axer1234 - Monday, July 20, 2020 - link
honestly i would love to know how different generation processor perform today especially higher core count. like prescott series pentium 4 athlon II phenomX6 core2 duo core2quad nehlam sandy bridge bulldozer etc with todays generation work loads and offeringin many scenario like word excel ppt photoshop it all works very well still in many offices
its just the new generation of application slowing it down for almost the same work etc
herefortheflops - Monday, July 20, 2020 - link
@Dr. Cutress.,As someone that has been dealing with similar or greater product testing challenges and configuration complexity for the better part of a decade or so, I would like to commend you for your ambitious goals and efforts so far. Additionally, I could be of high value to your effort if you are willing to discuss. I have reviewed in-depth the bench database (as well as competing websites) and I have come to the conclusion the Anandtech bench data is of very limited usefulness at present--and would require some significant changes to the data being collected/reported and the way things have been done to this point. I do understand where the industry is going, the questions the readers are going to be asking of the data, and the major comparisons that will be attempted with the data. Unfortunately, much of your effort may easily become irrelevant unless you proceed with some extreme caution to provide data with more utility. I also know methods to accomplish the desired result while reducing the size and cost of the task at hand. Reply by e-mail if you are interested in talking.
Best,
-A potential contributor to your effort.
Bensam123 - Tuesday, July 21, 2020 - link
Despite how impressive this is, one thing that hasn't been tackled is still multiplayer performance and it vastly changes recommendations for CPUs (doesn't effect GPUs as much).It goes from recommending a 6 core chip hands down to trying to make a case for 4 core chips still in this day and age. I own a 3900x and 2800 and I can tell you hands down Modern Warfare will gobble 70% of that 12 core chip, sometimes a bit more, that's equivalent to maxing out a 8 core of the same series. That vastly changes recommendations and data points. It's not just Modern Warfare. Overwatch, Black Ops 3(same engine as MW), and recently Hyper Scape will will make use of those extra cores. I have a widget to monitor CPU utilization in the background and I can check Task Manager. If I had a better video card I'm positive it would've sucked down even more of those 12 cores (my GPU is running at 100% load according to MSI AB).
This is a huge deal and while I understand, I get it, it's hard to reliably reproduce the same results in a multiplayer environment because it changes so much and generally seen as taboo from a hardware benchmarking standpoint, it is vastly different then singleplayer workloads to the point at which it requires completely different recommendations. Given how many people are making expensive hardware choices specifically because they play multiplayer games, I would even say most tech reviews in this day and age are irrelevant for CPU recommendations outside of the casual single player gamer. GPU recommendations are still very much on par, CPU is not remotely.
I talk about this frequently on my stream and why I still recommended the 1600 AF even when it was sitting at $105-125, it's a steal if you play multiplayer games, while most people that either read benchmarking websites or run benchmarks themselves will start making a case for a 4c Intel. 6 core is a must at the very least in this day and age.
Anandtech it's time to tread new ground and go into the uncharted area. Singleplayer results and multiplayer results are too different, you can't keep spinning the wheel and expect things to remain the same. You can verify this yourself just by running task manager in the background while playing one of the games I mentioned at the lowest settings regardless of being able to repeat those results exactly you'll see it's definitely a multi-core landscape for newer multiplayer games.
Not even touched on in the article.
Bensam123 - Tuesday, July 21, 2020 - link
70%, I have SMT off for clarification.