Launching the #CPUOverload Project: Testing Every x86 Desktop Processor since 2010
by Dr. Ian Cutress on July 20, 2020 1:30 PM ESTGaming Tests: Civilization 6
Originally penned by Sid Meier and his team, the Civilization series of turn-based strategy games are a cult classic, and many an excuse for an all-nighter trying to get Gandhi to declare war on you due to an integer underflow. Truth be told I never actually played the first version, but I have played every edition from the second to the sixth, including the fourth as voiced by the late Leonard Nimoy, and it a game that is easy to pick up, but hard to master.
Benchmarking Civilization has always been somewhat of an oxymoron – for a turn based strategy game, the frame rate is not necessarily the important thing here and even in the right mood, something as low as 5 frames per second can be enough. With Civilization 6 however, Firaxis went hardcore on visual fidelity, trying to pull you into the game. As a result, Civilization can taxing on graphics and CPUs as we crank up the details, especially in DirectX 12.
For this benchmark, we are using the following settings:
- 480p Low
- 1440p Low
- 4K Low
- 1080p Max
For automation, Firaxis supports the in-game automated benchmark from the command line, and output a results file with frame times. We call the game, wait until the file is created, pull it out and regex the relevant data. For the in-game settings, because Civ 6 seems to require the hardware from the last run to be the same as the current run else it defaults to base settings, we delete the settings file, run the benchmark once, and then use regex on the generated settings files to call the relevant resolutions and quality settings. We do as many runs within 10 minutes per resolution/setting combination, and then take averages and percentiles.
| AnandTech | IGP | Low | Medium | High |
| Average FPS |  |
 |
 |
 |
| 95th Percentile |  |
 |
 |
 |
All of our benchmark results can also be found in our benchmark engine, Bench.
Facebook
Twitter
RSS
110 Comments
View All Comments
ruthan - Monday, July 27, 2020 - link
Well lots of bla, bla, bla.. I checked graphs in archizlr they are classic just few entries.. there is link to your benchmark database, but here i see preselected some Crysis benchmark, which is not part of article.. and dont lead to some ultimate lots of cpus graphs. So it need much more streamlining.i usually using old Geekbench for cpus tests and there i can compare usually what i want.. well not with real applications and games, but its quick too. Otherwise usually have enough knowledge to know if is some cpu good enough for some games or not.. so i dont need some very old and very need comparisions. Something can be found at Phoronix.
These benchmarks will always lots relevancy with new updates, unless all cpus would in own machines and update and running and reresting constantly - which could be quite waste of power and money.
Maybe some golden path is some simple multithreaded testing utility with 2 benchmarks one for integers and one for floats.
Ian Cutress - Wednesday, August 5, 2020 - link
When you're in Bench, Check the drop down menu on your left for the individual testshnlog - Wednesday, July 29, 2020 - link
> For our testing on the 2020 suite, we have secured three RTX 2080 Ti GPUs direct from NVIDIA.Congrats!
Koenig168 - Saturday, August 1, 2020 - link
It would be more efficient to focus on the more popular CPUs. Some of the less popular SKUs which differ only by clock speed can have their performance extrapolated. Testing 900 CPUs sound nice but quickly hit diminishing returns in terms of usefulness after the first few hundred.You might also wish to set some minimum performance standards using just a few tests. Any CPU which failed to meet those standards should be marked as "obsolete, upgrade already dude!" and be done with them rather than spend the full 30 to 40 hours testing each of them.
Finally, you need to ask yourself "How often do I wish to redo this project and how much resources will I be able to devote to it?" Bearing in mind that with new drivers, games etc, the database needs to be updated oeriodically to stay relevant. This will provide a realistic estimate of how many CPUs to include in the database.
Meteor2 - Monday, August 3, 2020 - link
I think it's a labour of love...TrevorX - Thursday, September 3, 2020 - link
My suggestion would be to bench the highest performing Xeons that supported DDR3 RAM. Why? Because the cost of DDR3 RDIMMs is so amazingly cheap (as in, less than 10%) compared with DDR4. I personally have a Xeon E5-1660v2 @4.1GHz with 128GB DDR3 1866MHz RDIMMs that's the most rock stable PC I've ever had. Moving up to a DDR4 system with similar memory capacity would be eye-wateringly expensive. I currently have 466 tabs open in Chrome, Outlook, Photoshop, Word, several Excel spreadsheets, and I'm only using 31.3% of physical RAM. I don't game, so I would be genuinely interested in what actual benefit would be derived from an upgrade to Ryzen / Threadripper.Also very keen to see server/hypervisor testing of something like Xeon E5-2667v2 vs Xeon W-1270P or Xeon Silver 4215R for evaluation of on-prem virtualisation hosts. A lot of server workloads are being shifted to the cloud for very good reasons, but for smaller businesses it might be difficult to justify the monthly expense of cloud hosting (and Azure licensing) when they still have a perfectly serviceable 5yo server with plenty of legs left on it. It would be great to be able to see what performance and efficiency improvements can be had jumping between generations.
Tilmitt - Thursday, October 8, 2020 - link
When is this going to be done?Mil0 - Friday, October 16, 2020 - link
Well they launched with 12 results if I count correctly, and currently there are 38 listed, that's close to 10/month. With the goal of 900, that would mean over 7 years (in which ofc more CPUs would be released)Mil0 - Friday, October 16, 2020 - link
Well they launched with 12 results if I count correctly, and currently there are 44 listed, that's about a dozen a month. With the goal of 900, that would mean 6 years (in which ofc more CPUs would be released)Mil0 - Friday, October 16, 2020 - link
Caching hid my previous comment from me, so instead of a follow up there are now 2 pretty similar ones. However, in the mean time I found Ian is actually updating on twitter, which you can find here: https://twitter.com/IanCutress/status/131350328982...He actually did 36 CPU's in 2.5 months, so it should only take 5 years! :D