Benchmarking Suite 2017

2017 CPU

For our Ryzen review, we are implementing our new CPU testing benchmark suite, fully scripted as of 2/17. This means that with a fresh OS install, we can configure the OS to be more consistent, install the new benchmarks, maintain version consistency without random updates and start running the tests in under 5 minutes. After that it's a one button press to start an 8-10hr test (with a high-performance core) with nearly 100 relevant data points in the benchmarks given below. The tests cover a wide range of segments, some of which will be familiar but some of the tests are new to benchmarking in general, but still highly relevant for the markets they come from.

Our new CPU tests go through six main areas. We cover the Web (we've got an un-updateable version of Chrome 56), general system tests (opening tricky PDFs, emulation, brain simulation, AI, 2D image to 3D model conversion), rendering (ray tracing, modeling), encoding (compression, AES, h264 and HEVC), office based tests (PCMark and others), and our legacy tests, throwbacks from another generation of bad code but interesting to compare.

A side note on OS preparation. As we're using Windows 10, there's a large opportunity for something to come in and disrupt our testing. So our default strategy is multiple: disable the ability to update as much as possible, disable Windows Defender, uninstall OneDrive, disable Cortana as much as possible, implement the high performance mode in the power options, and disable the internal platform clock which can drift away from being accurate if the base frequency drifts (and thus the timing ends up inaccurate).

Web on Chrome 56

Sunspider
Kraken
Octane
Web13
Web15

System

PDF Opening
FCAT
3DPM v21
Dolphin
DigiCortex
Civilization 6
Agisoft PS v1.0 

Rendering

Corona
Blender
LuxMark CPU C++
LuxMark CPU OpenCL
POV-Ray
CB15 ST
CB15 MT

Encoding

7-Zip
WinRAR
TrueCrypt
HandBrake 264-LQ
HandBrake 264-HQ
HandBrake 265-4K

Office

PCMark8 
SYSmark 2014 / SE

Legacy

3DPM v1 ST / MT
x264 HD 3 Pass 1, Pass 2
CB 11.5 ST / MT
CB 10 ST / MT

A side note - a couple of benchmarks (Dolphin, Civ 6) weren't fully 100% giving good data during testing. Need to go back and re-work this part of our testing.

2017 GPU

The bad news for our Ryzen review is that our new 2017 GPU testing stack not yet complete. We recieved our Ryzen CPU samples on February 21st, and tested in the hotel at the event for 6hr before flying back to Europe.

I spent two days back in London, where ~12 CPUs relevant to the review today were testing on our new CPU benchmarks. This was before I had to fly to Barcelona for Mobile World Congress, and I brought 30kg of kit with me to help with the review. I have had Ryzen set up in our shared flat for the past few days, and had Ryzen benchmarks running while attending meetings. As a result, our CPU data is good, but we lack any substantial GPU comparison data, power numbers (some idiot senior editor forgot his power meter...) or overclocking numbers. Based on a few Twitter polls conducted over at @IanCutress, people seemed more interested in CPU performance anyway, so we'll do a Pt 2 with more GPU data in the next couple of weeks.

 

 

AMD Stock Coolers and Memory: Wraith v2 and DDR4 Test Bed Setup and Hardware
Comments Locked

574 Comments

View All Comments

  • zangheiv - Thursday, March 2, 2017 - link

    Hard to believe how a company like intel that repeatedly and knowingly engaged in illegal acts and other tactics to monopolize the market and cheat the consumers into high-prices, can still have dumb happy consumers after Ryzen
  • lmcd - Thursday, March 2, 2017 - link

    Some people like 256-bit vector ops I guess :-/ who would've guessed?
  • Ratman6161 - Thursday, March 2, 2017 - link

    Have to agree. To me, the i7-7700K seems like the better bargain right now. Then again, I'm looking at a $329 I7-6700K motherboard and CPU bundle and the 7700K isn't really all that much of an upgrade from the 6700K. But in the final analysis, after all this reading, I'm still not seeing anything that makes me want to rush out and replace my trusty old i7-2600K.
  • Meteor2 - Friday, March 3, 2017 - link

    +1. Maybe, as Rarson says above, a 4C/8T Zen might clock fast enough to challenge the 7700K. But in the workloads run at home, the 1800X does not challenge the (cheaper) 7700K.

    HPC and data centre are completely different and here Zen looks like it has real promise.
  • Meteor2 - Friday, March 3, 2017 - link

    ...Sadly the R5s are clocked equally low.

    https://www.google.co.uk/amp/wccftech.com/amd-ryze...

    Limited by process, I guess.
  • Cooe - Sunday, February 28, 2021 - link

    Again. You're an absolute idiot for thinking that the only "workloads done at home" are 1080p gaming & browsing the web.... You are so out of touch with the desktop PC market, it's almost unbelievable. Here's hoping you were able to aquire some common sense over the past 4 years.
  • cmdrdredd - Saturday, March 4, 2017 - link

    " I'm still not seeing anything that makes me want to rush out and replace my trusty old i7-2600K."

    I agree with you. I have an overclocked 3570k and I don't see anything that makes me feel like it's too old. I'm mostly gaming on my system when I use it heavily, otherwise it's just general internet putzing around
  • Jimster480 - Thursday, March 2, 2017 - link

    Sorry but this is not the case.
    This is competing against Intel's HEDT line and not against the 7700k.

    2011v3 offers more PCI-E lanes only if you buy the top end CPU (which ofc isn't noted in most places) a cheaper chip like the 5820k for example only offers like 24 lanes TOTAL. Meaning that in price comparison there is no actual comparison.
  • Ratman6161 - Thursday, March 2, 2017 - link

    Well, whomever is trying to compete against, I7-7700K is about the top of the price range I am willing to spend. So Intel's 2011V3 lineup isn't in the cards for me either. AMD really isn't offering anything much for the mid range or regular desktop user either. In web browsing, office tasks, etc, their $499 CPU is often beaten by an i3. Now, the i3 is just as good as an i7-6900K too and in at least one test the i3 7350K is top of the charts. Why does this matter? Well, where does AMD go from here? If the i3 out performs the 1800x for office tasks, what will happen when they cut it to 4 cores to make a cheaper variant? Seems like they are set up for very expensive CPU's and for CPU's they have to sell for next to nothing. Where will their mid range come from?
  • silverblue - Thursday, March 2, 2017 - link

    Something tells me that if I decide to work on something complicated in Excel, that i3 isn't going to come anywhere near an R7. Besides, the 4- and 6-core variants may end up clocked higher, we don't know for sure yet.

Log in

Don't have an account? Sign up now