Benchmarking Suite 2017

2017 CPU

For our Ryzen review, we are implementing our new CPU testing benchmark suite, fully scripted as of 2/17. This means that with a fresh OS install, we can configure the OS to be more consistent, install the new benchmarks, maintain version consistency without random updates and start running the tests in under 5 minutes. After that it's a one button press to start an 8-10hr test (with a high-performance core) with nearly 100 relevant data points in the benchmarks given below. The tests cover a wide range of segments, some of which will be familiar but some of the tests are new to benchmarking in general, but still highly relevant for the markets they come from.

Our new CPU tests go through six main areas. We cover the Web (we've got an un-updateable version of Chrome 56), general system tests (opening tricky PDFs, emulation, brain simulation, AI, 2D image to 3D model conversion), rendering (ray tracing, modeling), encoding (compression, AES, h264 and HEVC), office based tests (PCMark and others), and our legacy tests, throwbacks from another generation of bad code but interesting to compare.

A side note on OS preparation. As we're using Windows 10, there's a large opportunity for something to come in and disrupt our testing. So our default strategy is multiple: disable the ability to update as much as possible, disable Windows Defender, uninstall OneDrive, disable Cortana as much as possible, implement the high performance mode in the power options, and disable the internal platform clock which can drift away from being accurate if the base frequency drifts (and thus the timing ends up inaccurate).

Web on Chrome 56

Sunspider
Kraken
Octane
Web13
Web15

System

PDF Opening
FCAT
3DPM v21
Dolphin
DigiCortex
Civilization 6
Agisoft PS v1.0 

Rendering

Corona
Blender
LuxMark CPU C++
LuxMark CPU OpenCL
POV-Ray
CB15 ST
CB15 MT

Encoding

7-Zip
WinRAR
TrueCrypt
HandBrake 264-LQ
HandBrake 264-HQ
HandBrake 265-4K

Office

PCMark8 
SYSmark 2014 / SE

Legacy

3DPM v1 ST / MT
x264 HD 3 Pass 1, Pass 2
CB 11.5 ST / MT
CB 10 ST / MT

A side note - a couple of benchmarks (Dolphin, Civ 6) weren't fully 100% giving good data during testing. Need to go back and re-work this part of our testing.

2017 GPU

The bad news for our Ryzen review is that our new 2017 GPU testing stack not yet complete. We recieved our Ryzen CPU samples on February 21st, and tested in the hotel at the event for 6hr before flying back to Europe.

I spent two days back in London, where ~12 CPUs relevant to the review today were testing on our new CPU benchmarks. This was before I had to fly to Barcelona for Mobile World Congress, and I brought 30kg of kit with me to help with the review. I have had Ryzen set up in our shared flat for the past few days, and had Ryzen benchmarks running while attending meetings. As a result, our CPU data is good, but we lack any substantial GPU comparison data, power numbers (some idiot senior editor forgot his power meter...) or overclocking numbers. Based on a few Twitter polls conducted over at @IanCutress, people seemed more interested in CPU performance anyway, so we'll do a Pt 2 with more GPU data in the next couple of weeks.

 

 

AMD Stock Coolers and Memory: Wraith v2 and DDR4 Test Bed Setup and Hardware
Comments Locked

574 Comments

View All Comments

  • theuglyman0war - Saturday, March 4, 2017 - link

    I'd like to see a lot more older i7 extreme editions covered all the way to westmere so I can sell clients on new builds with such a comparison.
  • mapesdhs - Sunday, March 5, 2017 - link

    Which older i7s interest you specifically?
  • theuglyman0war - Saturday, March 4, 2017 - link

    Checking what I paid last month for i7-7700k at Microcenter...
    Although I did get the motherboard combo price sale they "usually" offer...
    The supposed $60 off for $319 is the cheapest price I found with a quick survey of new egg, amazon etc... And only $20 less then what I paid! Hardly A slashed priced answer shot across the bow by Intel! Not by a long shot!
    I thought I was going to recommend the new cheap price to all my customer's new builds but I am pushing RYZEN and AM4 for a real combined price that makes a difference. ( the cheap price for enthusiast Am4 is enticing but the loss of PCI lanes is of concern for extreme cpu comparison anyway. Not so much compared to i7-7700k though which brings the comparison back to 16 lane parity! )
  • theuglyman0war - Saturday, March 4, 2017 - link

    Could anyone actually point me to the amazing slashed deals that "BEAT" what I couldn't get last month by a long shot?

    ( which was $349 BEFORE rebate. In other words it's not like there were not sales last month as well. And I see nothing now that really amounts to AMAZING compared to last month? )

    Pretty dam insulting from somewhere in the pipe? Not sure if it's Intel. Or it's resellers clinging on to greedy margins not reflecting the savings to save their own ass's and bottom line due to stock considerations? Which iz no excuse considering the writing was on the wall. Someone needs to do a lot better. A heck of a lot better. Particularly considering I was thinking I could jes laff off AMD with an Intel savings and now have egg on my face! :)
  • rpns - Saturday, March 4, 2017 - link

    The 'Test Bed Setup' section could do with some more details. E.g. what BIOS version? Windows 10 build version? Any notable driver versions?

    These details aren't useful just now, but also when looking back at the review a few months down the line.
  • jorkevyn - Saturday, March 4, 2017 - link

    why they don't get 4 channel for DDR4 memory? I think, if you get that you will may be the real I7 6950K Killer
  • sedra - Saturday, March 4, 2017 - link

    have a look at this:
    "Many software programmers consider Intel's compiler the best optimizing compiler on the market, and it is often the preferred compiler for the most critical applications. Likewise, Intel is supplying a lot of highly optimized function libraries for many different technical and scientific applications. In many cases, there are no good alternatives to Intel's function libraries.

    Unfortunately, software compiled with the Intel compiler or the Intel function libraries has inferior performance on AMD and VIA processors. The reason is that the compiler or library can make multiple versions of a piece of code, each optimized for a certain processor and instruction set, for example SSE2, SSE3, etc. The system includes a function that detects which type of CPU it is running on and chooses the optimal code path for that CPU. This is called a CPU dispatcher. However, the Intel CPU dispatcher does not only check which instruction set is supported by the CPU, it also checks the vendor ID string. If the vendor string says "GenuineIntel" then it uses the optimal code path. If the CPU is not from Intel then, in most cases, it will run the slowest possible version of the code, even if the CPU is fully compatible with a better version."

    http://www.agner.org/optimize/blog/read.php?i=49&a...
  • HomeworldFound - Saturday, March 4, 2017 - link

    Everyone here already knew that ten years ago.
  • Notmyusualid - Sunday, March 5, 2017 - link

    Indeed it was.
  • sedra - Sunday, March 5, 2017 - link

    it is worth to bring it up now.

Log in

Don't have an account? Sign up now