Benchmarking Suite 2017

2017 CPU

For our Ryzen review, we are implementing our new CPU testing benchmark suite, fully scripted as of 2/17. This means that with a fresh OS install, we can configure the OS to be more consistent, install the new benchmarks, maintain version consistency without random updates and start running the tests in under 5 minutes. After that it's a one button press to start an 8-10hr test (with a high-performance core) with nearly 100 relevant data points in the benchmarks given below. The tests cover a wide range of segments, some of which will be familiar but some of the tests are new to benchmarking in general, but still highly relevant for the markets they come from.

Our new CPU tests go through six main areas. We cover the Web (we've got an un-updateable version of Chrome 56), general system tests (opening tricky PDFs, emulation, brain simulation, AI, 2D image to 3D model conversion), rendering (ray tracing, modeling), encoding (compression, AES, h264 and HEVC), office based tests (PCMark and others), and our legacy tests, throwbacks from another generation of bad code but interesting to compare.

A side note on OS preparation. As we're using Windows 10, there's a large opportunity for something to come in and disrupt our testing. So our default strategy is multiple: disable the ability to update as much as possible, disable Windows Defender, uninstall OneDrive, disable Cortana as much as possible, implement the high performance mode in the power options, and disable the internal platform clock which can drift away from being accurate if the base frequency drifts (and thus the timing ends up inaccurate).

Web on Chrome 56

Sunspider
Kraken
Octane
Web13
Web15

System

PDF Opening
FCAT
3DPM v21
Dolphin
DigiCortex
Civilization 6
Agisoft PS v1.0 

Rendering

Corona
Blender
LuxMark CPU C++
LuxMark CPU OpenCL
POV-Ray
CB15 ST
CB15 MT

Encoding

7-Zip
WinRAR
TrueCrypt
HandBrake 264-LQ
HandBrake 264-HQ
HandBrake 265-4K

Office

PCMark8 
SYSmark 2014 / SE

Legacy

3DPM v1 ST / MT
x264 HD 3 Pass 1, Pass 2
CB 11.5 ST / MT
CB 10 ST / MT

A side note - a couple of benchmarks (Dolphin, Civ 6) weren't fully 100% giving good data during testing. Need to go back and re-work this part of our testing.

2017 GPU

The bad news for our Ryzen review is that our new 2017 GPU testing stack not yet complete. We recieved our Ryzen CPU samples on February 21st, and tested in the hotel at the event for 6hr before flying back to Europe.

I spent two days back in London, where ~12 CPUs relevant to the review today were testing on our new CPU benchmarks. This was before I had to fly to Barcelona for Mobile World Congress, and I brought 30kg of kit with me to help with the review. I have had Ryzen set up in our shared flat for the past few days, and had Ryzen benchmarks running while attending meetings. As a result, our CPU data is good, but we lack any substantial GPU comparison data, power numbers (some idiot senior editor forgot his power meter...) or overclocking numbers. Based on a few Twitter polls conducted over at @IanCutress, people seemed more interested in CPU performance anyway, so we'll do a Pt 2 with more GPU data in the next couple of weeks.

 

 

AMD Stock Coolers and Memory: Wraith v2 and DDR4 Test Bed Setup and Hardware
POST A COMMENT

551 Comments

View All Comments

  • Crono - Thursday, March 2, 2017 - link

    A Hero Has Ryzen Reply
  • Sweeprshill - Thursday, March 2, 2017 - link

    Lived up to the hype. Ryzen is a beast. Intel needs massive price cuts on their 2011-v3 chips. Well done AMD, best price/performance CPUs on the market and as fast or faster than Intel performance. Reply
  • sans - Thursday, March 2, 2017 - link

    Hey, what you have found which features improving on AMD's crap has been found in Intel's products for years. Reply
  • Nem35 - Thursday, March 2, 2017 - link

    Yeah, and it's beating the Intel. Funny, right? Reply
  • Sweeprshill - Thursday, March 2, 2017 - link

    Yeah these new AMD chips are monsters. Wondering how large the price cuts are that Intel will bring to their 2011-v3 chips to compete. Reply
  • czerro - Friday, March 3, 2017 - link

    Intel already slashed prices pretty drastically 4 days ago, to kinda deflate Ryzen's release. Before price cuts, Ryzen had a huge price and performance advantage at all metrics, and Intel would have looked ridiculous.

    I can't believe people aren't reporting the price-cutting right before Ryzen release more. Intel only did it to save face on graphs and confuse people. Ryzen definitely had Intel by the balls a week ago before the price cuts.

    It's great that we all have options now, but this really smeared Ryzen's release in a cheap way that anybody can point out all those Intel chips were 100-200 dollars more expensive less than a WEEK ago.
    Reply
  • SodaAnt - Saturday, March 4, 2017 - link

    No, Intel hasn't slashed prices. There was a sale at microcenter a few days back, but there's no across the board official price cut on Intel chips. Reply
  • Notmyusualid - Monday, March 6, 2017 - link

    @ SodaAnt

    Agreed, I see no Intel price drops either.
    Reply
  • Notmyusualid - Friday, March 3, 2017 - link

    @ Nem35

    Incomplete review.

    After seeing a gaming-focused review, I'd say the AMD procs are just OK. I welcome AMD is back with a fighting chance, but about half my purchase choice will be game-inspired.

    Quote:

    "For gaming, it’s a hard pass. We absolutely do not recommend the 1800X for gaming-focused users or builds, given i5-level performance at two times the price."

    I'm not a 'fanboi', as I'd have no trouble fitting a 1700X in a build I wouldn't game in. But otherwise, like another reviewer said, its a hard pass.
    Reply
  • Alexvrb - Saturday, March 4, 2017 - link

    For gaming builds the upcoming Ryzen 5 and 3 series will offer a lot more bang for your buck and will compete much more aggressively. However, the Ryzen 7 still offers decent gaming performance and excellent performance everywhere else. The gobs of cores may come in handy in the future too, even in games - as more threads will be available on more rigs, devs will take notice. This year AMD is definitely lowering the pricing for 8-16 thread processors, clearing a path for the future of gaming.

    With that being said I still think that when strictly considering gaming, their Ryzen 3/5 quadcore models will be a far better value, especially as current-gen games aren't often built in such a way that they can take advantage of the Ryzen 7.
    Reply

Log in

Don't have an account? Sign up now