Benchmarking Suite 2017

2017 CPU

For our Ryzen review, we are implementing our new CPU testing benchmark suite, fully scripted as of 2/17. This means that with a fresh OS install, we can configure the OS to be more consistent, install the new benchmarks, maintain version consistency without random updates and start running the tests in under 5 minutes. After that it's a one button press to start an 8-10hr test (with a high-performance core) with nearly 100 relevant data points in the benchmarks given below. The tests cover a wide range of segments, some of which will be familiar but some of the tests are new to benchmarking in general, but still highly relevant for the markets they come from.

Our new CPU tests go through six main areas. We cover the Web (we've got an un-updateable version of Chrome 56), general system tests (opening tricky PDFs, emulation, brain simulation, AI, 2D image to 3D model conversion), rendering (ray tracing, modeling), encoding (compression, AES, h264 and HEVC), office based tests (PCMark and others), and our legacy tests, throwbacks from another generation of bad code but interesting to compare.

A side note on OS preparation. As we're using Windows 10, there's a large opportunity for something to come in and disrupt our testing. So our default strategy is multiple: disable the ability to update as much as possible, disable Windows Defender, uninstall OneDrive, disable Cortana as much as possible, implement the high performance mode in the power options, and disable the internal platform clock which can drift away from being accurate if the base frequency drifts (and thus the timing ends up inaccurate).

Web on Chrome 56

Sunspider
Kraken
Octane
Web13
Web15

System

PDF Opening
FCAT
3DPM v21
Dolphin
DigiCortex
Civilization 6
Agisoft PS v1.0 

Rendering

Corona
Blender
LuxMark CPU C++
LuxMark CPU OpenCL
POV-Ray
CB15 ST
CB15 MT

Encoding

7-Zip
WinRAR
TrueCrypt
HandBrake 264-LQ
HandBrake 264-HQ
HandBrake 265-4K

Office

PCMark8 
SYSmark 2014 / SE

Legacy

3DPM v1 ST / MT
x264 HD 3 Pass 1, Pass 2
CB 11.5 ST / MT
CB 10 ST / MT

A side note - a couple of benchmarks (Dolphin, Civ 6) weren't fully 100% giving good data during testing. Need to go back and re-work this part of our testing.

2017 GPU

The bad news for our Ryzen review is that our new 2017 GPU testing stack not yet complete. We recieved our Ryzen CPU samples on February 21st, and tested in the hotel at the event for 6hr before flying back to Europe.

I spent two days back in London, where ~12 CPUs relevant to the review today were testing on our new CPU benchmarks. This was before I had to fly to Barcelona for Mobile World Congress, and I brought 30kg of kit with me to help with the review. I have had Ryzen set up in our shared flat for the past few days, and had Ryzen benchmarks running while attending meetings. As a result, our CPU data is good, but we lack any substantial GPU comparison data, power numbers (some idiot senior editor forgot his power meter...) or overclocking numbers. Based on a few Twitter polls conducted over at @IanCutress, people seemed more interested in CPU performance anyway, so we'll do a Pt 2 with more GPU data in the next couple of weeks.

 

 

AMD Stock Coolers and Memory: Wraith v2 and DDR4 Test Bed Setup and Hardware
Comments Locked

574 Comments

View All Comments

  • BurntMyBacon - Friday, March 3, 2017 - link

    @Gothmoth: "gamer... as if the world is only full with idiotic people who waste their lives playing shooter or RPG´s."

    PC Gaming happens to be one of the few growing areas in the PC market. Not everyone games, but for those that do, the 7700K is still worth considering. Dropping $500 on the 1800X may not be the best call for those that don't take advantage of the parallelism. Of course, the 1800X wasn't really meant for people who can't take advantage of the parallelism. AMD will have lower cost narrower processors to address that gap. I'm curious as to how the performance/price equation will stand once AMD releases their upper end 6c/12t and 4c/8t processors.
  • Beany2013 - Friday, March 3, 2017 - link

    Sod the 1800X - I need a new VM server, and if I want all the threads (sixteen), I can either drop £450 on a Xeon E5 2620 at 2.1-3ghz (cheapest Intel 16 thread option I can find), or I can spend £100 less, and get a Ryzen 7 1700 (3.0-3.7ghz) and put that extra money towards more RAM so I can run more VMs and get more work done.

    For those of us who aren't high end gamers - which is basically almost everyone, and a far more significant market - these chips may well give Intel a bloody nose in the workstation space; AMD have confirmed they'll use ECC RAM quite happily.

    Photographers, videographers, CAD-CAM, developers etc are a bigger market in terms of raw units than high end gamers, and these chips look like being a pretty compelling option as it stands.

    Steven R
  • Beany2013 - Friday, March 3, 2017 - link

    (VM server for home, I should have noted - for work, I'll see how the Ryzen based opterons and supermicro mobos etc pan out - money is important in these factors, but I'm not a moron, and I'm not going to run production gear on gaming hardware, natch....)
  • BurntMyBacon - Friday, March 3, 2017 - link

    @Beany2013: "I need a new VM server, and if I want all the threads (sixteen), I can either drop £450 on a Xeon E5 2620 at 2.1-3ghz (cheapest Intel 16 thread option I can find), or I can spend £100 less, and get a Ryzen 7 1700 (3.0-3.7ghz) and put that extra money towards more RAM so I can run more VMs and get more work done."

    It is clear by this statement that you fall into the category of people that can take advantage of the parallelism. Therefore, my statement doesn't apply to your presented in the slightest.

    I don't disagree that the Ryzen 7 series has a lot to offer to a lot of people (myself included). If I were in the market today, I'd be looking long and hard at an R7 1700X. The minor drop in gaming performance is less significant to me than the increase in performance for many other tasks I use my computer for. I do a little bit of dabbling in a lot of different things (most of which benefit from high thread count). I have noticed that for the set of applications I have open simultaneously and the tasks I have running, my computer is more responsive with more cores or threads, but single threaded performance is still important to the individual tasks.
    In my workflow: (i3 < i5/FX-8xxx < i7 <? R7)

    My point was that there is in fact a not so insignificant market of people putting computers together for the primary purpose of gaming. This market appears, by all metrics, to be growing. For this market, Intel's i7-7700K or better yet i5-7600K are still viable options that provide better performance/price than AMD's current options. I'll repeat: "AMD will have lower cost narrower processors to address that gap. I'm curious as to how the performance/price equation will stand once AMD releases their upper end 6c/12t and 4c/8t processors."
  • Cooe - Sunday, February 28, 2021 - link

    "or better yet i5-7600K"
    Arguably the most short-sited statement in this entire comments section lol. The 4c/4t i5's had roughly equal gaming performance to Ryzen at launch but with ZERO headroom left for the future. This is why the i5-7600K gets absolutely freaking ROFLSTOMPED by the R5 1600 in modern titles/game engines.
  • JMB1897 - Friday, March 3, 2017 - link

    Compelling, but I don't think it's totally there yet. I'd be worried about the memory issues. Increased latency as you add more DIMMs and dual vs quad channel. I'd spend that extra 100 on a Xeon personally.
  • Sttm - Friday, March 3, 2017 - link

    Thats who buys off the shelf CPUs thats cost $$$, Gamers. Thats who AMD needs to please with their product. GAMERS. Thats why AMD's stock has been tanking since Ryzen reviews went up, because GAMERS are the demographic that matters when it comes to performance CPU sales.
  • deltaFx2 - Saturday, March 4, 2017 - link

    @Sttm: You have an inflated opinion of the impact of gamers. No, AMD's stock isn't tanking because of gamers. I suggest you also look at Nvidia's stock, which is well down from its high of ~120, to ~98. Wed-Friday, Nvidia dropped from 105 to 98, and it dipped below that to ~96 at one point. That's roughly 7-8%. The two stocks are often correlated on drops, with AMD amplifying nvidia's drop. Both do GPUs, see? Some people make tonnes of money shorting AMD (and in recent times have lost their shirt doing so).

    Here's the truth: All Desktop, as per Lisa Su, is a 5 bn TAM market and gaming is part of this (let's say 50%). Nothing to scoff at, sure, but compared to laptop and server, it's a rounding error. There's NOTHING in these tests/reviews to suggest that AMD will suck in those markets; in fact, quite the opposite: power looks good, perf looks good. AMD's stock (long term) won't tank on the whims of gamers. They help get the mindshare, which is the only reason they're worth catering to (they tend to be a vocal, passionate, and sometimes irrational lot. You won't see datacenter gurus doing the stuff that gamers do. They certainly won't shoot each other over whose GPU is the best).
  • cmdrdredd - Saturday, March 4, 2017 - link

    Believe it or not there are millions of people worldwide who pretty much use their PC for two things. The internet (web browsing, email etc) and gaming. You don't need 16 threads to check email and read forums either so gaming performance is going to be critical. It's not just the CPU performance, it's the entire platform that contributes to Gaming related performance.
  • sans - Thursday, March 2, 2017 - link

    Yeah, stick with Intel because Intel is the standard and its products are the best for each respective market. AMD is a total failure.

Log in

Don't have an account? Sign up now