Benchmark Overview

2017 CPU

For our review, we are implementing our fresh CPU testing benchmark suite, using new scripts developed specifically for this testing. This means that with a fresh OS install, we can configure the OS to be more consistent, install the new benchmarks, maintain software version consistency without random updates and start running the tests in under 5 minutes. After that it's a one button press to start an 8-10hr test (with a high-performance core) with nearly 100 relevant data points in the benchmarks given below. The tests cover a wide range of segments, some of which will be familiar but some of the tests are new to benchmarking in general, but still highly relevant for the markets they come from.

Our new CPU tests go through six main areas. We cover the Web (we've got an un-updateable version of Chrome 56), general system tests (opening tricky PDFs, emulation, brain simulation, AI, 2D image to 3D model conversion), rendering (ray tracing, modeling), encoding (compression, AES, h264 and HEVC), office based tests (PCMark and others), and our legacy tests, throwbacks from another generation of bad code but interesting to compare.

All of our benchmark results can also be found in our benchmark engine, Bench.

A side note on OS preparation. As we're using Windows 10, there's a large opportunity for something to come in and disrupt our testing. So our default strategy is multiple: disable the ability to update as much as possible, disable Windows Defender, uninstall OneDrive, disable Cortana as much as possible, implement the high performance mode in the power options, and disable the internal platform clock which can drift away from being accurate if the base frequency drifts (and thus the timing ends up inaccurate).

New Tests

SpecWPC v2.1 – A lot of the industry turn to SPEC to produce standard benchmarks suitable for various markets. The latest version of the workstation focused benchmark suite was released this year, and tackles six main areas with over 30 different benchmarks. These include compute, visualization, medical, oil and gas, finance, and typical workstation areas. For consistency we run all the tests (except IOMeter) on Windows 10, using an RX 460 graphics card at 1080p resolution with an MX200 SSD.

PCMark10 – We had several requests to include PCMark10 in our new testing suite. Normally we wait until a new benchmark has most of the problems ironed out, however our initial performance scaling metrics show that PCMark10 is basically there already. The extended suite covers ‘Essential, Productivity and Creativity’ benchmarks such as GIMP, Blender, video editing, conferencing, complex spreadsheets and other tests. We use the subtest values as well as the gaming physics result.

Agisoft PhotoScan 1.3.3 – Again, requests to use a more updated version of Photoscan were also coming through the inbox. Over the older version, Photoscan includes various throughput enhancements to each of the core points of the algorithm. Agisoft also gave us a new larger set of more detailed test images to generate our 3D models, giving a longer benchmark (but results are not comparable to the old data). We’ve run this benchmark on about a dozen CPUs ready for this review.

Office / Professional Tests

PCMark8
Chromium Compile (v56)

Rendering Tests

Corona 1.3
Blender 2.78
LuxMark v3.1 CPU C++
LuxMark v3.1 CPU OpenCL
POV-Ray 3.7.1b4
Cinebench R15 ST
Cinebench R15 MT

Encoding Tests

7-Zip 9.2
WinRAR 5.40
AES Encoding (TrueCrypt 7.2)
HandBrake v1.0.2 x264 LQ
HandBrake v1.0.2 x264-HQ
HandBrake v1.0.2 HEVC-4K

System Tests

PDF Opening
FCAT
3DPM v2.1
Dolphin v5.0
DigiCortex v1.20
Agisoft PhotoScan v1.0

Legacy Tests

3DPM v1 ST / MT
x264 HD 3 Pass 1, Pass 2
Cinebench R11.5 ST / MT
Cinebench R10 ST / MT

A Note on CPU Gaming Tests (Repeat from Page 1)

I know a lot of our readers are gamers, and are interested in seeing how well (or poorly) these massive multi-core chips perform in the latest titles at the highest resolutions. Apologies to disappoint, but I am going to tackle the more traditional consumer tasks in a second review, and which will mean that gaming will be left for that review. For the users that have followed my reviews (and Twitter) of late, I am still having substantial issues with my X299 test beds on the gaming results, with Skylake-X massively underperforming where I would expect a much higher result.

After having to dedicate recent time to business trips (Hot Chips, IFA) as well as other releases (Threadripper), I managed to sit down in the two weeks between trips to figure what exactly what was going on. I ended up throwing out the two X299 pre-launch engineering samples I was using for the Skylake-X testing, and I received a new retail motherboard only a few days before this review.  This still has some issues that I spent time trying to debug, which I think are related to how turbo is implemented, which could either be Intel related or BIOS specific.

To cause insult to injury to everyone who wants to see this data, I have jumped on a plane to travel half-way around the world for a business trip during the week of this launch, which leaves the current results inconclusive. I have reached out to the two other motherboard vendors that I haven’t received boards from; just in case the issue I seem to be having is vendor specific. If I ever find out what this issue is, then I will write it up, along with a full Skylake-X gaming suite. It will have to wait to mid-late October, due to other content (and more pre-booked event travel).

Test Bed and Setup Benchmarking Performance: SPECwpc v2.1
Comments Locked

152 Comments

View All Comments

  • extide - Monday, September 25, 2017 - link

    No, TDP should include Turbo as that is part of the base/stock operation mode of the CPU.
  • IGTrading - Monday, September 25, 2017 - link

    TDP = Total Design Power by definition.

    This is used to design the motherboard and the cooling system to give designers a clear limit over which the system doesn't go unless it is purposely overcloked.

    Wikipedia : "The thermal design power (TDP), sometimes called thermal design point, is the maximum amount of heat generated by a computer chip or component (often the CPU or GPU) that the cooling system in a computer is designed to dissipate under any workload."

    Intel : "TDP (Thermal Design Power) Intel defines TDP as follows: The upper point of the thermal profile consists of the Thermal Design

    Power (TDP) and the associated Tcase value. Thermal Design Power (TDP) should be used for
    processor thermal solution design targets. TDP is not the maximum power that the processor can
    dissipate. TDP is measured at maximum TCASE.1"

    Intel : "Due to normal manufacturing variations, the exact thermal characteristics of each individual processor are unique. Within the specified parameters of the part, some processors may operate at a slightly higher or lower voltage, some may dissipate slightly higher or lower power and some may draw slightly higher or lower current. As such, no two parts have identical power and thermal characteristics.

    However the TDP specifications represent a “will not exceed” value. "

    This is what we've understood by TDP in the past 21 years while in IT hardware industry.

    If you have a different definition, then perhaps we're talking about different things.
  • whatevs - Monday, September 25, 2017 - link

    Specification for 7980xe says "Thermal Design Power (TDP) represents the average power, in watts, the processor dissipates when operating at Base Frequency with all cores active under an Intel-defined, high-complexity workload. Refer to Datasheet for thermal solution requirements."
    There's a different specification for electrical design. This is not your ancient Xeon TDP.
  • IGTrading - Monday, September 25, 2017 - link

    You mean the definition of TDP should change every year to suit Intel's marketing ?! :)

    "Ancient" Xeon TDP ?! :)

    I've quoted Intel's own definition.

    If the company just came up with a NEW and DIFFERENT definition just for the Core i9 series, then that's just plain deceiving marketing, changing with the wind (read : new generation of products) .

    Plus, why the heck are they calling it TDP ?!

    If they now claim that TDP "represents the average power, in watts, the processor dissipates when operating at Base Frequency with all cores active " then they basically use AMD's ACP from 2011.

    What a load of nonsense from Intel ...

    https://www.intel.com/content/dam/doc/white-paper/...
  • whatevs - Monday, September 25, 2017 - link

    You have quoted 6 year old Xeon definition, different products have different operating conditions, deal with it.
  • Spunjji - Monday, September 25, 2017 - link

    Your name suggests that you're kind of a dick and your comments confirm it. Your point is weak and doesn't at all do the work you think it does.
  • whatevs - Monday, September 25, 2017 - link

    You may be unhappy with what Intel promised you, but to claim that you could burn a system with increased power usage from turbo clocks is ridiculous, thermal throttling is not fire, and it's ridiculous to argue on a cpu that can run overclocked at >400w power consumption.
  • Notmyusualid - Monday, September 25, 2017 - link

    +1
  • wolfemane - Tuesday, September 26, 2017 - link

    You can't talk rationale with a loyalist sympathizer. TDP is a set definition in the industry and one Intel seems to be misleading about with their Extreme HEDT CPU. That seems to be a fact clearly made among almost all reviews of the 7980xe.

    I think I read a few articles yesterday talking about how the 7980xe was having major issues and wasn't boosting correctly but showing high power draw. But yesterday was a long time ago and I cant remember where I read that.
  • someonesomewherelse - Saturday, October 14, 2017 - link

    So why not call it 'Average Design Power - ADP'?

Log in

Don't have an account? Sign up now