Benchmarking Performance: CPU System Tests

Our first set of tests is our general system tests. These set of tests are meant to emulate more about what people usually do on a system, like opening large files or processing small stacks of data. This is a bit different to our office testing, which uses more industry standard benchmarks, and a few of the benchmarks here are relatively new and different.

All of our benchmark results can also be found in our benchmark engine, Bench.

PDF Opening

First up is a self-penned test using a monstrous PDF we once received in advance of attending an event. While the PDF was only a single page, it had so many high-quality layers embedded it was taking north of 15 seconds to open and to gain control on the mid-range notebook I was using at the time. This put it as a great candidate for our 'let's open an obnoxious PDF' test. Here we use Adobe Reader DC, and disable all the update functionality within. The benchmark sets the screen to 1080p, opens the PDF to in fit-to-screen mode, and measures the time from sending the command to open the PDF until it is fully displayed and the user can take control of the software again. The test is repeated ten times, and the average time taken. Results are in milliseconds.

System: PDF Opening with Adobe Reader DC

PDF opening is all about single thread frequency and IPC, giving the win to the new KBL-X chips.

FCAT Processing: link

One of the more interesting workloads that has crossed our desks in recent quarters is FCAT - the tool we use to measure stuttering in gaming due to dropped or runt frames. The FCAT process requires enabling a color-based overlay onto a game, recording the gameplay, and then parsing the video file through the analysis software. The software is mostly single-threaded, however because the video is basically in a raw format, the file size is large and requires moving a lot of data around. For our test, we take a 90-second clip of the Rise of the Tomb Raider benchmark running on a GTX 980 Ti at 1440p, which comes in around 21 GB, and measure the time it takes to process through the visual analysis tool.

System: FCAT Processing ROTR 1440p GTX1080 Data

FCAT similarly favors frequency and IPC. For this sort of workload, the Core i7 is the best chip to get.

3D Movement Algorithm Test v2.1: link

This is the latest version of the self-penned 3DPM benchmark. The goal of 3DPM is to simulate semi-optimized scientific algorithms taken directly from my doctorate thesis. Version 2.1 improves over 2.0 by passing the main particle structs by reference rather than by value, and decreasing the amount of double->float->double recasts the compiler was adding in. It affords a ~25% speed-up over v2.0, which means new data.

System: 3D Particle Movement v2.1

As 3DPM expands into several threads, the new quad-core parts will easily get trounced here by AMD's 8-cores for the same price. The Core i7-7800X puts on a good showing, as per core Intel's chips give a higher score.

DigiCortex v1.20: link

Despite being a couple of years old, the DigiCortex software is a pet project for the visualization of neuron and synapse activity in the brain. The software comes with a variety of benchmark modes, and we take the small benchmark which runs a 32k neuron/1.8B synapse simulation. The results on the output are given as a fraction of whether the system can simulate in real-time, so anything above a value of one is suitable for real-time work. The benchmark offers a 'no firing synapse' mode, which in essence detects DRAM and bus speed, however we take the firing mode which adds CPU work with every firing.

System: DigiCortex 1.20 (32k Neuron, 1.8B Synapse)

DigiCortex likes a bit of everything: cores, threads, IPC, frequency, uncore frequency, and memory frequency. The Core i7 parts roughly double the Core i5s due to the thread count, and also the AMD Ryzen parts skip ahead as well due to having double the threads to the Core i7.

Agisoft Photoscan 1.0: link

Photoscan stays in our benchmark suite from the previous version, however now we are running on Windows 10 so features such as Speed Shift on the latest processors come into play. The concept of Photoscan is translating many 2D images into a 3D model - so the more detailed the images, and the more you have, the better the model. The algorithm has four stages, some single threaded and some multi-threaded, along with some cache/memory dependency in there as well. For some of the more variable threaded workload, features such as Speed Shift and XFR will be able to take advantage of CPU stalls or downtime, giving sizeable speedups on newer microarchitectures.

System: Agisoft Photoscan 1.0 Total Time

Agisoft is like a Formula 1 race circuit: the long fast straights and techical corners make it a nightmare to have the technology to be the best at both, and Photoscan has enough serial code for high single thread performance to take advantage but also massively parallel sections where having 12-18 threads makes a difference.  Despite having half the threads, the single core performance of the Core i7-7740X makes it pull ahead of the Ryzen 7 chips, but when comparing the four threads of the Core i5-7640X to the twelve threads of the Ryzen 5 processors, having 12 threads wins.

Benchmark Overview Benchmarking Performance: CPU Rendering Tests
Comments Locked

176 Comments

View All Comments

  • Santoval - Tuesday, July 25, 2017 - link

    That is not how IPC works, since it explicitly refers to single core - single thread performance. As the number of cores rises the performance of a *single* task never scales linearly because there is always some single thread code involved (Amdahl's law). For example if your task has 90% parallel and 10% serial code its performance will max out at x10 that of a single core at ~512 cores. From then on even if you had a CPU with infinite cores you couldn't extract half an ounce of additional performance. If your code was 95% parallel the performance of your task would plateau at x20. For that though you would need ~2048 cores. And so on.

    Of course Amdahl's law does not provide a complete picture. It assumes, for example, that your task and its code will remain fixed no matter how many cores you add on them. And it disregards the possibility of computing distinct tasks in parallel on separate cores. That's where Gustafson's Law comes in. This "law" is not concerned with speeding up the performance of tasks but computing larger and more complex tasks at the same amount of time.

    An example given in Wikipedia involves boot times : Amdahl's law states that you can speed up the boot process, assuming it can be made largely parallel, up to a certain number of cores. Beyond that -when you become limited by the serial code of your bootloader- adding more cores does not help. Gustafson's law, on the contrary, states that instead of speeding up the boot process by adding more cores and computing resources, you could add colorful GUIs, increase the resolution etc, while keeping the boot time largely the same. This idea could be applied to many -but not all- computing tasks, for example ray tracing (for more photorealistic renderings) and video encoding (for smaller files or videos with better quality), and many other heavily multi-threaded tasks.
  • Rickyxds - Monday, July 24, 2017 - link

    I just agree XD.
  • Diji1 - Wednesday, July 26, 2017 - link

    "Overall speed increase 240%."

    LMAO. Ridiculous.
  • Alistair - Wednesday, July 26, 2017 - link

    No reason to laugh. I compared the 6600k vs the Ryzen 1700. 1 year speed increase of 144 percent (2.44 times the speed). Same as this: 1135 vs 466 points.

    http://cpu.userbenchmark.com/Compare/Intel-Core-i5...
  • Dr. Swag - Tuesday, July 25, 2017 - link

    I disagree, best value is 1600 as it oces as well as 1600x, comes with a decent stock cooler, and is cheaper.
  • vext - Monday, July 24, 2017 - link

    Interesting article but it seems intended to play down the extremely bad press x299 has received which is all over the internet and Youtube.

    Once you get past Mr. Cuttress' glowing review, it's clear that the I5-7640x is not worth the money because of lackluster performance, the I7-7740X is marginally faster than the older 7700k, and the I7-7800x is regularly beaten by the 7740X in many benchmarks that actually count and is a monstrously inefficient energy pig. Therefore the only Intel CPUs of this batch worth buying are the 7700k/7740x, and there is no real advantage to x299. In summary, it doesn't actually change anything.

    It's very telling that Mr. Cutress doesn't comment on the absolutely egregious energy consumption of the 7800x. The Test Bed setup section doesn't list the 7800x at all. The 7840x and 7740x are using a Thermalright True Copper (great choice!) but no info on the 7800x cooler. Essentially, the 7800x cameo appearance is only to challenge the extremely strong Ryzen multi-threaded results, but its negative aspects are not discussed, perhaps because it might frighten people from x299. Tsk, tsk. As my 11 year old daughter would say "No Fair." By the way, the 7800x is selling for ~ $1060 right now on Newegg, not $389.

    Proudly typed on my Ryzen 1800x/Gigabyte AB350 Gaming 3. # ;-)
  • Ian Cutress - Monday, July 24, 2017 - link

    You may not have realised but this is the Kaby Lake-X review, so it focuses on the KBL-X parts. We already have a Skylake-X review for you to mull over. There are links on the first page.
  • mapesdhs - Monday, July 24, 2017 - link

    Nevertheless, the wider picture is relevant here. The X299 platform is a mess. Intel is aiming KL-X at a market which doesn't exist, they've locked out features that actually make it useful, it's more power hungry, and a consumer needs a lot of patience and plenty of coffee to work out what the heck works and what doesn't on a mbd with a KL-X fitted.

    This is *exactly* the sort of criticism of Intel which should have been much stronger in the tech journalism space when Intel started pulling these sorts of stunts back with the core-crippled 3930K, heat-crazy IB and PCIe-crippled 5820K. Instead, except for a few exceptions, the tech world has been way too forgiving of Intel's treading-on-water attitude ever since SB, and now they've panicked in response to Ryzen and released a total hodgebodge of a chipset and CPU lineup which makes no sense at all. And if you get any disagreement about what I've said by anyone at Intel, just wave a 4820K in their face and say well explain this then (quad-core chip with 40 PCIe lanes, da daa!).

    I've been a big fan of Z68 and X79, but nothing about Intel's current lineup appeals in the slightest.
  • serendip - Tuesday, July 25, 2017 - link

    There's also the funny bit about motherboards potentially killing KBL-X CPUs if a Skylake-X was used previously.

    What's with Intel's insane product segmentation strategy with all the crippling and inconsistent motherboard choices? It's like they want to make it hard to choose, so buyers either get the cheapest or most expensive chip.
  • Haawser - Tuesday, July 25, 2017 - link

    'EmergencyLake-X' is just generally embarrassing. Intel should just find a nearby landfill site and quietly bury it.

Log in

Don't have an account? Sign up now