Benchmarking Performance: CPU System Tests

Our first set of tests is our general system tests. These set of tests are meant to emulate more about what people usually do on a system, like opening large files or processing small stacks of data. This is a bit different to our office testing, which uses more industry standard benchmarks, and a few of the benchmarks here are relatively new and different.

All of our benchmark results can also be found in our benchmark engine, Bench.

PDF Opening

First up is a self-penned test using a monstrous PDF we once received in advance of attending an event. While the PDF was only a single page, it had so many high-quality layers embedded it was taking north of 15 seconds to open and to gain control on the mid-range notebook I was using at the time. This put it as a great candidate for our 'let's open an obnoxious PDF' test. Here we use Adobe Reader DC, and disable all the update functionality within. The benchmark sets the screen to 1080p, opens the PDF to in fit-to-screen mode, and measures the time from sending the command to open the PDF until it is fully displayed and the user can take control of the software again. The test is repeated ten times, and the average time taken. Results are in milliseconds.

System: PDF Opening with Adobe Reader DC

PDF opening is all about single thread frequency and IPC, giving the win to the new KBL-X chips.

FCAT Processing: link

One of the more interesting workloads that has crossed our desks in recent quarters is FCAT - the tool we use to measure stuttering in gaming due to dropped or runt frames. The FCAT process requires enabling a color-based overlay onto a game, recording the gameplay, and then parsing the video file through the analysis software. The software is mostly single-threaded, however because the video is basically in a raw format, the file size is large and requires moving a lot of data around. For our test, we take a 90-second clip of the Rise of the Tomb Raider benchmark running on a GTX 980 Ti at 1440p, which comes in around 21 GB, and measure the time it takes to process through the visual analysis tool.

System: FCAT Processing ROTR 1440p GTX1080 Data

FCAT similarly favors frequency and IPC. For this sort of workload, the Core i7 is the best chip to get.

3D Movement Algorithm Test v2.1: link

This is the latest version of the self-penned 3DPM benchmark. The goal of 3DPM is to simulate semi-optimized scientific algorithms taken directly from my doctorate thesis. Version 2.1 improves over 2.0 by passing the main particle structs by reference rather than by value, and decreasing the amount of double->float->double recasts the compiler was adding in. It affords a ~25% speed-up over v2.0, which means new data.

System: 3D Particle Movement v2.1

As 3DPM expands into several threads, the new quad-core parts will easily get trounced here by AMD's 8-cores for the same price. The Core i7-7800X puts on a good showing, as per core Intel's chips give a higher score.

DigiCortex v1.20: link

Despite being a couple of years old, the DigiCortex software is a pet project for the visualization of neuron and synapse activity in the brain. The software comes with a variety of benchmark modes, and we take the small benchmark which runs a 32k neuron/1.8B synapse simulation. The results on the output are given as a fraction of whether the system can simulate in real-time, so anything above a value of one is suitable for real-time work. The benchmark offers a 'no firing synapse' mode, which in essence detects DRAM and bus speed, however we take the firing mode which adds CPU work with every firing.

System: DigiCortex 1.20 (32k Neuron, 1.8B Synapse)

DigiCortex likes a bit of everything: cores, threads, IPC, frequency, uncore frequency, and memory frequency. The Core i7 parts roughly double the Core i5s due to the thread count, and also the AMD Ryzen parts skip ahead as well due to having double the threads to the Core i7.

Agisoft Photoscan 1.0: link

Photoscan stays in our benchmark suite from the previous version, however now we are running on Windows 10 so features such as Speed Shift on the latest processors come into play. The concept of Photoscan is translating many 2D images into a 3D model - so the more detailed the images, and the more you have, the better the model. The algorithm has four stages, some single threaded and some multi-threaded, along with some cache/memory dependency in there as well. For some of the more variable threaded workload, features such as Speed Shift and XFR will be able to take advantage of CPU stalls or downtime, giving sizeable speedups on newer microarchitectures.

System: Agisoft Photoscan 1.0 Total Time

Agisoft is like a Formula 1 race circuit: the long fast straights and techical corners make it a nightmare to have the technology to be the best at both, and Photoscan has enough serial code for high single thread performance to take advantage but also massively parallel sections where having 12-18 threads makes a difference.  Despite having half the threads, the single core performance of the Core i7-7740X makes it pull ahead of the Ryzen 7 chips, but when comparing the four threads of the Core i5-7640X to the twelve threads of the Ryzen 5 processors, having 12 threads wins.

Benchmark Overview Benchmarking Performance: CPU Rendering Tests
Comments Locked

176 Comments

View All Comments

  • mapesdhs - Monday, July 24, 2017 - link

    Let the memes collide, focus the memetic radiation, aim it at IBM and get them to jump into the x86 battle. :D
  • dgz - Monday, July 24, 2017 - link

    Man, I could really use an edit button. my brain has shit itself
  • mapesdhs - Monday, July 24, 2017 - link

    Have you ever posted a correction because of a typo, then realised there was a typo in the correction? At that point my head explodes. :D
  • Glock24 - Monday, July 24, 2017 - link

    "The second is for professionals that know that their code cannot take advantage of hyperthreading and are happy with the performance. Perhaps in light of a hyperthreading bug (which is severely limited to minor niche edge cases), Intel felt a non-HT version was required."

    This does not make any sense. All motherboards I've used since Hyper Threading exists (yes, all the way back to the P4) lets you disable HT. There is really no reason for the X299 i5 to exist.
  • Ian Cutress - Monday, July 24, 2017 - link

    Even if the i5 was $90-$100 cheaper? Why offer i5s at all?
  • yeeeeman - Monday, July 24, 2017 - link

    First interesting point to extract from this review is that i7 2600K is still good enough for most gaming tasks. Another point that we can extract is that games are not optimized for more than 4 core so all AMD offerings are yet to show what they are capable of, since all of them have more than 4 cores / 8 threads.
    I think single threading argument absolute performance argument is plain air, because the differences in single thread performance between all top CPUs that you can currently buy is slim, very slim. Kaby Lake CPUs are best in this just because they are sold with high clocks out of the box, but this doesn't mean that if AMD tweaks its CPUs and pushes them to 5Ghz it won't get back the crown. Also, in a very short time there will be another uArch and another CPU that will have again better single threaded performance so it is a race without end and without reason.
    What is more relevant is the multi-core race, which sooner or later will end up being used more and more by games and software in general. And when games will move to over 4 core usage then all these 4 cores / 8 threads overpriced "monsters" will become useless. That is why I am saying that AMD has some real gems on their hands with the Ryzen family. I bet you that the R7 1700 will be a much better/competent CPU in 3 years time compared to 7700K or whatever you are reviewing here. Dirt cheap, push it to 4Ghz and forget about it.
  • Icehawk - Monday, July 24, 2017 - link

    They have been saying for years that we will use more cores. Here we are almost 20 years down the road and there are few non professional apps and almost no games that use more than 4 cores and the vast majority use just two. Yes, more cores help with running multiple apps & instances but if we are just looking at the performance of the focused app less cores and more MHz is still the winner. From all I have read the two issues are that not everything is parallelizable and that coding for more cores/threads is more difficult and neither of those are going away.
  • mapesdhs - Monday, July 24, 2017 - link

    Thing is, until now there hasn't been a mainstream-affordable solution. It's true that parallel coding requires greater skill, but that being the case then the edu system should be teaching those skills. Instead the time is wasted on gender studies nonsense. Intel could have kick started this whole thing years ago by releasing the 3930K for what it actually was, an 8-core CPU (it has 2 cores disabled), but they didn't have to because back then AMD couldn't even compete with mid-range SB 2500K (hence why they never bothered with a 6-core for mainstream chipsets). One could argue the lack of market sw evolvement to exploit more cores is Intel's fault, they could have helped promote it a long time ago.
  • cocochanel - Tuesday, July 25, 2017 - link

    +1!!!
  • twtech - Monday, July 24, 2017 - link

    What can these chips do with a nice watercooling setup, and a goal of 24x7 stability? Maybe 4.7? 4.8?

    These seem like pretty moderate OCs overall, but I guess we were a bit spoiled by Sandy Bridge, etc., where a 1GHz overclock wasn't out of the question.

Log in

Don't have an account? Sign up now