CPU System Tests

Our first set of tests is our general system tests. These set of tests are meant to emulate more about what people usually do on a system, like opening large files or processing small stacks of data. This is a bit different to our office testing, which uses more industry standard benchmarks, and a few of the benchmarks here are relatively new and different.

All of our benchmark results can also be found in our benchmark engine, Bench.

PDF Opening

First up is a self-penned test using a monstrous PDF we once received in advance of attending an event. While the PDF was only a single page, it had so many high-quality layers embedded it was taking north of 15 seconds to open and to gain control on the mid-range notebook I was using at the time. This put it as a great candidate for our 'let's open an obnoxious PDF' test. Here we use Adobe Reader DC, and disable all the update functionality within. The benchmark sets the screen to 1080p, opens the PDF to in fit-to-screen mode, and measures the time from sending the command to open the PDF until it is fully displayed and the user can take control of the software again. The test is repeated ten times, and the average time taken. Results are in milliseconds.

System: PDF Opening with Adobe Reader DC

This opening test is single threaded, so the high-frequency Intel parts get a clear win. There's not much between the Threadripper CPUs here.

FCAT Processing: link

One of the more interesting workloads that has crossed our desks in recent quarters is FCAT - the tool we use to measure and visually analyze stuttering in gaming due to dropped or runt frames. The FCAT process requires enabling a color-based overlay onto a game, recording the gameplay, and then parsing the video file through the analysis software. The software is mostly single-threaded, however because the video is basically in a raw format, the file size is large and requires moving a lot of data around. For our test, we take a 90-second clip of the Rise of the Tomb Raider benchmark running on a GTX 980 Ti at 1440p, which comes in around 21 GB, and measure the time it takes to process through the visual analysis tool.

System: FCAT Processing ROTR 1440p GTX980Ti Data

Similar to PDF opening, single threaded performance wins out.

Dolphin Benchmark: link

Many emulators are often bound by single thread CPU performance, and general reports tended to suggest that Haswell provided a significant boost to emulator performance. This benchmark runs a Wii program that ray traces a complex 3D scene inside the Dolphin Wii emulator. Performance on this benchmark is a good proxy of the speed of Dolphin CPU emulation, which is an intensive single core task using most aspects of a CPU. Results are given in minutes, where the Wii itself scores 17.53 minutes.

System: Dolphin 5.0 Render Test

Dolphin likes single thread performance as well, although having some cores to back it up seems to be required. 

3D Movement Algorithm Test v2.1: link

This is the latest version of the self-penned 3DPM benchmark. The goal of 3DPM is to simulate semi-optimized scientific algorithms taken directly from my doctorate thesis. Version 2.1 improves over 2.0 by passing the main particle structs by reference rather than by value, and decreasing the amount of double->float->double recasts the compiler was adding in. It affords a ~25% speed-up over v2.0, which means new data.

System: 3D Particle Movement v2.1

Our first pure multithreaded test, and the 1950X wins with 32 threads. The 1920X beats the 1950X in SMT-off mode, due to 24 threads beating 16 threads.

DigiCortex v1.20: link

Despite being a couple of years old, the DigiCortex software is a pet project for the visualization of neuron and synapse activity in the brain. The software comes with a variety of benchmark modes, and we take the small benchmark which runs a 32k neuron/1.8B synapse simulation. The results on the output are given as a fraction of whether the system can simulate in real-time, so anything above a value of one is suitable for real-time work. The benchmark offers a 'no firing synapse' mode, which in essence detects DRAM and bus speed, however we take the firing mode which adds CPU work with every firing.

System: DigiCortex 1.20 (32k Neuron, 1.8B Synapse)

DigiCortex requires a mash of CPU frequency and DRAM performance to get a good result, and anything with quad-channel memory is usually preferred. The 1950X in SMT-off mode wins here due to its low main memory latency combined with having 16 threads to access it. The Broadwell-E is the nearest competitor, over Skylake-X, most likely due to the mesh vs ring topology. The 1950X in Creator mode scores way down the field however, lower than the standard Ryzen chips, showing that under a unified memory architecture there can be significant performance drops. The 1920X failed in this test for an unknown reason.

Agisoft Photoscan 1.0: link

Photoscan stays in our benchmark suite from the previous version, however now we are running on Windows 10 so features such as Speed Shift on the latest processors come into play. The concept of Photoscan is translating many 2D images into a 3D model - so the more detailed the images, and the more you have, the better the model. The algorithm has four stages, some single threaded and some multi-threaded, along with some cache/memory dependency in there as well. For some of the more variable threaded workload, features such as Speed Shift and XFR will be able to take advantage of CPU stalls or downtime, giving sizeable speedups on newer microarchitectures.

System: Agisoft Photoscan 1.0 Total Time

The variable threaded nature of Agisoft shows that in our workflow, it's a mix of cores, IPC and frequency required to win. AMD takes a back seat here, likely due to its AVX implementation.

The 2017 Benchmark Suite Benchmarking Performance: CPU Rendering Tests
Comments Locked

347 Comments

View All Comments

  • nitin213 - Thursday, August 10, 2017 - link

    Thanks for your reply. Hopefully the test suite can be expanded as Intel's CPUs probably also move to higher core count and IO ranges in future.
    and i completely understand the frustration trying to get a 3rd party to change their defaults. Cheers
  • deathBOB - Thursday, August 10, 2017 - link

    It's clear to me . . . Ian is playing both sides and making out like a bandit! /s
  • FreckledTrout - Thursday, August 10, 2017 - link

    Ian can we get an updated comments section so we can +/- people and after x number of minuses they wont show by default. I'm saying this because some of these comments(the one in this chain included) are not meaningful responces. The comments section is by far the weakest link on Anantech.

    Nice review btw.
  • mapesdhs - Thursday, August 10, 2017 - link

    toms has that, indeed it's kinda handy for blanking out the trolls. Whether it's any useful indicator of "valid" opinion though, well, that kinda varies. :D (there's nowt to stop the trolls from voting everything under the sun, though one option would be to auto-suspend someone's ability to vote if their own posts get hidden from down voting too often, a hands-off way of slapping the trolls)

    Given the choice, I'd much rather just be able to *edit* what I've posted than up/down-vote what others have written. I still smile recalling a guy who posted a followup to apologise for the typos in his o.p., but the followup had typos aswell, after which he posted aaaaagh. :D

    Ian.
  • Johan Steyn - Thursday, August 10, 2017 - link

    Ian thanks for at least responding, I appreciate it. Please compare your review to sites like PCPer and many others. They have no problem to also point out the weak points of TR, yet clearly understand for what TR was mostly designed and focus properly on it and even though they did not test the 64 PCI lanes as an example, mention that they are planning a follow-up to do it, since it is an important point. You do mention these as well, but could have said more than just mention it by the way.

    Look at your review, most of it is about games. Are you serious?

    I have to give you credit to at least mention the problems with Sysmark.

    Let me give you an example of slanted journalism, When you do the rendering benchmarks, where AMD is known to shine, you only mention at each benchmark what they do etc, and fail to mention that AMD clearly beats Intel, even though other sites focus more ons these benchmarks. In the one benchmark where Intel get a descent score, you take time to mention that:

    "Though it's interesting just how close the 10-core Core i9-7900X gets in the CPU (C++) test despite a significant core count disadvantage, likely due to a combination of higher IPC and clockspeeds."

    Not in one of the rendering benchmarks do you give credit to AMD, yet you found it fitting to end the section of with:

    "Intel recently announced that its new 18-core chip scores 3200 on Cinebench R15. That would be an extra 6.7% performance over the Threadripper 1950X for 2x the cost."

    Not slanted journalism? At least you mention "2x the cost," but for most this will not defer them in buying the monopoly.

    After focussing so much time on game performance, I am not sure you understand TR at all. AMD still has a long way to go in many areas. Why? Because corrupt Intel basically drove them to bankruptcy, but that is a discussion for another day. I lived through those days and experienced it myself.

    Maybe I missed it, but where did you discuss the issue of memory speed? You mention in the beginning of memory overclock. Did you test the system running at 3200 or 2666? It is important to note. If you ran at 2666, then you are missing a very important point. Ryzen is known to gain a huge amount with memory speed. You should not regard 3200 as an overclock, since that is what that memory is made for, even if 2666 is standard spec. Most other sites I checked, used it like that. If you did use 3200, don't you think you should mention it?

    Why is it that your review ends up meh about TR and leave you rather wanting an i9 an almost all respects, yet most of the other site gives admiration where deserved, even though they have criticism as well. Ian I see that you clearly are disappointed with TR, which is OK, maybe you just like playing games and that is why you are so.

    It was clear how much you admire Intel in your previous article. You say that I gave no examples of slanted journalism, maybe you should read my post again. "Most Powerful, Most scalable." It is well known that people don't read the fine print. This was intentional. If not, you are a very unlucky guys for having so many unintended mishaps. Then I truly need to say I am sorry.

    For once, please be a bit excited that there is some competition against the monopoly of Intel, or maybe you are also deluded that they became so without any underhanded ways.

    By the way, sorry that I called you Anand. I actually wanted to type Anandtech, but left it like it. This site still carries his name and he should still take responsibility. After I posted, I realised I should have just checked the author, so sorry about that.
  • vanilla_gorilla - Thursday, August 10, 2017 - link

    "Intel recently announced that its new 18-core chip scores 3200 on Cinebench R15. That would be an extra 6.7% performance over the Threadripper 1950X for 2x the cost."

    How do you not understand that is a dig at Intel? He's saying you have to pay twice as much for only a 6.7% improvement.
  • smilingcrow - Thursday, August 10, 2017 - link

    The memory speed approach taken was clearly explained in the test and was stated as being consistent with how they always test.
    I don't take issue with testing at stock speeds at launch day as running memory out of spec for the system can be evaluated in depth later on.
  • Johan Steyn - Friday, August 11, 2017 - link

    That is just rubbish. Threadripper has no problem with 3200 memory and other sites has no problem running it at that speed. 3200 memory is designed to run 3200, why run it at 2666? There is just no excuse except being paid by Intel.

    Maybe then you can accuse other sites of being unscientific?
  • fanofanand - Tuesday, August 15, 2017 - link

    Anandtech always tests at JDEC, regardless of the brand.
  • Manch - Friday, August 11, 2017 - link

    ""Intel recently announced that its new 18-core chip scores 3200 on Cinebench R15. That would be an extra 6.7% performance over the Threadripper 1950X for 2x the cost."

    Not slanted journalism? At least you mention "2x the cost," but for most this will not defer them in buying the monopoly."

    You call Intel the monopoly and call him out for not wording the sentence to dissuade people from buying Intel. Who has the bias here? If he was actively promoting Intel over AMD or vice versa, you'd be OK with the latter, but to do neither. He's an Intel shill? Come on. That's unfair. HOW should he have wrote it so it would satisfy you?

    FYI Anand is gone. He's NOT responsible for anything at Anandtech. Are you going to hold Wozniak's feet to the fire for the lack of ports on a Mac too?

Log in

Don't have an account? Sign up now