Benchmark Overview

For our testing, depending on the product, we attempt to tailor the presentation of our global benchmark suite down into what users who would buy this hardware might actually want to run. Barring abnormalities, our full test suite is typically used to gather data and all the results are placed into Bench, our benchmark database for users that want to look at non-typical benchmarks or legacy data.

The benchmarks fall into several areas:

Short Form CPU

Our short form testing script uses a straight run through of a mixture of known apps or workloads, and requires about four hours. These are typically the CPU tests we run in our motherboard suite, to identify any performance anomolies.

CPU Short Form Benchmarks
Three Dimensional Particle Movement (3DPM) v1 3DPM is a self-penned benchmark, derived from my academic research years looking at particle movement parallelism. The coding for this tool was rough, but emulates the real world in being non-CompSci trained code for a scientific endeavor. The code is unoptimized, but the test uses OpenMP to move particles around a field using one of six 3D movement algorithms in turn, each of which is found in the academic literature. This test is performed in single thread and multithreaded workloads, and uses purely floating point numbers. The code was written in Visual Studio 2008 in Release mode with all optimizations (including fast math and -Ox) enabled. We take the average of six runs in each instance.
v2 The second version of this benchmark is similar to the first, however it has been re-written in VS2012 with one major difference: the code has been written to address the issue of false sharing. If data required by multiple threads, say four, is in the same cache line, the software cannot read the cache line once and split the data to each thread - instead it will read four times in a serial fashion. The new software splits the data to new cache lines so reads can be parallelized and stalls minimized.

As v2 is fairly new, we are still gathering data and results are currently limited.
WinRAR 5.01 WinRAR is a compression based software to reduce file size at the expense of CPU cycles. We use the version that has been a stable part of our benchmark database through 2015, and run the default settings on a 1.52GB directory containing over 2800 files representing a small website with around thirty half-minute videos. We take the average of several runs in this instance.
POV-Ray 3.7 beta POV-Ray is a common ray-tracing tool used to generate realistic looking scenes. We've used POV-Ray in its various guises over the years as a good benchmark for performance, as well as a tool on the march to ray-tracing limited immersive environments. We use the built-in multithreaded benchmark.
HandBrake  HandBrake is a freeware video conversion tool. We use the tool in to process two different videos - first a 'low quality' two hour video at 640x388 resolution to x264, then a 'high quality' ten minute video at 4320x3840. The low quality video scales at lower performance hardware, whereas the buffers required for high-quality can stretch even the biggest processors. At current, this is a CPU only test.
7-Zip 7-Zip is a freeware compression/decompression tool that is widely deployed across the world. We run the included benchmark tool using a 50MB library and take the average of a set of fixed-time results.

Web, Synthetic and Legacy

The web tests are a usual mix of Octane/Kraken with WebXPRT in the mix. Synthetic and Legacy CPU testing relates to our long-term data under CineBench and x264.

Web and Synthetic Benchmarks
Google Octane 2.0 Lots of factors go into web development, including the tools used and the browser those tools play in. One of the common and widely used benchmarks to judge performance is Google Octane, now in version 2.0. To quote: 'The updated Octane 2.0 benchmark includes four new tests to measure new aspects of JavaScript performance, including garbage collection / compiler latency and asm.js-style JavaScript performance.'
Mozilla Kraken 1.1 Kraken is a similar tool to Google, focusing on web tools and processing power. Kraken's tools include searching algorithms, audio processing, image filtering, flexible database parsing and cryptographic routines.
WebXPRT 2013/2015 WebXPRT aims to be a souped up version of Octane and Kraken, using these tools in real time to display data in photograph enhancement, sorting, stock options, local storage manipulation, graphical enterfaces and even filtering algorithms on scientific datasets. We run the 2013 and 2015 versions of the benchmark.
Cinebench Cinebench is a widely known benchmarking tool for measuring performance relative to MAXON's animation software Cinema 4D. Cinebench has been optimized over a decade and focuses on purely CPU horsepower, meaning if there is a discrepancy in pure throughput characteristics, Cinebench is likely to show that discrepancy. Arguably other software doesn't make use of all the tools available, so the real world relevance might purely be academic, but given our large database of data for Cinebench it seems difficult to ignore a small five minute test. We run the modern version 15 in this test, as well as the older 11.5 due to our back data.
x264 HD 3.0 Similarly, the x264 HD 3.0 package we use here is also kept for historic regressional data. The latest version is 5.0.1, and encodes a 1080p video clip into a high quality x264 file. Version 3.0 only performs the same test on a 720p file, and in most circumstances the software performance hits its limit on high end processors, but still works well for mainstream and low-end. Also, this version only takes a few minutes, whereas the latest can take over 90 minutes to run.
TrueCrypt 7.1 Before its discontinuation, TrueCrypt was a popular tool for WindowsXP to offer software encryption to a file system. The near-latest version, 7.1, is still widely used however the developers have stopped supporting it since the introduction of encrypted disk support in Windows 8/7/Vista from 5/2014, and as such any new security issues are unfixed.

Long Form and Professional Benchmarks

For reviews that require a little more depth, we invoke our long form CPU tests. These scripts include the short form tests, the web tests, and bundle some real-world tests that are influenced by processor frequency, core count, cache sizes and memory support. Generational advances between CPU microarchitectures show up here as well. Our professional tests involve a 2D to 3D image and model conversion tool used by archivists and modelers as well as Linux Bench. We are currently looking into expanding our professional testing suite to include code compilation as well as FPGA workflows.

Long Form and Professional Benchmarks
Dolphin 4.0
(Wii Emulation)
Many emulators are often bound by single thread CPU performance, and general reports tended to suggest that Haswell provided a significant boost to emulator performance. This benchmark runs a Wii program that raytraces a complex 3D scene inside the Dolphin Wii emulator. Performance on this benchmark is a good proxy of the speed of Dolphin CPU emulation, which is an intensive single core task using most aspects of a CPU. 
Agisoft Photoscan 1.0 Photoscan is professional software that takes a series of 2D images (as little as 50, usually 250+) and 'performs calculations' to determine where the pictures were taken and if it can create a 3D model and textures of what the images are of. This model can then be exported to other software for touch-ups or implementation in physics engines/games or, as the reader that directed me to it, national archiving. The tool has four phases, one of which can be OpenCL accelerated, while the other three are a mix of single thread and variable thread workloads.
Linux Bench Linux Bench is a collection of Linux based benchmarks compiled together by ServeTheHome. The idea for this is to have some non-windows based tools that are easy enough to run with a USB key, an internet connection and three lines of code in a terminal. The tests in Linux Bench include standard synthetic compute, compression, matrix manipulation, database tools and key-value storage.

 

Gaming 

Our Gaming test suite is still our 2015 implementation, which remains fairly solid over gaming title updates. We are still working on a 2016 suite update, with a move to Windows 10. This will allow most of the titles to be replaced with DirectX 12, indie and eSports games.

Gaming Benchmarks
Alien: Isolation If first person survival mixed with horror is your sort of thing, then Alien: Isolation, based off of the Alien franchise, should be an interesting title. Developed by The Creative Assembly and released in October 2014, Alien: Isolation has won numerous awards from Game Of The Year to several top 10s/25s and Best Horror titles, ratcheting up over a million sales by February 2015. Alien: Isolation uses a custom built engine which includes dynamic sound effects and should be fully multi-core enabled.
Total War: Attila The Total War franchise moves on to Attila, another The Creative Assembly development, and is a stand-alone strategy title set in 395AD where the main story line lets the gamer take control of the leader of the Huns in order to conquer parts of the world. Graphically the game can render hundreds/thousands of units on screen at once, all with their individual actions and can put some of the big cards to task.
Grand Theft Auto V The highly anticipated iteration of the Grand Theft Auto franchise finally hit the shelves on April 14th 2015, with both AMD and NVIDIA in tow to help optimize the title. GTA doesn’t provide graphical presets, but opens up the options to users and extends the boundaries by pushing even the hardest systems to the limit using Rockstar’s Advanced Game Engine. Whether the user is flying high in the mountains with long draw distances or dealing with assorted trash in the city, when cranked up to maximum it creates stunning visuals but hard work for both the CPU and the GPU.
GRID: Autosport No graphics tests are complete without some input from Codemasters and the EGO engine, which means for this round of testing we point towards GRID: Autosport, the next iteration in the GRID and racing genre. As with our previous racing testing, each update to the engine aims to add in effects, reflections, detail and realism, with Codemasters making ‘authenticity’ a main focal point for this version.
Middle-Earth: Shadow of Mordor The final title in our testing is another battle of system performance with the open world action-adventure title, Shadow of Mordor. Produced by Monolith using the LithTech Jupiter EX engine and numerous detail add-ons, SoM goes for detail and complexity to a large extent, despite having to be cut down from the original plans. The main story itself was written by the same writer as Red Dead Redemption, and it received Zero Punctuation’s Game of The Year in 2014.
Test Bed and Setup Performance at 3 GHz: Real World
POST A COMMENT

131 Comments

View All Comments

  • The_Countess - Tuesday, July 19, 2016 - link

    actually bulldozer on 14nm would have been a completely different beast. it would have allowed AMD to use far more transistors per core while still making it way smaller in terms of size. that would have allowed AMD to create a far wider execution core, eliminating most of its bottlenecks.

    the high latency cache would probably still means it wouldn't be great for games but for everything else it would be a far more competitive design.

    it is also 14nm that will allow zen to make such a massive leap in IPC's as it will be a very wide Core, while still being pretty small, something that just can't be done on 28nm.

    bulldozer might not have been the best idea, but being stuck on 32/28nm for so long made all it's issues infinitely worse.
    Reply
  • nandnandnand - Thursday, July 14, 2016 - link

    "Well better late than never for Andantech,"

    There was no point in Adanantech writing this review, because it is a chip for those people too stupid to wait until Zen. Zen is the only thing that matters.
    Reply
  • BurntMyBacon - Friday, July 15, 2016 - link

    @nandnandnand: "There was no point in Adanantech writing this review, because it is a chip for those people too stupid to wait until Zen. Zen is the only thing that matters."

    Now, because this review exists, people as yet uninformed have concrete data to avoid decisions that might make them look (as you put it) stupid. There is very much a point.
    Reply
  • Byte - Thursday, July 14, 2016 - link

    Zen will probably be the RX480 in the CPU world. Better performance, still trounced by the competition, but competently priced. Reply
  • looncraz - Friday, July 15, 2016 - link

    That would be an improvement on the current situation. AMD is pricing their CPUs quite poorly right now.

    An Intel Celeron G3900 is $50 right now. AMD's closest competition is the A6-7400k - at $55.

    Both are dual cores, both are 65W, both have middling (but usable) graphics performance... quite similar at first glance... except the Intel runs at 2.8Ghz and the AMD runs at 3.5Ghz w/ 3.9Ghz turbo and can rather easily exceed 4Ghz when overclocked.

    Sounds like AMD should be taking home the gold on that one, until you find that the Celeron is nearly 25% faster in single threaded programs and is ~40% faster in multi-threaded programs... Bad deal going for the AMD... especially since the same board that hosts the Celeron can accept much faster CPUs and the AMD board simply doesn't have notably more powerful options available - you can upgrade to a quad core, but you won't be getting better single threaded performance no matter how hard you try. You might break even around 5Ghz, if you can manage it...

    AMD has a 40% clock-speed advantage out the gate, but loses by a large margin.
    Reply
  • bananaforscale - Friday, July 15, 2016 - link

    You know what's funny? The fact that if I want to get a CPU that's faster than the FX-6100 I bought almost 5 years ago I still have to pay more than what I paid for it. Sure, Intel gives better single thread performance but I'd get fewer cores and no overclockability. Then there's the fact that I've been running that original Bulldozer with a 20% OC and it seems more stable than at stock clocks.

    Comparing single data points tells nobody a thing. Anyway, isn't that A6 in your comparison unlocked? :P
    Reply
  • wumpus - Friday, July 15, 2016 - link

    I'm sure you missed an FX-8320 sale, or you really nailed the low point. Unfortunately Intel can match AMD's performance at nearly the same price, and is cutting off AMD's air supply that way. Reply
  • artk2219 - Monday, July 18, 2016 - link

    Whats crazy is that Microcenter sells the FX 8320E's for $89.99. They also have a motherboard bundle option that you can get for $125 to $170 depending on which board you choose. Theoretically you can get a processor, motherboard, cooler, and memory for the price of a non-K core I5, or just a motherboard and processor for the price of an I3. The unfortunate thing is that not everyone has a microcenter near them, but for the ones that do you can get quite the deal, especially since those 8320E's will easily OC to FX 8350 levels, and more likely 4.2 to 4.6 from a stock clock of 3.2 Reply
  • BlueBlazer - Friday, July 15, 2016 - link

    From the leaks plus AMD's vague announcements, all points to AMD's Zen is going to be quite late (right into 2017). Why put use 28nm "placeholder" for AM4 if Zen is due soon? Also Global Foundries only has 14nm LPP which is a low power process. That may mean the frequency is going to be low (just look at the chips made on 14nm LPP like Qualcomm's Snapdragon 820, or even AMD's latest Radeon RX480). Reference http://semiengineering.com/high-performance-and-lo... quote "The “LP” processes are optimized for low power and feature design rules targeted for the lowest leakage, support lower operative voltages, and tend to have the slowest transistors of the three options". Reply
  • wiboonsin - Sunday, November 12, 2017 - link

    I have read your article, it is very informative and helpful for me.I admire the valuable information you offer in your articles. Thanks for posting http://www.fanaticrunningwear.com/ Reply

Log in

Don't have an account? Sign up now