CPU Real World Performance

A small note on real world testing against synthetic testing – due to the way that DRAM affects a system, there can be a large disconnect between what we can observe in synthetic tests against real world testing. Synthetic tests are designed to exploit various feature XYZ, usually in an unrealistic scenario, such as pure memory read speeds or bandwidth numbers. While these are good for exploring the peak potential of a system, they often to not translate as well as CPU speed does if we invoke some common prosumer real world task. So while spending 10x on memory might show a large improvement in peak bandwidth numbers, users will have to weigh up the real world benefits in order to find the day-to-day difference when going for expensive hardware. Typically a limiting factor might be something else in the system, such as the size of a cache, so with all the will in the world a faster read speed won’t make much difference. As a result, we tend to stick to real world tests for almost all of our testing (with a couple of minor suggestions). Our benchmarks are either derived from areas such as transcoding a film or come from a regular software format such as molecular dynamics running a consistent scene.

Handbrake v0.9.9

For HandBrake, we take two videos (a 2h20 640x266 DVD rip and a 10min double UHD 3840x4320 animation short) and convert them to x264 format in an MP4 container.  Results are given in terms of the frames per second processed, and HandBrake uses as many threads as possible.

HandBrake v0.9.9 LQ Film

HandBrake v0.9.9 HQ Film

The low quality conversion is more reliant on CPU cycles available, while the high resolution conversion seems to have a very slight ~3% benefit moving up to DDR4-3000 memory.

WinRAR 5.01

Our WinRAR test from 2013 is updated to the latest version of WinRAR at the start of 2014. We compress a set of 2867 files across 320 folders totaling 1.52 GB in size – 95% of these files are small typical website files, and the rest (90% of the size) are small 30 second 720p videos.

WinRAR 5.01

The biggest difference showed a 5% gain over DDR4-2133 C15, although this seemed at random.

FastStone Image Viewer 4.9

FastStone Image Viewer is a free piece of software I have been using for quite a few years now. It allows quick viewing of flat images, as well as resizing, changing color depth, adding simple text or simple filters. It also has a bulk image conversion tool, which we use here. The software currently operates only in single-thread mode, which should change in later versions of the software. For this test, we convert a series of 170 files, of various resolutions, dimensions and types (of a total size of 163MB), all to the .gif format of 640x480 dimensions. Results shown are in seconds, lower is better.

FastStone Image Viewer 4.9

No difference between the memory speeds in FastStone.

x264 HD 3.0 Benchmark

The x264 HD Benchmark uses a common HD encoding tool to process an HD MPEG2 source at 1280x720 at 3963 Kbps. This test represents a standardized result which can be compared across other reviews, and is dependent on both CPU power and memory speed. The benchmark performs a 2-pass encode, and the results shown are the average frame rate of each pass performed four times. Higher is better this time around.

x264 HD 3.0, 1st Pass

x264 HD 3.0, 2nd Pass

The faster memory showed a 2.5% gain on the first pass, but less than a 1% gain in the second pass.

7-Zip 9.2

As an open source compression tool, 7-Zip is a popular tool for making sets of files easier to handle and transfer. The software offers up its own benchmark, to which we report the result.

7-Zip 9.2

At most a 2% gain was shown by 3000+ memory.

Mozilla Kraken 1.1

One of the more popular web benchmarks that stresses various codes, we run this benchmark in Chrome 35.

Mozilla Kraken 1.1

Kraken seemed to prefer the fast 1.2V memory, giving a 4.8% gain at DDR4-2800 C16, although this did not translate into the faster memory.

WebXPRT

A more in-depth web test featuring stock price rendering, image manipulation and face recognition algorithms, also run in Chrome 35.

WebXPRT

The DDR4-3200 gave an 11% gain over the base JEDEC memory, although this seemed to be more of a step than a slow rise.

Enabling XMP Memory Scaling on Haswell: Professional Performance
Comments Locked

120 Comments

View All Comments

  • Harry Lloyd - Friday, February 6, 2015 - link

    So no difference whatsoever no matter which test? Not surprising, considering the quad channel controller.
    I hope to see a similar test when dual channel Skylake comes out. Also, please find some CPU-bound games. BioShock, Tomb Raider and Sleeping Dogs do not need more than two cores, which makes them completely pointless for this kind of test. Try games like Battlefield 4 MP or Dying Light (extremely CPU-bound and easy to repeat).
  • Arbie - Friday, February 6, 2015 - link

    @nwrigley - I also agree. I have a 2008 build using a Yorkfield quad at 3.6GHz, still running 32-bits and the original 4GB of DDR2. The three things I have really needed to add since then are SSDs, a new graphics card (expected), and adapters for USB3 ports. All of these are "bolt-on", not fundamental changes, and the only one I researched was the gfx board. I know a Haswell build would be 2x more powerful and run much cooler, but neither of those justifies a system replacement. I almost never max out the CPU, or even the RAM.

    This "good enough" syndrome is obviously affecting the industry, and even the websites dealing with it. One well established and very good equipment review site has recently gone, probably because too few people still care about small differences in desktop motherboard, PSU, DRAM, and cooler performance. I suppose this trend will continue.
  • jabber - Friday, February 6, 2015 - link

    I have to admit I stopped looking seriously at RAM reviews once we hit DDR2. I wince when I see a reviewer has wasted a week of their life to do a DDR3 'performance' RAM round up. Well thanks for telling us AGAIN that there is a performance difference of 2% or 0.5FPS between stock $50 RAM and the $300 top of the range. Why do they keep doing RAM group tests?
  • nwarawa - Friday, February 6, 2015 - link

    It wasn't very clear, but it sounded like the ddr3/4 comparison was dual channel vs quad channel. A better apples to apples test would run the x99 system is dual channel.
  • halcyon - Friday, February 6, 2015 - link

    TL;DR: Does NOT scale.

    The price difference between 2133 and any of the higher speeds makes no sense, unless you are a super-high res competitive pro-gamer or if you run real-time intensive huge dataloads 24/7.

    For even heavy users, workstations, etc - no point. Just buy the most reliable 2133 or 2400 that is the cheapest.

    Last graph is horrible, baseline doesn't start from zero. Differences are minimal.

    Sad is the day when the element of interest for pro users is : "Firstly is the design, and finding good looking memory".
  • jnkweaver - Friday, February 6, 2015 - link

    So for example, when given DDR3-2133 C10 (PI of 213) against DDR3-1866 C10 (PI or 187), the first one should be chosen. However with DDR3-2133 C10 (PI of 213) and DDR3-2400 C12 (PI of 200) at the same price, the results would suggest the latter is a better option.

    So 213 beats 187 (1st example) but 213 doesn't beat 200? (2nd example)
  • Wwhat - Saturday, February 7, 2015 - link

    So from the looks of the tests the speed absolutely makes no difference, but now what I'm wondering is what happens if you have many things running at the same time, several programs simultaneously, maybe that will bring some differences to light? Or is there really no difference at all? That seems a bit odd, and a flaw in the CPU design since it can't utilize the extra speed. The RAM speed is suppose to be a bottleneck for the CPU after all.

    Maybe we should hear some comments on the subject from intel and AMD.
  • DarkXale - Saturday, February 7, 2015 - link

    Its not at all a flaw; on the contrary its all about intelligently predicting what data we need to have access to soon.
  • gsuburban - Saturday, February 7, 2015 - link

    DDR4 is not that much a performance change and 4 times the cost so, DDR3 will still be around.
    It's overpriced RAM in the least.
  • YoloPascual - Sunday, February 8, 2015 - link

    DDR4 = half doa tech

Log in

Don't have an account? Sign up now