The Kits and The Markets

In our Haswell DDR3 scaling article, we introduced the concept of a Performance Index in order to compare memory kits of different frequencies and latencies against overall performance. The Performance Index is calculated by the rated speed divided by the CAS latency such that:

Performance Index = Frequency / CL

At the time it came across as a good indicator of performance when buying from the shelf, although most companies do not particularly advertise the latencies on the package. Our conclusion for DDR3 on Haswell was that the higher the Performance Index the better, although with two potential options close together, the one with the higher frequency is the better choice. So for example, when given DDR3-2133 C10 (PI of 213) against DDR3-1866 C10 (PI or 187), the first one should be chosen. However with DDR3-2133 C10 (PI of 213) and DDR3-2400 C12 (PI of 200) at the same price, the results would suggest the latter is a better option.

One of the big issues that the Performance Index does not take into account is the price, which fluctuates weekly depending on DRAM supply but also the capabilities of a kit through rarity. When purchasing memory ICs from the big three (Samsung, SK Hynix, Micron), the chips themselves are only rated at a basic speed and it is up to the module manufacturer to do further binning to find the best silicon for the high speed memory kits. As a result many of these companies will bid on certain batches with a history of high performance, and then the extra time required to separate the good chips from a batch adds into the high cost of the top frequency kits. Usually frequency defines the difficulty, rather than the sub-timings and latency because frequency is a defining factor.

When it comes to DDR4, we will be taking a similar broad approach to kit designation, taking the Performance Index of each memory kit in each benchmark and attempting to find a correlation.

The Market

At launch, DDR4 kits had the obvious premium of being a new technology as well as being is short supply due to Intel moving up the date for release. The prices at this time, as we reported, were the equivalent to $213 for a 4x4GB 2133 C15 kit going through $300 for a 4x4 GB 2666 C15 up to $413 for 4x4GB 3000 C15. Based on these numbers it would seem that the high end modules have come down in price quickly, but the lower range products have stayed similar. We took a range of pricing from Newegg to see the effect of being at market for just over six months has done.

The cheapest standard kit of 2133 C15 4x4 GB comes in at $200. The best kit in this layout would be towards the bottom left, as indicated by the performance index in each square:

Here I have marked four areas, representing the low end memory in orange, the more standard in white, the performance modules in green and the super-high performance in dark green. There are currently no modules in that last group, going through all the pricing from 2x8GB kits to 8x8GB kits:

The lowest price per GB is at $387 for 4x8GB of DDR4-2133 C15, at $12.09 per GB, compared to $1800 for 8x8GB of 2800 C16 which makes out to $28.13 per GB.

The interesting segments based on price alone that catch my in that case are:

4x4GB DDR3-2133 C13 for $213
4x8GB DDR3-2133 C13 for $400
At this point, a CAS latency below 15 is something of a novelty. It seems a little more esoteric than usual, as none of the manufacturers we spoke to even considered sampling us something of this nature. I would be interested to actually see how it performs.
4x4GB DDR4-2666 C13 for $290
4x4GB DDR4-3000 C15 for $300
I picked both of these based on the closeness of price and on Performance Index. The latter kit is something we have in for review, but similar to the previous kits listed, a CAS latency of 13 is an interesting element to the equation.
4x8GB DDR4-2800 C15 for $510 Measuring up at nearly $16 per GB, this kit mixes the elements of on-paper specifications, density and price for a nice X99 system.

The Kits

For this roundup and subsequent reviews, we received kits from almost every major memory manufacturer. G.Skill, Corsair, ADATA and Crucial were all willing to send various speeds and densities of memory, ranging from the basic 4x4 GB of DDR4-2133 C15 to 8x8GB of DDR4-2133 C15, or 8x8GB of DDR4-2400 C16 to 4x4GB of DDR4-3200 C16.

The main conclusions from this testing will be from the 4x4 GB modules in order to keep consistency, however the 4x8 GB and 8x8 GB results will be included for comparison. Larger modules (and more modules in a kit) tend to lead to relaxed sub-timings in order to ensure full compatibility with all CPUs in all motherboards. This means that in synthetic testing we may end up with some slightly different results, although this may differ in real-world tests.

Another point to note is module compatibility. When DDR4 was first launched with Haswell-E in September 2014, compatibility issues were a problem. Intel had moved up the time of the launch from mid-September to early September very late in the day, leaving memory vendors to scramble kits to market. This gives them a shorter time to work with ASUS, GIGABYTE, MSI and ASRock in order to insure no issues when working with motherboards, especially with high end memory. Due to this shortened timeframe there were some issues to begin with but these should have been ironed out since. Also on the high speed memory front, it would seem that early motherboard BIOSes were also unable to cope with the higher speed, higher density memory kits. Therefore it is important to make sure that all BIOSes are up to date when buying the expensive memory sets.

DDR4 Module Comparison
  SKU Kit Size Kit Speed Sub-Timings Voltage PI
Corsair CMD16GX4M4B3200C16 16 GB (4x4) DDR4-3200 16-18-18 2T 1.35 V 200
G.Skill F4-3000C15Q-16GRR 16 GB (4x4) DDR4-3000 15-15-15 2T 1.35 V 200
G.Skill F4-2800C16Q-16GRK 16 GB (4x4) DDR4-2800 16-16-16 2T 1.20 V 175
G.Skill F4-2666C15Q-16GRR 16 GB (4x4) DDR4-2666 15-15-15 2T 1.20 V 177
Crucial BLS8G4D240FSA 16 GB (4x4) DDR4-2400 16-16-16 2T 1.20 V 150
G.Skill F4-2133C15Q-16GRR 16 GB (4x4) DDR4-2133 15-15-15 2T 1.20 V 142
ADATA AX4U2400W8G16-QRZ 64GB (8x8) DDR4-2400 16-16-16 2T 1.20 V 150
Corsair CMV8GX4M1A2133C15 64GB (8x8) DDR4-2133 15-15-15 2T 1.20 V 142
 

The memory in this test is as follows, starting with the 4x4GB kits and fastest/most expensive:

DDR4-3200 16-18-18 2T 4x4 GB 1.35 V Corsair CMD16GX4M4B3200C16, PI of 200
$746 on Amazon
$685 on Newegg

DDR4-3000 15-15-15 2T 4x4 GB 1.35 V G.Skill F4-3000C15Q-16GRR, PI of 200
$436 on Amazon
$300 on Newegg

DDR4-2800 16-16-16 2T 4x4 GB 1.2 V G.Skill F4-2800C16Q-16GRK, PI of 175
$305 on Amazon
$270 on Newegg

DDR4-2666 15-15-15 2T 4x4 GB 1.2 V G.Skill F4-2666C15Q-16GRR, PI of 177
$290 on Amazon
$250 on Newegg

DDR4-2400 16-16-16 2T 4x4 GB 1.2 V Crucial BLS8G4D240FSA.16FAD, PI of 150
$180 on Amazon

DDR4-2133 15-15-15 2T 4x4 GB 1.2 V G.Skill F4-2133C15Q-16GRR, PI of 142
$315 Amazon
$250 Newegg

For good measure, we also have the following kits for testing:

DDR4-2400 16-16-16 2T 4x8 GB and 8x8 GB 1.2 V ADATA AX4U2400W8G16-QRZ, PI of 150
$400 for 4x8GB

DDR4-2133 15-15-15 2T 4x8 GB and 8x8GB 1.2 V Corsair CMV8GX4M1A2133C15-ESM, PI of 142
$120 per module

We also have more kits from Crucial, Corsair, G.Skill and KLEVV incoming for when we tackle individual reviews. The purpose of this scaling piece is merely to demonstrate the general effect of speed across the modules currently on the market. As mentioned, some of those kits with a CL of less than 15 look interesting, so I will have to give Mushkin a call or get a contact at Kingston.

Test Setup

Test Setup
Processor Intel Core i7-5960X ES, 8C/16T overclocked to 4.0 GHz
Motherboards ASUS X99 Deluxe
Cooling Cooler Master Nepton 140XL
Power Supply OCZ 1250W Gold ZX Series
Memory Corsair DDR4-3200 16-18-18 4x4GB, CMD16GX4M4B3200C16
G.Skill DDR4-3000 15-15-15 4x4GB, F4-3000C15Q-16GRR
G.Skill DDR4-2800 16-16-16 4x4GB, F4-2800C16Q-16GRK
G.Skill DDR4-2666 15-15-15 4x4GB, F4-2666C15Q-16GRR
Crucial DDR4-2400 16-16-16 4x4GB, BLS8G4D240FSA
G.Skill DDR4-2133 15-15-15 4x4GB, F4-2133C15Q-16GRR
ADATA DDR4-2400 16-16-16 8x8GB, AX4U2400W8G16-QRZ
Corsair DDR4-2133 15-15-15 8x8GB, CMV8GX4M1A2133C15-ESM
Memory Settings XMP
Video Cards MSI GTX 770 Lightning 2GB (1150/1202 Boost)
Hard Drive OCZ Vertex 3 256GB
Case Open Test Bed
Operating System Windows 7 64-bit SP1
DDR4 Haswell-E Scaling Review Enabling XMP
Comments Locked

120 Comments

View All Comments

  • dgingeri - Thursday, February 5, 2015 - link

    Really, what applications use this bandwidth now?

    I'm the admin of a server software test lab, and we've been forced to move to the Xeon E5 v3 platform for some of our software, and it isn't seeing any enhancement from DDR4 either. These are machines and software using 256GB of memory at a time. The steps from Xeon E5 and DDR3 1066 to E5 v2 and DDR3 1333 and then up to the E5 v3 and DDR4 2133 are showing no value whatsoever. We have a couple aspects with data dedup and throughput are processor intensive, and require a lot of memory, but the memory bandwidth doesn't show any enhancement. However, since Dell is EOLing their R720, under Intel's recommendation, we're stuck moving up to the new platform. So, it's driving up our costs with no increase in performance.

    I would think that if anything would use memory bandwidth, it would be data dedup or storage software. What other apps would see any help from this?
  • Mr Perfect - Thursday, February 5, 2015 - link

    Have you seen the reported reduction in power consumption? With 256GBs per machine, it sounds like you should be benefiting from the lower power draw(and lower cooling costs) of DDR4.
  • Murloc - Thursday, February 5, 2015 - link

    depending on the country and its energy prices, the expense to upgrade and the efficiency gains made, you may not even be able to recoup the costs, ever.
    From a green point of view it may be even worse due to embodied energy going to waste depending on what happens to the old server.
  • Mr Perfect - Friday, February 6, 2015 - link

    True, but if you have to buy DDR4 machines because the DDR3 ones are out of production(like the OP), then dropping power and cooling would be a neat side bonus.

    And now, just because I'm curios: If the max DDR4 DIMM is 8GB, and there's 256GB per server, then that's 32 DIMMs. 32 times 1 to 2 watts less a DIMM would be 32 to 64 watts less load on the PSU. If the PSU is 80% efficient, then that should be 38.4 to 76.8 watts less at the wall per machine. Not really spectacular, but then you've also got cooling. If the AC is 80% efficient, that would be 46.08 to 92.16 watts less power to the AC. So in total, the new DDR4 server would cost you (wall draw plus AC draw) 84.48 to 168.96 watts lower load per server versus the discontinued DDR3 ones. Not very exciting if you've only got a couple of them, but I could see large server farms benefiting.

    Anyone know how to work out the KWh and resulting price from electric rates?
  • menting - Friday, February 6, 2015 - link

    100W for an hour straight = 0.1KWH. If you figure 10-20 cents per KWH, it's about 1-2 cents per hour for a 100W difference. That's comes to about $7-$14 per month in bills provided that 100W is consistent 24/7.
  • menting - Thursday, February 5, 2015 - link

    pattern recognition is one that comes to mind.
  • Murloc - Thursday, February 5, 2015 - link

    physical restraints of light speed? Isn't any minuscule parasitic capacitance way more speed limiting than that?
  • menting - Thursday, February 5, 2015 - link

    there's tons of limiting factors, with capacitance being one of those. But even if you take pains to optimize those, the one factor that nobody can get around is the speed of light.
  • menting - Thursday, February 5, 2015 - link

    i guess i should say speed of electricity in a conductive medium instead of speed of light.
  • retrospooty - Friday, February 6, 2015 - link

    Agreed if an app required high total bandwidth it would benefit.

    Now see if you can name a few that actually need that.

Log in

Don't have an account? Sign up now