LRDIMMs, RDIMMs, and Supermicro's Latest Twin
by Johan De Gelas on August 3, 2012 4:45 AM EST- Posted in
- IT Computing
- Intel
- Samsung
- Xeon
- Cloud Computing
- Supermicro
The Purpose of this Test
We will not state that we had the most optimal testing configuration for our objectives. The most interesting LRDIMMs are the 32GB ones, and we had the cheaper (and thus easier to borrow) 16GB parts. The most interesting Supermicro Server is the 6027TR-D70RF+; we had the slightly older but very similar 6027TR-D71FRF. As always, we try to make the best with what we have in the lab. We believe that with this testing configuration we can still answer the questions that will pop up when you consider the different server configuration such as:
- How much bandwidth and latency will you sacrifice when buying LRDIMMs instead of RDIMMs?
- Does investing in expensive high capacity DIMMs pay off?
- Can you really host twice as many VMs in twice as much memory?
- How much performance do you gain by giving your VMs more physical memory?
Those two last questions seem silly at first sight but they are not. In most cases, virtual machines get (much) more memory than they really need. Most administrators prefer to give the VM quite a bit of memory headroom. It is a good practice, but it means that it is not necessarily a bad thing if the total amount of virtual memory is quite a bit higher than the total available physical memory. Unless all VMs are working hard, a modern hypervisor can make sure that the "needy" VMs get what they need and the "lazy" ones get only the bare minimum of physical RAM.
Indeed, an advanced hypervisor such as ESXi (especially ESXi) has a lot of tricks up its sleeve to make sure that even if you don't have enough physical memory, your VM will still run fine. Physical memory use is optimized by:
- Transparent page sharing (TPS): the Hypervisor will only claim one page for several pages of several different VMs with identical content (e.g. the Windows kernel and HAL in several Windows based VMs)
- Ballooning: the Hypervisor reclaims memory that a VM does not use and gives it to more "needy" VMs
- Memory compression: pages that need to be swapped out (see further) are checked if they have a high compression ratio. If that is the case they are compressed and kept in a memory cache. As a result...
- Hypervisor swapping: memory that is not active and not compressible can be swapped to disk, similar to a "normal" OS (Supervisors). This does not necessarily result in a large performance hit, as pages swapped out are rarely used.
So we thought it would be interesting to design a scenario where we could measure the performance differences between a system with lots of memory and a more budget limited one.
Facebook
Twitter
RSS
26 Comments
View All Comments
ddr3memory - Monday, August 6, 2012 - link
Your article is very interesting - and the first mainstream (and belated) examination of the LRDIMM (new standard - incompatible with RDIMMs) vs. HCDIMM (100% DDR3 RDIMM compatbile) choice for Romley.I have whittled down the use case for HCDIMMs/LRDIMMs and RDIMMs as follows:
The HCDIMM use case is at:
- 16GB at 3 DPC use
- 32GB (outperform both RDIMMs and LRDIMMs)
LRDIMMs are not viable at:
- 16GB (RDIMMs are better)
- 32GB (HCDIMMs are better)
RDIMMs are not viable at:
- 32GB (because they are 4-rank - trumped by LRDIMMs/HCDIMMs)
There is a reason the Netlist HCDIMMs were only released on the virtualization servers from IBM/HP - because at 16GB levels the only niche available for LRDIMM/HCDIMM vs. RDIMM is the 3 DPC space. This will expand considerably at 32GB to mainstream levels as soon as 32GB HCDIMMs are released (they are currently in qualification with IBM/HP and have not been announced yet - though maybe expected shortly).
I had created an infographic covering the memory choices - search the net for the article entitled:
Infographic - memory buying guide for Romley 2-socket servers
HCDIMMs are not available at SuperMicro (as they are for IBM/HP) - so I was surprised you even covered HCDIMMs (since the article is after all referring to the SuperMicro line of servers).
Casper42 - Friday, August 10, 2012 - link
BTW, Johan, I work for HP and asked some of the guys in ISS Technical Marketing why we don't send you our servers for eval like you get from SuperMicro and sometimes DellThey felt that you guys didn't do alot of Server Reviews, and that your readership wasn't generally the kind of folks that buy HP Servers.
So I am curious if you could spin up a poll or something in the future to prove them wrong.
If there is enough support I'm sure we can you some gear to play with.
I sometimes giggle when I see the stuff people on here get excited about in these reviews though. "Can you see the BIOS through IPMI?". Thats the kind of thing Compaq offered back with the RILOE II and have been integrated into the motherboard since iLO 1 which is like 4 or 5 years old at least.
iLO4 on the Gen8 line have taken that a step further and we now hook the Display system BEFORE POST starts so instead of an invalid memory config getting you a series of beeps, you now get a full blown screen either on local VGA or on the Remote Console that straight up tells you you have a memory mismatch and why. i have seen his demo'd with NO DIMMs even installed in the server and you still get Video and obvious status messages.
Casper42 - Friday, August 10, 2012 - link
Also you are about $2000 high on the HP SL unless I am missing something.I found these prices with QuickSpecs part numbers and Google, nothing magical inside HP.
Half of one of these:
http://www.provantage.com/hewlett-packard-hp-62923...
Includes 8 fans and 3 PS
2 of these
http://www.provantage.com/hewlett-packard-hp-65904...
2x2665 with 8GB
Comes to about $11,600
JohanAnandtech - Tuesday, August 14, 2012 - link
Hey Casper, contact me on my mail... thx!ad99 - Monday, April 1, 2013 - link
You say：a quad rank DIMM with 4Gb chips is a 32GB DIMM (4 Gbit x 8 x 4 ranks)，but I think 4 Gbit x 8 x 4 ranks make only 16GB,is that right?ad99 - Monday, April 1, 2013 - link
No,4 Gbit x 8 x 4 ranks should be 128GB，not 32GB or 16GB,is that right?