The Intel Xeon E7 v2 Review: Quad Socket, Up to 60 Cores/120 Threads
by Johan De Gelas on February 21, 2014 6:00 AM EST- Posted in
- IT Computing
- Intel
- Xeon
- Ivy Bridge EX
- server
- Brickland
Our Benchmark Choices
To make the comparison more interesting, we decided to include both the Quad Xeon "Westmere-EX" as well as the "Nehalem-EX". Remember these heavy duty, high RAS servers continue to be used for much longer in the data center than their dual socket counterparts. Many people considering the newest Xeon E7-4800 v2 probably still own a Xeon X7500.
Of course, the comparison would not be complete without the latest dual Xeon 2600 v2 server and at least one Opteron based server. Due to the large number of platforms and the fact that we developed a brand new HPC test (see further), we quickly ran out of time. These time constrains and the fact that we have neglected our Linux testing in recent reviews in favor of Windows 2012 and ESXi led to the decision to limit ourself to testing on top of Ubuntu Linux 13.10 (kernel 3.11). You'll see our typical ESXi and Windows benchmarks in a later review.
Benchmark Configuration
There are some differences in the RAM and SSD configurations. The use of different SSDs was due to time constraints as we wanted to test the servers as much as possible in parallel. The RAM configuration differences are a result of the platforms: for example, the quad Intel CPUs only perform at their best when each CPU gets eight DIMMs. The Opteron and Dual Xeon E5-2680 v2 server perform best with one DIMM per channel (1 DPC).
None of these differences have a tangible influence on the results of our benchmarks, as none of them were bottlenecked by the storage system or the amount of RAM that was used. The minimum amount of 64GB of RAM was more than enough for all benchmarks in this review.
We also did not attempt to do power measurements. We will try to do an apples-to-apples power comparison at a later time.
Intel S4TR1SY3Q "Brickland" IVT-EX 4U-server
The latest and greatest from Intel consists of the following components:
CPU |
4x Xeon E7-4890 v2 (D1 stepping) 2.8GHz 15 cores, 37.5MB L3, 155W TDP |
RAM |
256GB, 32x8GB Samsung 8GB DDR3 M393B1K70DH0-YK0 at 1333MHz |
Motherboard | Intel CRB Baseboard "Thunder Ridge" |
Chipset | Intel C602J |
PSU | 2x1200W (2+0) |
Total amount of DIMM slots is 96. When using 64GB LRDIMMs, this server can offer up to 6TB of RAM! In some cases, we have tested the E7-4890 v2 at a lower maximum clock in order to do clock-for-clock comparisons with the previous generation, and in a few cases we have also disabled three of the cores in order to simulate performance of some of the 12-core Ivy Bridge EX parts. For example, a E7-4890 v2 at 2.8 GHz with 3 cores disabled (12 cores total) gives you a good idea how the much less expensive E7- 8857 v2 at 3 GHz would perform: it would perform about 7% higher than the 12-core E7-4890 v2.
Intel Quanta QSCC-4R Benchmark Configuration
The previous quad Xeon E7 server, as reviewed here.
CPU |
4x Xeon X7560 at 2.26GHz or 4x Xeon E7-4870 at 2.4GHz |
RAM |
16x8GB Samsung 8GB DDR3 M393B1K70DH0-YK0 at 1066MHz |
Motherboard | QCI QSSC-S4R 31S4RMB00B0 |
Chipset | Intel 7500 |
BIOS version | QSSC-S4R.QCI.01.00.S012,031420111618 |
PSU | 4x850W Delta DPS-850FB A S3F E62433-004 850W |
The server can accept up to 64 32GB Load Reduced DIMMs (LR-DIMMs) or 2TB.
Intel's Xeon E5 server R2208GZ4GSSPP (2U Chassis)
This is the server we used in our Xeon "Ivy bridge EP" review.
CPU | 2x Xeon processor E5-2680 (2.8GHz, 10c, 25MB L3, 115W) |
RAM |
128GB (8 x 16GB) Micron MT36JSF2G72PZ – BDDR3-1866 |
Internal Disks | 2 x Intel MLC SSD710 200GB |
Motherboard | Intel Server Board S2600GZ "Grizzly Pass" |
Chipset | Intel C600 |
BIOS version | SE5C600.86B (August the 6th, 2013) |
PSU | Intel 750W DPS-750XB A (80+ Platinum) |
The Xeon E5 CPUs have four memory channels per CPU and support up to DDR3-1866, and thus our dual CPU configuration gets eight DIMMs for maximum bandwidth.
Supermicro A+ Opteron server 1022G-URG (1U Chassis)
This Opteron server is not comparable in any way with the featured Intel systems as it is not targeted at the same market and costs a fraction of the other machines. Nevertheless, here's our test configuration.
CPU | 2x Opteron "Abu Dhabi" 6376 at 2.3GHz |
RAM | 64GB (8x8GB) DDR3-1600 Samsung M393B1K70DH0-CK0 |
Motherboard | SuperMicro H8DGU-F |
Internal Disks | 2 x Intel MLC SSD710 200GB |
Chipset | AMD Chipset SR5670 + SP5100 |
BIOS version | R3.5 |
PSU | SuperMicro PWS-704P-1R 750Watt |
The Opteron server in this review is only here to satisfy curiosity. We want to see how well the Opteron fares in our new Linux benchmarks.
125 Comments
View All Comments
Kevin G - Friday, February 21, 2014 - link
And a quick addition:There will indeed be a quick adoption to Haswell-EX not because of AVX2 or DDR4 but rather transactional memory support (TSX). For the large databases and applications these systems are targeted at, TSX should prove to be helpful.
TiGr1982 - Friday, February 21, 2014 - link
I agree, TSX should make a lot of sense for these E7's - they have a huge core count and huge shared memory at the same time.Schmide - Friday, February 21, 2014 - link
I think your L3 latency numbers are off. I think typical Intel L3 latencies are 30-40 clocks ~3-4ns.Schmide - Friday, February 21, 2014 - link
Oops my bad i miss used the calculator. Ignore.dylan522p - Friday, February 21, 2014 - link
No power consumption numbers?JohanAnandtech - Saturday, February 22, 2014 - link
Coming...we had to run lots of test in parallel, so it was not possible to make sure all systems were similar. Also we should test with workloads that require a lot more memory to get an idea.mslasm - Friday, February 21, 2014 - link
Note that E7-8857 v2 has 12 cores but no HT, so only has 12 threads as well (see http://ark.intel.com/products/75254/Intel-Xeon-Pro... Thus it is not equivalent to a 3Ghz E7-4860V2, as 4860 has HT for a total of 24 threadsAlso, there must be a typo either in the graph or in the text on the "single thread" integer performance test: "Opteron ... at 2.4GHz would deliver about 2481 MIPs", while - according to the graph - it already delivers 2636 @ 2.3Ghz.
JohanAnandtech - Saturday, February 22, 2014 - link
Good point. There is little gain from HT in OpenFoam, but it will influence the LZMA benchmarks. So the Openfoam findings are still valid, but not the LZMA. The kernel compile is somewhat in between.JohanAnandtech - Saturday, February 22, 2014 - link
I will rerun the benchmarks without HT to check.mslasm - Saturday, February 22, 2014 - link
Thanks! I did not mean to imply HT matters "a lot", but it may influence some (and I admit I don't know much about how your benchmarks behave, other than parallel LZMA which I worked a lot with) - so it just does not sound right to outright call it equivalent, and I wish AT only has statements anyone can just trust :)