The Intel Xeon E7 v2 Review: Quad Socket, Up to 60 Cores/120 Threads
by Johan De Gelas on February 21, 2014 6:00 AM EST- Posted in
- IT Computing
- Intel
- Xeon
- Ivy Bridge EX
- server
- Brickland
Our Benchmark Choices
To make the comparison more interesting, we decided to include both the Quad Xeon "Westmere-EX" as well as the "Nehalem-EX". Remember these heavy duty, high RAS servers continue to be used for much longer in the data center than their dual socket counterparts. Many people considering the newest Xeon E7-4800 v2 probably still own a Xeon X7500.
Of course, the comparison would not be complete without the latest dual Xeon 2600 v2 server and at least one Opteron based server. Due to the large number of platforms and the fact that we developed a brand new HPC test (see further), we quickly ran out of time. These time constrains and the fact that we have neglected our Linux testing in recent reviews in favor of Windows 2012 and ESXi led to the decision to limit ourself to testing on top of Ubuntu Linux 13.10 (kernel 3.11). You'll see our typical ESXi and Windows benchmarks in a later review.
Benchmark Configuration
There are some differences in the RAM and SSD configurations. The use of different SSDs was due to time constraints as we wanted to test the servers as much as possible in parallel. The RAM configuration differences are a result of the platforms: for example, the quad Intel CPUs only perform at their best when each CPU gets eight DIMMs. The Opteron and Dual Xeon E5-2680 v2 server perform best with one DIMM per channel (1 DPC).
None of these differences have a tangible influence on the results of our benchmarks, as none of them were bottlenecked by the storage system or the amount of RAM that was used. The minimum amount of 64GB of RAM was more than enough for all benchmarks in this review.
We also did not attempt to do power measurements. We will try to do an apples-to-apples power comparison at a later time.
Intel S4TR1SY3Q "Brickland" IVT-EX 4U-server
The latest and greatest from Intel consists of the following components:
| CPU |
4x Xeon E7-4890 v2 (D1 stepping) 2.8GHz 15 cores, 37.5MB L3, 155W TDP |
| RAM |
256GB, 32x8GB Samsung 8GB DDR3 M393B1K70DH0-YK0 at 1333MHz |
| Motherboard | Intel CRB Baseboard "Thunder Ridge" |
| Chipset | Intel C602J |
| PSU | 2x1200W (2+0) |
Total amount of DIMM slots is 96. When using 64GB LRDIMMs, this server can offer up to 6TB of RAM! In some cases, we have tested the E7-4890 v2 at a lower maximum clock in order to do clock-for-clock comparisons with the previous generation, and in a few cases we have also disabled three of the cores in order to simulate performance of some of the 12-core Ivy Bridge EX parts. For example, a E7-4890 v2 at 2.8 GHz with 3 cores disabled (12 cores total) gives you a good idea how the much less expensive E7- 8857 v2 at 3 GHz would perform: it would perform about 7% higher than the 12-core E7-4890 v2.
Intel Quanta QSCC-4R Benchmark Configuration
The previous quad Xeon E7 server, as reviewed here.
| CPU |
4x Xeon X7560 at 2.26GHz or 4x Xeon E7-4870 at 2.4GHz |
| RAM |
16x8GB Samsung 8GB DDR3 M393B1K70DH0-YK0 at 1066MHz |
| Motherboard | QCI QSSC-S4R 31S4RMB00B0 |
| Chipset | Intel 7500 |
| BIOS version | QSSC-S4R.QCI.01.00.S012,031420111618 |
| PSU | 4x850W Delta DPS-850FB A S3F E62433-004 850W |
The server can accept up to 64 32GB Load Reduced DIMMs (LR-DIMMs) or 2TB.
Intel's Xeon E5 server R2208GZ4GSSPP (2U Chassis)
This is the server we used in our Xeon "Ivy bridge EP" review.
| CPU | 2x Xeon processor E5-2680 (2.8GHz, 10c, 25MB L3, 115W) |
| RAM |
128GB (8 x 16GB) Micron MT36JSF2G72PZ – BDDR3-1866 |
| Internal Disks | 2 x Intel MLC SSD710 200GB |
| Motherboard | Intel Server Board S2600GZ "Grizzly Pass" |
| Chipset | Intel C600 |
| BIOS version | SE5C600.86B (August the 6th, 2013) |
| PSU | Intel 750W DPS-750XB A (80+ Platinum) |
The Xeon E5 CPUs have four memory channels per CPU and support up to DDR3-1866, and thus our dual CPU configuration gets eight DIMMs for maximum bandwidth.
Supermicro A+ Opteron server 1022G-URG (1U Chassis)
This Opteron server is not comparable in any way with the featured Intel systems as it is not targeted at the same market and costs a fraction of the other machines. Nevertheless, here's our test configuration.
| CPU | 2x Opteron "Abu Dhabi" 6376 at 2.3GHz |
| RAM | 64GB (8x8GB) DDR3-1600 Samsung M393B1K70DH0-CK0 |
| Motherboard | SuperMicro H8DGU-F |
| Internal Disks | 2 x Intel MLC SSD710 200GB |
| Chipset | AMD Chipset SR5670 + SP5100 |
| BIOS version | R3.5 |
| PSU | SuperMicro PWS-704P-1R 750Watt |
The Opteron server in this review is only here to satisfy curiosity. We want to see how well the Opteron fares in our new Linux benchmarks.
Facebook
Twitter
RSS
125 Comments
View All Comments
colonelclaw - Monday, February 24, 2014 - link
I would like to see V-Ray benchmarked. It's fast becoming an industry standard across a number of 3D industries (started in ArchVis, is now moving into animation feature films and FX)PowerTrumps - Sunday, February 23, 2014 - link
The author is misleading with statements and data not to mention @Brutalizer comes across very knowledgeable but only backs up claims or Oracle server performance with platitudes and boasts.Starting with the article - comparing various cores regardless if you adjust the frequency is misleading. You need to normalize the values to show what the per core improvement is. To stay with sockets is useless and lazy. Yes, Intel customers buy servers by the socket but to understand what they are really gaining this is a much better metric. To say there is a 20 or 30% gain when there might be 50% more cores tells me the per core performance is actually lower than Westmere. This is important when using software like Oracle that would price a 15 core socket at 7.5 or 8 Oracle licenses. For software licensed by the core, customers should demand the highest performance available otherwise all you do is subsidize Uncle Larry's island. For the Power comparisons in the SAP benchmarks. You compare a 60 core to a 32 core N-1 generation Power7 server. Since Power servers scale almost linearly by frequency, the 8 core @ 4.22 GHz is 54,700. If we extrapolate that to a 4 socket or 32 cores we would be around 200K SAPS. That is quite a bit more than the 60 core Dell. Also, you could deploy a Power server as a standalone server. Nobody would deploy a mission critical workload on a standalone x86 server. Yes, I'm sure somebody will argue with me and say they do and have done it for years. Ok, but by and large we know they are always clustered and used to scale-out. Secondly, you claim how expensive the Power servers are. When was the last time you priced one Mr De Gelas? You can get a Power7+ 7R1, 7R2, or 7R4 that has price parity with a typical x86 price that includes Linux and VMware and comparably equipped. The 710 and 730 servers would be just a bit more but definitely competitive. Factor in the software savings and reduction in the number of servers required and the TCA and TCO will favor Power quickly. I do it all of the time and can back it up with hard data. You can run Power servers up to 90% utilization but rarely run x86 over 30%, maybe 35% tops.
With regard to @Brutalizer - Big claims of big servers, up to 96 TB of RAM. Who needs that? Who needs a server with 100's or 1000's of cores? The Oracle M6-32 has 1000 DIMMs to get 32 TB of memory. Tell us how this influences the MTBF of the server since the number of components is a major factor in the calculation. Next, you scoff at IBM for comparing to older servers. That is because they are talking to customers who are running older servers - consolidate those older servers onto a few or just 1 server that is inherently reliable - nothing more than a IBM mainframe followed by a IBM Power servers. Oracles M6-32 and M5-32 are just cables T5 servers scaled back from 16 to 12 cores. They have little RAS and built for marketing hype and to drive Oracle software licensing revenue. You say the Oracle M processor pricing is X and then try to paint picture that Power servers are more expensive for a 32 socket than a 8 socket - really. A v8 luxury car is more expensive than a 4 cyl econobox. The server price is moot when the real cost is the software you run on it. With Oracle EE + RAC at $70,500 + 22% annual maintenance per core it matters. On Power I only have to license the cores I need. If I need 2 cores for Oracle then I license 2 cores. On x86, the 15 core is 8. (15 x .5 = 7.5 rounds to 8). Oracle M series is also .5 so your 128 cores on SAP S&D to my 64 co Power7 at 1.0 puts us about equal. However, most customers don't run the servers with one workload. You will say your LDOMs is efficient but compared to Power Hypervisor it won't hold a candle to efficiently using the cores and threads - all of them in true multi-thread fashion. With Power8 coming out soon both Intel and Oracle will go back to smelling the fumes of Power servers. To customers out there. It isn't about being Ford or Chevy. This isn't college - don't root for your team even when they are no good. Your business has to not only survive but hopefully thrive. Do that on a platform that controls the largest cost which is software and Full Time Equivalents - that is Power servers.
Phil_Oracle - Monday, February 24, 2014 - link
Well I must say that this article is clearly Intel biased with a lot of misleading and downright wrong statements about Oracle and SPARC. Heres some accurate and substantiated counters:"Sun/Oracle's server CPUs have been lagging severely in performance"
This is wrong, considering that since the SPARC T4 release, and now SPARC T5 and SPARC M6 announcements, Oracle has announced 20+ world record benchmarks across *all* of the public, audited benchmarks from TPC-C, TPC-H @ 1TB, 3TB, 10TB to SPECjEnterprise2010 and SPECjbb2013. Many of them are still valid today, almost a year later.
What I'd like to ask, is where are the 8-socket Xeon E7 v2 benchmarks to compare to SPARC? Theres only one today - SAP. And this doesn’t demonstrate database performance nor java application performance.
Theres also no 4-socket or 8-socket benchmarks on TPC-C, TPC-H, SPECjEnterprise2010.
Even with SPECjbb2013, theres just a 4-socket result, and if you compare performance/core, the SPARC T5-2 @ 114,492 max-jOPS (just 32-cores) has a 1.3x performance/core advantage over the NEC Express5800/A040b with 60 x Intel E7-4890 v2 2.8 GHz cores @ 177,753 max-jOPS.
"As usual, the published benchmarks are very vague and are only available for the top models "
As of today, there is not a single real world application/database benchmark that shows Xeon having superior throughput, response times or even price/performance comparing systems with same # of CPUs to SPARC T5. You can go here to see all the comparisons with full transparency. https://blogs.oracle.com/BestPerf/
"and the best performing systems come with astronomic price tags ($950,000 for two servers, some networking, and storage... really?)."
You do realize you are linking to Oracle Exadata which isn't a server but an Engineered system with many servers, storage and networking all built-in and based on XEON??
Why are you not linking to SPARC T5 server pricing which is here since that’s what you are trying to discredit? Heres the SPARC T5-2 pricing which is very aggressively priced to x86 & IBM Power7+ systems.
https://shop.oracle.com/pls/ostore/f?p=dstore:5:90...
Or better yet, look at a public benchmark where full HW and SW pricing is disclosed?
A SPARC T5-4 is 2.4x faster than the 8-socket Xeon E7-4870 based HP DL980 G7 on TPC-H at 10TB.
The SPARC T5-4 server HW fully configured costs $268,853, HP DL 980 costs $268,431.
Basically same costs, SPARC T5 is 2.4x faster than Westmere-EX. Wheres Xeon E7 v2 to showcase its 2x faster??
Details of pricing and results are here.
http://www.tpc.org/tpch/results/tpch_result_detail...
http://www.tpc.org/results/individual_results/orac...
http://c970058.r58.cf2.rackcdn.com/individual_resu...
On TPC-C OLTP benchmark, a SPARC T5-8 has a $/perf of .55USD/tpmC, versus fastest Oracle x2-8 of .89 USD/tpmC and IBM x3850 of .59USD/tpmC. SPARC T5-8 is 70% faster per CPU than Westmere-EX based Oracle x2-8. http://www.tpc.org/tpcc/results/tpcc_results.asp?o...
Haravikk - Tuesday, February 25, 2014 - link
I wouldn't mind a 60 core Xeon in the next version of Apple's Mac Pro ;)Desert Dude - Thursday, April 3, 2014 - link
Interesting discussions. Just for clarification, there is an x86 server that goes beyond 8 sockets-bullion (Xeon E7 48xx up to 16sockets with near linear scale). Bull (legacy GE & Honeywell Mainframe) has leveraged technology used in its mainframe & HPC to build bullion...the world's FASTEST x86 server. bull.us/ bullion