Memory Subsystem: Bandwidth

As we have reported before, measuring the full bandwidth potential with John McCalpin's Stream bandwidth benchmark has become a matter of extreme tuning, requiring a very deep understanding of the platform. 

If we used our previous binaries, both the first and second generation EPYC could not get past 200-210 GB/s. It gave the impression of running into a "bandwidth wall", despite the fact that we now had 8-channel DDR4-3200. So we used the results that Intel and AMD best binaries produce using AVX-512 (Intel) and AVX-2 (AMD). 

The results are expressed in gigabytes per second.

Stream Triad

AMD can reach even higher numbers with the setting "number of nodes per socket" (NPS) set to 4. With 4 nodes per socket, AMD reports up to 353 GB/s. NPS4 will cause the CCX to only access the memory controllers with the lowest latency at the central IO Hub chip.

Those numbers only matter to a small niche of carefully AVX(-256/512) optimized HPC applications. AMD claims a 45% advantage compared to the best (28-core) Intel SKUs. We have every reason to believe them but it is only relevant to a niche. 

For the rest of the enterprise world (probably 95+%), memory latency has much larger impact than peak bandwidth. 

Benchmark Configuration and Methodology Memory Subsystem: Latency
POST A COMMENT

184 Comments

View All Comments

  • negusp - Wednesday, August 7, 2019 - link

    hard F in the chat for intel Reply
  • pancakes - Wednesday, August 7, 2019 - link

    F in chat for wallets of people running Windows server Reply
  • azfacea - Wednesday, August 7, 2019 - link

    windows server in 2019 LUL Reply
  • diehardmacfan - Wednesday, August 7, 2019 - link

    on-prem Windows Server is probably at an all time high in 2019? Reply
  • azfacea - Thursday, August 8, 2019 - link

    desperate for a comeback huh? cool hold your 10% tight and gloat about upcoming bfloat16 Reply
  • diehardmacfan - Thursday, August 8, 2019 - link

    Sorry, who is desperate for a comeback? Bring up a floating point format when called out on the ridiculous notion that Windows Server isn't still a large part of the marketplace? say wha Reply
  • mkaibear - Thursday, August 8, 2019 - link

    Just hopping in to say that I am an IT manager for a major employer in the UK and of our 1800 servers more than 80% of them are Windows... this is not a trend which I see changing any time soon. Reply
  • npz - Thursday, August 8, 2019 - link

    Many smaller IT depts in smaller companies use Windows because of familiarity for desktop support such as Active Directory for domains, but none of major critical data center centric, HPC, military, infrastructure are running Windows. Most especially not with EPYC since the Windows scheduler is broken. Reply
  • Manch - Thursday, August 8, 2019 - link

    NPZ, You may be speaking for your bubble, but not for the rest. Reply
  • blaktron - Thursday, August 8, 2019 - link

    this is 100% false. I do infrastructure consulting for 9 figure companies and they are all primarily windows in their corporate infrastructure. all of them. the only linux you will find in the Fortune 50 is legacy applications and web presentation layer. There are exceptions, but that's true enough to form a rule. Reply

Log in

Don't have an account? Sign up now