Announcement Three: Skylake-X's New L3 Cache Architecture

(AKA I Like Big Cache and I Cannot Lie)

SKU madness aside, there's more to this launch than just the number of cores at what price. Deviating somewhat from their usual pattern, Intel has made some interesting changes to several elements of Skylake-X that are worth discussing. Next is how Intel is implementing the per-core cache.

In previous generations of HEDT processors (as well as the Xeon processors), Intel implemented an three stage cache before hitting main memory. The L1 and L2 caches were private to each core and inclusive, while the L3 cache was a last-level cache covering all cores and that also being inclusive. This, at a high level, means that any data in L2 is duplicated in L3, such that if a cache line is evicted into L2 it will still be present in the L3 if it is needed, rather than requiring a trip all the way out to DRAM. The sizes of the memory are important as well: with an inclusive L2 to L3 the L3 cache is usually several multiplies of the L2 in order to store all the L2 data plus some more for an L3. Intel typically had 256 kilobytes of L2 cache per core, and anywhere between 1.5MB to 3.75MB of L3 per core, which gave both caches plenty of room and performance. It is worth noting at this point that L2 cache is closer to the logic of the core, and space is at a premium.

With Skylake-X, this cache arrangement changes. When Skylake-S was originally launched, we noted that the L2 cache had a lower associativity as it allowed for more modularity, and this is that principle in action. Skylake-X processors will have their private L2 cache increased from 256 KB to 1 MB, a four-fold increase. This comes at the expense of the L3 cache, which is reduced from ~2.5MB/core to 1.375MB/core.

With such a large L2 cache, the L2 to L3 connection is no longer inclusive and now ‘non-inclusive’. Intel is using this terminology rather than ‘exclusive’ or ‘fully-exclusive’, as the L3 will still have some of the L3 features that aren’t present in a victim cache, such as prefetching. What this will mean however is more work for snooping, and keeping track of where cache lines are. Cores will snoop other cores’ L2 to find updated data with the DRAM as a backup (which may be out of date). In previous generations the L3 cache was always a backup, but now this changes.

The good element of this design is that a larger L2 will increase the hit-rate and decrease the miss-rate. Depending on the level of associativity (which has not been disclosed yet, at least not in the basic slide decks), a general rule I have heard is that a double of cache size decreases the miss rate by the sqrt(2), and is liable for a 3-5% IPC uplift in a regular workflow. Thus here’s a conundrum for you: if the L2 has a factor 2 better hit rate, leading to an 8-13% IPC increase, it’s not the same performance as Skylake-S. It may be the same microarchitecture outside the caches, but we get a situation where performance will differ.

Fundamental Realisation: Skylake-S IPC and Skylake-X IPC will be different.

This is something that fundamentally requires in-depth testing. Combine this with the change in the L3 cache, and it is hard to predict the outcome without being a silicon design expert. I am not one of those, but it's something I want to look into as we approach the actual Skylake-X launch.

More things to note on the cache structure. There are many ‘ways’ to do it, one of which I imagined initially is a partitioned cache strategy. The cache layout could be the same as previous generations, but partitions of the L3 were designated L2. This makes life difficult, because then you have a partition of the L2 at the same latency of the L3, and that brings a lot of headaches if the L2 latency has a wide variation. This method would be easy for silicon layout, but hard to implement. Looking at the HCC silicon representation in our slide-deck, it’s clear that there is no fundamental L3 covering all the cores – each core has its partition. That being the case, we now have an L2 at approximately the same size as the L3, at least per core. Given these two points, I fully suspect that Intel is running a physical L2 at 1MB, which will give the design the high hit-rate and consistent low-latency it needs. This will be one feather in the cap for Intel.

Announcement Two: High Core Count Skylake-X Processors Announcement Four: The Other Stuff (AVX-512, Favored Core)
POST A COMMENT

203 Comments

View All Comments

  • WorldWithoutMadness - Tuesday, May 30, 2017 - link

    18C/36T. Soon, AMD'll revise threadripper up to 20C/40T and we'd have two giant fighting of +1/+2 of each others. Reply
  • SunLord - Tuesday, May 30, 2017 - link

    AMD can probably move to 32C/64T at any time they please as Threadripper uses the same socket as the Naples/Epyc server cpu the biggest limiter would be motherboards as they are designed to limits AMD imposed on the Threadripper system such as quad channel and 44 pcie so AMD would have to do some tweaking to make it work with those limiters. Reply
  • tuxfool - Tuesday, May 30, 2017 - link

    AFAIK, it isn't the same socket. Threadripper uses SP3r2 whereas Epyc uses R3 sockets. Reply
  • Samus - Tuesday, May 30, 2017 - link

    Much like Intel's HEDT CPU's use a different socket (2011 opposed to 1151) so it makes sense to have a different socket especially if more PCIe lanes are being introduced.

    Thank you AMD for making Intel interesting again. LOL.
    Reply
  • ddriver - Thursday, June 01, 2017 - link

    The difference is that unlike intel, amd is not going to introduce a new and incompatible socket revision for every CPU revision.

    Obviously, the increased number of i\o requires a new socket, so you are going to have one for mainstream, one for hedt and one for server, that is pretty much inevitable. What is achievable is socket longevity, an aspect in which intel deliberately sucks big time.
    Reply
  • JKflipflop98 - Saturday, June 03, 2017 - link

    You know, I see AMD fanboys all over the internet use this same line and it just befuddles me. Why on Earth do you even care about socket compatibilities? Why would you EVER buy a brand new CPU, then immediately castrate its performance across the board by shoving it into some old and outdated motherboard? If you're really that strapped for cash, why are you spending money on PC upgrades? Reply
  • ddriver - Sunday, June 04, 2017 - link

    It is because you are technologically ignorant. Motherboards do not bottleneck CPUs. Even memory controllers are now integrated in the CPU, so how fast your memory is depends on the CPU, the mobo only provides the slot to plug it into.

    Intel is not deliberately rendering sockets obsolete to maximize performance, but to force people to purchase more mobos, thus more chipsets from them.

    A good mobo can be 300-500+$ investment. That's a significant amount of money to save. It can enable to you get substantially faster CPU or GPU for that saved money.
    Reply
  • JKflipflop98 - Thursday, June 08, 2017 - link

    Amazing how many people here are completely clueless how electronics actually work. Thanks for providing an example there, ddriver. Your moronic posts never cease to entertain. Reply
  • cpupro - Sunday, June 18, 2017 - link

    @JKflipflop98

    Explain why owners of original high-end expensive Intel motherboards, I think it was X99 chipset, were required to buy new motherborad while third-party motherboard manufacturers required only BIOS update to support new revision of Intel CPU for same socket?
    Reply
  • TheinsanegamerN - Monday, June 05, 2017 - link

    "old and outdated" doesnt really apply anymore. The only difference between my p8z77 v-pro and a brand new mobo is the number of USB 3 ports. other than that, it does everything I need it to.

    It isnt the 90s when mobo designs were leaping ahead.
    Reply

Log in

Don't have an account? Sign up now