Hot Chips is an annual conference that allows semiconductor companies to present their latest and greatest ideas or forthcoming products in an academic-style environment, and is predominantly aimed as the professional semiconductor engineer. This year has a number of talks about power management, upcoming IBM CPUs, upcoming Intel CPUs, upcoming NVIDIA SoCs and the final talk of the final day is from AMD, discussing Zen in even more depth than the previous week. While we were unable to attend the event in person, we managed to get some hands on time with information and put questions to Mike Clark, AMD Senior Fellow and design engineer.

What We Learned Last Week: L1/L2/L3 Caches and the Micro-Op Buffer

In AMD’s initial presentation for the general media, we were given a sense of the microarchitecture layout. We covered the material, but it contained a number of highlights.

AMD Zen Microarchitecture: Dual Schedulers, Micro-op Cache and Memory Hierarchy Revealed
AMD Server CPUs and Motherboard Analysis
Unpacking AMD's Zen Benchmark: Is Zen actually 2% Faster than Broadwell?

First up, and the most important, was the announcement of the inclusion of a micro-op cache. This allows for instructions that are frequently used to be closer to the micro-op queue and saves a trip through the core and caches to load the desired information. Typically micro-op caches are still relatively small, and while AMD isn’t giving any information for size and accessibility, we know that Intel’s version can support 1536 uOps with 8-way associativity; we expect AMD’s to be similar though there are many options in play.

Secondly is the cache structure. We were given details for the L1, L2 and L3 cache sizes, along with associativity, to compare it to former microarchitectures as well as Intel’s offering.

CPU Cache Comparison
  Zen
HEDT
Bulldozer
HEDT
Excavator
 
Skylake Broadwell
HEDT
L1-I 64KB/core 64KB/module 96KB/module 32KB/core 32KB/core
4-way 2-way 3-way 8-way 8-way
L1-D 32KB/core 16KB/thread 32KB/thread 32KB/core 32KB/core
8-way 4-way 8-way 8-way 8-way
L2 512KB/core 1MB/thread 512KB/thread 256KB/core 256KB/core
8-way 16-way 16-way 4-way 8-way
L3 2MB/core 1MB/thread - >2MB/core 1.5-3MB/core
16-way 64-way - 16-way 16/20-way
L3 Type Victim Victim - Write-back Write-back

In this case, AMD has given Zen a 64KB L1 Instruction cache per core with 4-way associativity, with a lop-sided 32KB L1 Data cache per core with 8-way associativity. The size and accessibility determines how frequently a cache line is missed, and it is typically a trade-off for die area and power (larger caches require more die area, more associativity usually costs power). The instruction cache, per cycle, can afford a 32byte fetch while the data cache allows for 2x 16-byte loads and one 16-byte store per cycle. AMD stated that allowing two D-cache loads per cycle is more representative of the most workloads that end up with more loads than stores.

The L2 is a large 512 KB, 8-way cache per core. This is double the size of Intel’s 256 KB 4-way cache in Skylake or 256 KB 8-way cache in Broadwell. Typically doubling the cache size affords a 1.414 (square root of 2) better chance of a cache hit, reducing the need to go further out to find data, but comes at the expense of die area. This will have a big impact on a lot of performance metrics, and AMD is promoting faster cache-to-cache transfers than previous generations. Both the L1 and L2 caches are write-back caches, improving over the L1 write-through cache in Bulldozer.

The L3 cache is an 8MB 16-way cache, although at the time last week it was not specified over how many cores this was. From the data release today, we can confirm rumors that this 8 MB cache is split over a four-core module, affording 2 MB of L3 cache per core or 16 MB of L3 cache for the whole 8-core Zen CPU. These two 8 MB caches are separate, so act as a last-level cache per 4-core module with the appropriate hooks into the other L3 to determine if data is needed. As part of the talk today we also learned that the L3 is a pure victim cache for L1/L2 victims, rather than a cache for prefetch/demand data, which tempers the expectations a little but the large L2 will make up for this. We’ll discuss it as part of today’s announcement.

The mid-week release also gave insight into the dual schedulers, one for INT and another for FP, which is different to Intel’s joint scheduler/buffer implementation. The talk at Hot Chips goes into detail about how the dispatch and schedulers operate

The New Information

As part of the Hot Chips presentation, AMD is reaffirming its commitment to at least +40% IPC improvement over Excavator. This has specifically been listed as a throughput goal at an equivalent energy per cycle, resulting in an increase in efficiency. Obviously a number of benefits come from moving the 28nm TSMC process to GloFo’s 14nm FinFET process which is used via a Samsung licence. Both the smaller node and FinFET improvements have been well documented so we won’t go over them here, but AMD is stating that Zen is much more than this as a direct improvement to immediate performance, not just efficiency. While Zen is initially a high-performance x86 core at heart, it is designed to scale all the way from notebooks to supercomputers, or from where the Cat cores (such as Jaguar and Puma) were all the way up to the old Opterons and beyond, all with at least +40% IPC.

The first immediate image out of the presentation is the CPU Complex (a CCX), which shows the Zen core design as a four-CPU cluster with caches. This shows the L2/L3 cache breakdown, and also confirms 2MB of L3 per core with 8 MB of L3 per CCX. It also states that the L3 is mostly exclusive of the L2 cache, which stems from the L3 cache as a victim cache for L2 data. AMD is stating that the protocols involved in the L3 cache design allow each core to access the L3 of each other core with an average (but a range) of latencies.

Over the next few pages, we’ll go through the slides. They detail more information about the application of Simultaneous Multithreading (SMT), New Instructions, the size of various queues and buffers, the back-end of the design, the front-end of the design, fetch, decode, execute, load/store and retire segments.

The High-Level Zen Overview
POST A COMMENT

104 Comments

View All Comments

  • Spunjji - Saturday, August 27, 2016 - link

    You numbers are different to everyone else's. Given that you don't cite any of your sources I believe everyone else. Reply
  • Krysto - Wednesday, August 24, 2016 - link

    I would hope they try to double the cores of Intel for notebooks.

    Dual-core Zen without SMT will DESTROY Intel's Atom-based Celerons and Pentiums at the low-end. There will be absolutely ZERO reason to get a Celeron or Pentium notebooks once Zen appears on the market at that price range.

    But at the Core i3 and Core i5 levels, I was hoping AMD would price a quad-core Zen with no SMT against dual-core Core i3 and Core i5, and a quad-core Zen with SMT against Intel's quad-core (no HT) Core i5, and finally 8-core with and without SMT variants against Intel's quad-core Core i7 chips (with HT).

    If they can basically double the cores compared to what Intel has to offer at around the same price level, and maybe with only slightly worse single-thread performance and slightly worse power consumption, AMD's chips should be a NO-BRAINER. The value would be incredible, and it would push the market towards having powerful quad-core chips by default for most PCs. Intel is going to HATE that, because it would seriously cut into their profits. So AMD could use that strategy to both offer great value products and hurt Intel significantly.
    Reply
  • looncraz - Wednesday, August 24, 2016 - link

    AMD is not seeking the low end, they are trying to redefine AMD as the top-tier CPU company they once were. They are aiming for the top and the bulk of the market.

    Zen+'s 15% IPC improvement over Zen might just give them the performance crown, but I'm sure Intel has taken note and planned accordingly.
    Reply
  • zaza - Wednesday, August 24, 2016 - link

    but the AMD CCX module is a quad core module. i am not sure if it is easy for AMD to just remove two. Reply
  • looncraz - Wednesday, August 24, 2016 - link

    Very easy, you just fuse off the defective core, that's the beauty of independent cores. The core complex just shares a common data bus and third level cache. Disabling a core in the complex will simply have it not ask for data on the common data bus. The L3 cache may or may not be cut down (probably will be). Reply
  • H2323 - Wednesday, August 24, 2016 - link

    "While Zen is initially a high-performance x86 core at heart, it is designed to scale all the way from notebooks to supercomputers, or from where the Cat cores (such as Jaguar and Puma) were all the way up to the old Opterons and beyond, all with at least +40% IPC."

    https://www.youtube.com/watch?v=eUSJfGehKDQ

    In the video its more than 40% across all of internal texting.
    Reply
  • Vigilant007 - Saturday, August 27, 2016 - link

    I don't know if AMD will ever have a major win as far as the PC industry again. Realistically they'll end up focusing on building custom x86 for consoles, and server chips. I can also see them exploiting their ability to do x86 to design custom chips for Apple.

    AMD could end up being a fantastic acquisition target as well.
    Reply
  • Tuna-Fish - Tuesday, August 23, 2016 - link

    From page 3:

    > and L2 with 512 entries and support for 4K and 256K pages only.

    Surely you meant 4k and 2MB pages only?
    Reply
  • deltaFx2 - Tuesday, August 23, 2016 - link

    Ian, an error here: "It also states that the L3 is mostly inclusive of the L2 cache, which stems from the L3 cache as a victim cache for L2 data." A victim L3 is by definition an exclusive cache (as you note elsewhere). Also I don't understand why you have the impression that a victim cache is less efficient than an inclusive cache. As you note, an inclusive cache has to keep duplicate copies of data in L2 and L3 whereas an exclusive cache stores exactly 1 copy (either L2 or L3 but never both). In an exclusive cache hierarchy, a cache block is inserted into the L2, and when evicted, is put into the L3. In an inclusive cache hierarchy, a cache block is inserted both into the L2 and L3. Doesn't the exclusive hierarchy make better use of space? Incidentally, AMD has done exclusive caches since K8 at least. This isn't new. Reply
  • bcronce - Tuesday, August 23, 2016 - link

    Exclusive L3 cache makes better use of space, but requires snooping other core's L2 caches for data. If the L3 cache has all of the data all of the L2 cache has, then you only need to check one place.

    This is important when you're trying to synchronize threads since locks are shared memory locations that each core is attempting to read and update. Common types of thread safe data-structures can take some pretty big performance scaling hits. Of course you can work around this in your data-structure.

    One research paper that I read showed exclusive caches having twice the latency of inclusive when snooping was required. If your data-structure has a scaling that works well up to 16 cores on Intel's inclusive cache, it may cap out around 8 cores on AMD's exclusive, thanks to Amdahl's law.

    Cache snooping gets slower as more cores are added. Gotta check them all.
    Reply

Log in

Don't have an account? Sign up now