AMD Zen Microarchiture Part 2: Extracting Instruction-Level Parallelism
by Ian Cutress on August 23, 2016 8:45 PM EST- Posted in
- CPUs
- AMD
- x86
- Zen
- Microarchitecture
Hot Chips is an annual conference that allows semiconductor companies to present their latest and greatest ideas or forthcoming products in an academic-style environment, and is predominantly aimed as the professional semiconductor engineer. This year has a number of talks about power management, upcoming IBM CPUs, upcoming Intel CPUs, upcoming NVIDIA SoCs and the final talk of the final day is from AMD, discussing Zen in even more depth than the previous week. While we were unable to attend the event in person, we managed to get some hands on time with information and put questions to Mike Clark, AMD Senior Fellow and design engineer.
What We Learned Last Week: L1/L2/L3 Caches and the Micro-Op Buffer
In AMD’s initial presentation for the general media, we were given a sense of the microarchitecture layout. We covered the material, but it contained a number of highlights.
AMD Zen Microarchitecture: Dual Schedulers, Micro-op Cache and Memory Hierarchy Revealed
AMD Server CPUs and Motherboard Analysis
Unpacking AMD's Zen Benchmark: Is Zen actually 2% Faster than Broadwell?
First up, and the most important, was the announcement of the inclusion of a micro-op cache. This allows for instructions that are frequently used to be closer to the micro-op queue and saves a trip through the core and caches to load the desired information. Typically micro-op caches are still relatively small, and while AMD isn’t giving any information for size and accessibility, we know that Intel’s version can support 1536 uOps with 8-way associativity; we expect AMD’s to be similar though there are many options in play.
Secondly is the cache structure. We were given details for the L1, L2 and L3 cache sizes, along with associativity, to compare it to former microarchitectures as well as Intel’s offering.
| CPU Cache Comparison | |||||
| Zen HEDT |
Bulldozer HEDT |
Excavator |
Skylake | Broadwell HEDT |
|
| L1-I | 64KB/core | 64KB/module | 96KB/module | 32KB/core | 32KB/core |
| 4-way | 2-way | 3-way | 8-way | 8-way | |
| L1-D | 32KB/core | 16KB/thread | 32KB/thread | 32KB/core | 32KB/core |
| 8-way | 4-way | 8-way | 8-way | 8-way | |
| L2 | 512KB/core | 1MB/thread | 512KB/thread | 256KB/core | 256KB/core |
| 8-way | 16-way | 16-way | 4-way | 8-way | |
| L3 | 2MB/core | 1MB/thread | - | >2MB/core | 1.5-3MB/core |
| 16-way | 64-way | - | 16-way | 16/20-way | |
| L3 Type | Victim | Victim | - | Write-back | Write-back |
In this case, AMD has given Zen a 64KB L1 Instruction cache per core with 4-way associativity, with a lop-sided 32KB L1 Data cache per core with 8-way associativity. The size and accessibility determines how frequently a cache line is missed, and it is typically a trade-off for die area and power (larger caches require more die area, more associativity usually costs power). The instruction cache, per cycle, can afford a 32byte fetch while the data cache allows for 2x 16-byte loads and one 16-byte store per cycle. AMD stated that allowing two D-cache loads per cycle is more representative of the most workloads that end up with more loads than stores.
The L2 is a large 512 KB, 8-way cache per core. This is double the size of Intel’s 256 KB 4-way cache in Skylake or 256 KB 8-way cache in Broadwell. Typically doubling the cache size affords a 1.414 (square root of 2) better chance of a cache hit, reducing the need to go further out to find data, but comes at the expense of die area. This will have a big impact on a lot of performance metrics, and AMD is promoting faster cache-to-cache transfers than previous generations. Both the L1 and L2 caches are write-back caches, improving over the L1 write-through cache in Bulldozer.
The L3 cache is an 8MB 16-way cache, although at the time last week it was not specified over how many cores this was. From the data release today, we can confirm rumors that this 8 MB cache is split over a four-core module, affording 2 MB of L3 cache per core or 16 MB of L3 cache for the whole 8-core Zen CPU. These two 8 MB caches are separate, so act as a last-level cache per 4-core module with the appropriate hooks into the other L3 to determine if data is needed. As part of the talk today we also learned that the L3 is a pure victim cache for L1/L2 victims, rather than a cache for prefetch/demand data, which tempers the expectations a little but the large L2 will make up for this. We’ll discuss it as part of today’s announcement.
The mid-week release also gave insight into the dual schedulers, one for INT and another for FP, which is different to Intel’s joint scheduler/buffer implementation. The talk at Hot Chips goes into detail about how the dispatch and schedulers operate
The New Information
As part of the Hot Chips presentation, AMD is reaffirming its commitment to at least +40% IPC improvement over Excavator. This has specifically been listed as a throughput goal at an equivalent energy per cycle, resulting in an increase in efficiency. Obviously a number of benefits come from moving the 28nm TSMC process to GloFo’s 14nm FinFET process which is used via a Samsung licence. Both the smaller node and FinFET improvements have been well documented so we won’t go over them here, but AMD is stating that Zen is much more than this as a direct improvement to immediate performance, not just efficiency. While Zen is initially a high-performance x86 core at heart, it is designed to scale all the way from notebooks to supercomputers, or from where the Cat cores (such as Jaguar and Puma) were all the way up to the old Opterons and beyond, all with at least +40% IPC.
The first immediate image out of the presentation is the CPU Complex (a CCX), which shows the Zen core design as a four-CPU cluster with caches. This shows the L2/L3 cache breakdown, and also confirms 2MB of L3 per core with 8 MB of L3 per CCX. It also states that the L3 is mostly exclusive of the L2 cache, which stems from the L3 cache as a victim cache for L2 data. AMD is stating that the protocols involved in the L3 cache design allow each core to access the L3 of each other core with an average (but a range) of latencies.
Over the next few pages, we’ll go through the slides. They detail more information about the application of Simultaneous Multithreading (SMT), New Instructions, the size of various queues and buffers, the back-end of the design, the front-end of the design, fetch, decode, execute, load/store and retire segments.
Facebook
Twitter
RSS
106 Comments
View All Comments
extide - Monday, August 29, 2016 - link
No,k dude, it's not the same 4 ALU's, it's 4 ALU's per core. 2 threads a core, so 2 ALU's/thread, up to 16 threads, or 4 ALU's /thread up to 8 threads, but I would think it would be hard for a single thread to use 4 ALU's, so having 2 threads per 4 ALU seems fine, plus all the INT execution resources.Outlander_04 - Thursday, August 25, 2016 - link
40% improvement is not over Bulldozer but over Excavator which is already 20% or more ahead of Bulldozerlooncraz - Wednesday, August 24, 2016 - link
Highly scalar code or vector code will exceed 40% easily. The core execution resources relative to that execution is 75% to 100% greater. That will only translate to 50~60% performance improvement for said code, but a larger impact than the overall 40% improvement.The cache system, schedulers, issue width, AGUs, L/S, and other factors come more into play in the more common code paths, which reduces the maximum potential benefit derived from the additional execution resources.
However, multi-threaded performance should be HIGHER, not lower. Excavator had relatively poor MT scaling, Zen will be worlds better. Add SMT to the mix - and AMD's solution looks nearly exactly as I anticipated - and you have another 20% or so better SMT scaling.
It is easily conceivable, given what we now know, that AMD has met Haswell's average IPC outside of wider AVX workloads, and exceeded it in certain areas with heavy mixed compute (floating and integer concurrently). It is also now conceivable that AMD's first SMT implementation will be better than Intel's Sandy Bridge era Hyper-Threading. I didn't expect that at all, but the core flexibility is far ahead of Intel's flexibility - and that is largely what determines SMT performance in Zen's design.
Finally, <3Ghz @ 200W is way worse than the currently known figures for their 8C parts. They have 3.2Ghz boost clocks and just 95W TDP. It is expected that the clocks will increase, particularly for the quad core, 65W, parts.
You may not realize this, but these numbers put AMD slightly ahead of Intel in perf/W on 14nm.
niva - Wednesday, August 24, 2016 - link
So are you telling me my Phenom 2 black edition rig might be getting a worthy upgrade?I'm with you, but I don't trust these benchmarks, wait until the retail CPU samples are out then we can decide.
looncraz - Wednesday, August 24, 2016 - link
I'm saying you'll be able to match that level of performance with a Dual core Zen CPU w/ SMT... if AMD were actually to make one (doubtful).I do expect AMD to release triple core CPUs again, though, but possibly not right away.
Myrandex - Thursday, August 25, 2016 - link
Yay finally I've been holding onto my Phenom II as well and this might be it! :)Bulat Ziganshin - Thursday, August 25, 2016 - link
For vector code - they added 4'th ALU, it's almost nothing (Skylake added 4th scalar ALU and got laughable +3% IPC).For scalar code - they advertize +40% IPC. I'm pretty sure that they advertize the best part of perfromance, not the average one. It's ADVERTIZEMENT, after all.
Now, it's easy to analyze Zen as Carrizo+. M/t performance shouldn't change much since it's still 4-wide core (which was called module in Carrizo). S/t performance should improve much more since it changed from 2 alu to 4 alu. Overall, the core looks like Skylake, but it's not enough to put a lot of resources - they need to be carefully placed. Intel gone a long way optimizing their CPUs, and AMD have to repeat that. If you think that AMD can make Skylake-speed CPU in 2 years, then ask yourself - why Intel hasn't done the same in 2008 or so? Why IBM, having WIDER cpu, still slower than Intel in s/t tests?
All we know that AMD was able to SELECT single CPU that was able to run at 3 GHz using cooler looking like one they ship with 95W cpus. Just ask yourself - why they not tried to run their cpu at the same 3.2 GHz which is stock freq. for Intel CPU? And yes, it's way more effificent than Intel CPUs can, making me highly suspicious.
In one of pictures here AMD claims that Zen has the same power usage as Carrizo, that is 28nm CPU. AFAIR Carrizo with 2 modules at ~3GHz use 35-65 Wt. Multiple it by 4, please.
> It is also now conceivable that AMD's first SMT implementation will be better than Intel's Sandy Bridge era Hyper-Threading.
Why?? Intel's first SMT implementation in Pentium4 made a few percents improvement (over s/t), second one in Nehalem give me +20% on deflate, Sandy was +40%, and Haswell is +50%. Why you think that FIRST AMD attempt on SMT will be better than Pentium4?
Overall, i think that m/t perfromance of Zen is more predictable - it's Carrizo with some improvements, but still 4-wide, so i expect usual 10-20% generation-to-generation improvement.
For s/t, it less predictable, but i'm sure that it's impossible to beat Intel in single step, and AMD already advertized +40%, which i'm sure is about s/t perfromance.
looncraz - Thursday, August 25, 2016 - link
"For vector code - they added 4'th ALU, it's almost nothing (Skylake added 4th scalar ALU and got laughable +3% IPC)."Well, that was the average program performance increase, but the vector code itself sped up more than that.
Also, Zen's ability to leverage its resources should be better than Intel's, but its scheduler setup is really unique, so we need more details on how it will handle holes in a scheduler when its neighbor is full. Having six 14-deep schedulers is a significant part of the design that is almost completely overlooked, IMHO.
"Now, it's easy to analyze Zen as Carrizo+. M/t performance shouldn't change much since it's still 4-wide core"
Only if you are comparing a full module to a single Zen core... There were many bottlenecks in the modules that prevented full performance for multi-threading - Zen does not have that. On top, Zen has SMT, so it will have even better MT performance per core.
"Why IBM, having WIDER cpu, still slower than Intel in s/t tests?"
The width is, as you say, only a part of the equation. It's all about being able to exploit that extra width. Intel does so decently well, but has restrictions as a result of their unified scheduler. A heavy FPU load reduces integer performance, for example, due to shared ports of the scheduler. The impact of this is not easily quantifiable - it would require some very specialized testing. Zen will not have this issue thanks to dedicated schedulers.
Intel uses their unified scheduler to be able to provide results more quickly to dependent instructions. Zen, from appearances, allows each scheduler to make fetch and load requests directly, thereby nullifying what used to be an Intel advantage - and maybe even turning it into a hindrance.
"Just ask yourself - why they not tried to run their cpu at the same 3.2 GHz which is stock freq. for Intel CPU?"
Because you don't push engineering sample CPUs, and 3Ghz is the defacto industry standard speed for IPC comparison testing. Just look around, you'll find 3Ghz is the most commonly chosen frequency when doing IPC comparisons on modern CPUs. Pushing both to 3.2Ghz would not have changed anything, but a Zen engineering sample chip is worth thousands more than that Intel CPU at this time, and is not easily replaceable. If you have to run 500 more tests with it, and hand it over to other departments or teams, you probably aren't being allowed to overclock it any.
deltaFx2 - Friday, August 26, 2016 - link
The answer to the IBM question is easy. 1) IBM designed the Power8 with SMT-2 as the sweet spot. Like bulldozer, or Alpha EV6, they have execution clusters. In 2T, each cluster runs a thread, in 1T, the thread is split across these clusters, with a penalty for moving between them. Hence their 1T->2T uplift is a lot higher than intel's 1T->2T (worse baseline). (2) You're comparing different ISAs. x86 is a lot more CISC'y than POWER. x86 supports load+compute, compute+store, load+compute+store, and this is dispatched as a single uop. The same "work" in a more RISC'y machine needs 2 or 3 uops. For the same reason, an ARM core that hopes to achieve the same performance as x86 will need to dispatch more ops, or fuse more ops before dispatch.Spunjji - Saturday, August 27, 2016 - link
The CPU they tasted with is an early engineering sample. Simple answer. You write a lot to make yourself sound smart but you're exercising either clear bias or ignorance here.