The Intel 12th Gen Core i9-12900K Review: Hybrid Performance Brings Hybrid Complexity
by Dr. Ian Cutress & Andrei Frumusanu on November 4, 2021 9:00 AM ESTInstruction Changes
Both of the processor cores inside Alder Lake are brand new – they build on the previous generation Core and Atom designs in multiple ways. As always, Intel gives us a high level overview of the microarchitecture changes, as we’ve written in an article from Architecture Day:
At the highest level, the P-core supports a 6-wide decode (up from 4), and has split the execution ports to allow for more operations to execute at once, enabling higher IPC and ILP from workflow that can take advantage. Usually a wider decode consumes a lot more power, but Intel says that its micro-op cache (now 4K) and front-end are improved enough that the decode engine spends 80% of its time power gated.
For the E-core, similarly it also has a 6-wide decode, although split to 2x3-wide. It has a 17 execution ports, buffered by double the load/store support of the previous generation Atom core. Beyond this, Gracemont is the first Atom core to support AVX2 instructions.
As part of our analysis into new microarchitectures, we also do an instruction sweep to see what other benefits have been added. The following is literally a raw list of changes, which we are still in the process of going through. Please forgive the raw data. Big thanks to our industry friends who help with this analysis.
Any of the following that is listed as A|B means A in latency (in clocks) and B in reciprocal throughput (1/instructions).
P-core: Golden Cove vs Cypress Cove
Microarchitecture Changes:
- 6-wide decoder with 32b window: it means code size much less important, e.g. 3 MOV imm64 / clks;(last similar 50% jump was Pentium -> Pentium Pro in 1995, Conroe in 2006 was just 3->4 jump)
- Triple load: (almost) universal
- every GPR, SSE, VEX, EVEX load gains (only MMX load unsupported)
- BROADCAST*, GATHER*, PREFETCH* also gains
- Decoupled double FADD units
- every single and double SIMD VADD/VSUB (and AVX VADDSUB* and VHADD*/VHSUB*) has latency gains
- Another ADD/SUB means 4->2 clks
- Another MUL means 4->3 clks
- AVX512 support: 512b ADD/SUB rec. throughput 0.5, as in server!
- exception: half precision ADD/SUB handled by FMAs
- exception: x87 FADD remained 3 clks
- Some form of GPR (general purpose register) immediate additions treated as NOPs (removed at the "allocate/rename/move ellimination/zeroing idioms" step)
- LEA r64, [r64+imm8]
- ADD r64, imm8
- ADD r64, imm32
- INC r64
- Is this just for 64b addition GPRs?
- eliminated instructions:
- MOV r32/r64
- (V)MOV(A/U)(PS/PD/DQ) xmm, ymm
- 0-5 0x66 NOP
- LNOP3-7
- CLC/STC
- zeroing idioms:
- (V)XORPS/PD, (V)PXOR xmm, ymm
- (V)PSUB(U)B/W/D/Q xmm
- (V)PCMPGTB/W/D/Q xmm
- (V)PXOR xmm
Faster GPR instructions (vs Cypress Cove):
- LOCK latency 20->18 clks
- LEA with scale throughput 2->3/clk
- (I)MUL r8 latency 4->3 clks
- LAHF latency 3->1 clks
- CMPS* latency 5->4 clks
- REP CMPSB 1->3.7 Bytes/clock
- REP SCASB 0.5->1.85 Bytes/clock
- REP MOVS* 115->122 Bytes/clock
- CMPXVHG16B 20|20 -> 16|14
- PREFETCH* throughput 1->3/clk
- ANDN/BLSI/BLSMSK/BLSR throughput 2->3/clock
- SHA1RNDS4 latency 6->4
- SHA1MSG2 throughput 0.2->0.25/clock
- SHA256MSG2 11|5->6|2
- ADC/SBB (r/e)ax 2|2 -> 1|1
Faster SIMD instructions (vs Cypress Cove):
- *FADD xmm/ymm latency 4->3 clks (after MUL)
- *FADD xmm/ymm latency 4->2 clks(after ADD)
- * means (V)(ADD/SUB/ADDSUB/HADD/HSUB)(PS/PD) affected
- VADD/SUB/PS/PD zmm 4|1->3.3|0.5
- CLMUL xmm 6|1->3|1
- CLMUL ymm, zmm 8|2->3|1
- VPGATHERDQ xmm, [xm32], xmm 22|1.67->20|1.5 clks
- VPGATHERDD ymm, [ym32], ymm throughput 0.2 -> 0.33/clock
- VPGATHERQQ ymm, [ym64], ymm throughput 0.33 -> 0.50/clock
Regressions, Slower instructions (vs Cypress Cove):
- Store-to-Load-Forward 128b 5->7, 256b 6->7 clocks
- PAUSE latency 140->160 clocks
- LEA with scale latency 2->3 clocks
- (I)DIV r8 latency 15->17 clocks
- FXCH throughput 2->1/clock
- LFENCE latency 6->12 clocks
- VBLENDV(B/PS/PD) xmm, ymm 2->3 clocks
- (V)AESKEYGEN latency 12->13 clocks
- VCVTPS2PH/PH2PS latency 5->6 clocks
- BZHI throughput 2->1/clock
- VPGATHERDD ymm, [ym32], ymm latency 22->24 clocks
- VPGATHERQQ ymm, [ym64], ymm latency 21->23 clocks
E-core: Gracemont vs Tremont
Microarchitecture Changes:
- Dual 128b store port (works with every GPR, PUSH, MMX, SSE, AVX, non-temporal m32, m64, m128)
- Zen2-like memory renaming with GPRs
- New zeroing idioms
- SUB r32, r32
- SUB r64, r64
- CDQ, CQO
- (V)PSUBB/W/D/Q/SB/SW/USB/USW
- (V)PCMPGTB/W/D/Q
- New ones idiom: (V)PCMPEQB/W/D/Q
- MOV elimination: MOV; MOVZX; MOVSX r32, r64
- NOP elimination: NOP, 1-4 0x66 NOP throughput 3->5/clock, LNOP 3, LNOP 4, LNOP 5
Faster GPR instructions (vs Tremont)
- PAUSE latency 158->62 clocks
- MOVSX; SHL/R r, 1; SHL/R r,imm8 tp 1->0.25
- ADD;SUB; CMP; AND; OR; XOR; NEG; NOT; TEST; MOVZX; BSSWAP; LEA [r+r]; LEA [r+disp8/32] throughput 3->4 per clock
- CMOV* throughput 1->2 per clock
- RCR r, 1 10|10 -> 2|2
- RCR/RCL r, imm/cl 13|13->11|11
- SHLD/SHRD r1_32, r1_32, imm8 2|2 -> 2|0.5
- MOVBE latency 1->0.5 clocks
- (I)MUL r32 3|1 -> 3|0.5
- (I)MUL r64 5|2 -> 5|0.5
- REP STOSB/STOSW/STOSD/STOSQ 15/8/12/11 byte/clock -> 15/15/15/15 bytes/clock
Faster SIMD instructions (vs Tremont)
- A lot of xmm SIMD throughput is 4/clock instead of theoretical maximum(?) of 3/clock, not sure how this is possible
- MASKMOVQ throughput 1 per 104 clocks -> 1 per clock
- PADDB/W/D; PSUBB/W/D PAVGB/PAVGW 1|0.5 -> 1|.33
- PADDQ/PSUBQ/PCMPEQQ mm, xmm: 2|1 -> 1|.33
- PShift (x)mm, (x)mm 2|1 -> 1|.33
- PMUL*, PSADBW mm, xmm 4|1 -> 3|1
- ADD/SUB/CMP/MAX/MINPS/PD 3|1 -> 3|0.5
- MULPS/PD 4|1 -> 4|0.5
- CVT*, ROUND xmm, xmm 4|1 -> 3|1
- BLENDV* xmm, xmm 3|2 -> 3|0.88
- AES, GF2P8AFFINEQB, GF2P8AFFINEINVQB xmm 4|1 -> 3|1
- SHA256RNDS2 5|2 -> 4|1
- PHADD/PHSUB* 6|6 -> 5|5
Regressions, Slower (vs Tremont):
- m8, m16 load latency 4->5 clocks
- ADD/MOVBE load latency 4->5 clocks
- LOCK ADD 16|16->18|18
- XCHG mem 17|17->18|18
- (I)DIV +1 clock
- DPPS 10|1.5 -> 18|6
- DPPD 6|1 -> 10|3.5
- FSIN/FCOS +12% slower
Facebook
Twitter
RSS
474 Comments
View All Comments
Oxford Guy - Sunday, November 7, 2021 - link
‘or maybe switch off their E-cores and enable AVX-512 in BIOS’This from exactly the same person who posted, just a few hours ago, that it’s correct to note that that option can disappear and/or be rendered non-functional.
I am reminded of your contradictory posts about ECC where you mocked advocacy for it (‘advocacy’ being merely its mention) and proceeded to claim you ‘wish’ for more ECC support.
Once again, it’s helpful to have a grasp of what one actually believes prior to posting. Allocating less effort to posting puerile insults and more toward substance is advised.
mode_13h - Sunday, November 7, 2021 - link
> This from exactly the same person who posted, just a few hours ago, that it’s> correct to note that that option can disappear and/or be rendered non-functional.
You need to learn to distinguish between what Intel has actually stated vs. the facts as we wish them to be. In the previous post you reference, I affirmed your acknowledgement that the capability disappearing would be consistent with what Intel has actually said, to date.
In the post above, I was leaving open the possibility that *maybe* Intel is actually "cool" with there being a BIOS option to trade AVX-512 for E-cores. We simply don't know how Intel feels about that, because (to my knowledge) they haven't said.
When I clarify the facts as they stand, don't confuse that with my position on the facts as I wish them to be. I can simultaneously acknowledge one reality, which maintaining my own personal preference for a different reality.
This is exactly what happened with the ECC situation: I was clarifying Intel's practice, because your post indicated uncertainty about that fact. It was not meant to convey my personal preference, which I later added with a follow-on post.
Having to clarify this to an "Oxford Guy" seems a bit surprising, unless you meant like Oxford Mississippi.
> you mocked advocacy
It wasn't mocking. It was clarification. And your post seemed more to express befuddlement than expressive of advocacy. It's now clear that your post was a poorly-executed attempt at sarcasm.
Once again, it's helpful not to have your ego so wrapped up in your posts that you overreact when someone tries to offer a factual clarification.
Oxford Guy - Monday, November 8, 2021 - link
I now skip to the bottom of your posts If I see more of the same preening and posing, I spare myself the rest of the nonsense.mode_13h - Tuesday, November 9, 2021 - link
> If I see more of the same preening and posing, I spare myself the rest of the nonsense.Then I suggest you don't read your own posts.
I can see that you're highly resistant to reason and logic. Whenever I make a reasoned reply, you always hit back with some kind of vague meta-critique. If that's all you've got, it can be seen as nothing less than a concession.
O-o-o-O - Saturday, November 6, 2021 - link
Anyone talking about dumping x64 ISA?I don't see AVX-512 a good solution. Current x64 chips are putting so much complexity in CPU with irrational clock speed that migrating process-node further into Intel4 on would be a nightmare once again.
I believe most of the companies with in-house developers expect the end of Xeon-era is quite near, as most of the heavy computational tasks are fully optimized for GPUs and that you don't want coal burning CPUs.
Even if it doesn't come in 5 year time-frame, there's a real threat and have to be ahead of time. After all, x86 already extended its life 10+ years when it could have been discontinued. Now it's really a dinosaur. If so, non-server applications would follow the route as well.
We want more simple / solid / robust base with scalability. Not an unreliable boost button that sometimes do the trick.
SystemsBuilder - Saturday, November 6, 2021 - link
I don't see AVX-512 that negatively it is just the same as AVX2 but double the vectors size and a with a richer instruction set. I find it pretty cool to work with especially when you've written some libraries that can take advantage of it. As I wrote before, it looks like Golden cove got AVX-512 right based on what Ian and Andrei uncovered. 0 negative offset (e.g. running at full speed), power consumption not much more than AVX2, and it supports both FP16 and BP16 vectors! I think that's pretty darn good! I can work with that! Now I want my Sapphire rapids with 32 or 48 Golden cove P cores! No not fall 2022 i want it now! lolmode_13h - Saturday, November 6, 2021 - link
> When you optimize code today (for pre Alder lake CPUs) to take advantage> of AVX-512 you need to write two paths (at least).
Ah, so your solution depends on application software changes, specifically requiring them to do more work. That's not viable for the timeframe of concern. And especially not if its successor is just going to add AVX-512 to the E-cores, within a year or so.
> There are many levels of AVX-512 support and effectively you need write customized
> code for each specific CPUID
But you don't expect the capabilities to change as a function of which thread is running, or within a program's lifetime! What you're proposing is very different. You're proposing to change the ABI. That's a big deal!
> It is absolutely possible and it will come with time.
Or not. ARM's SVE is a much better solution.
> I think in the future P and E cores might have more than just AVX-512 that is different
On Linux, using AMX will require a thread to "enable" it. This is a little like what you're talking about. AMX is a big feature, though, and unlike anything else. I don't expect to start having to enable every new ISA extension I want to use, or query how many hyperthreads actually support - this becomes a mess when you start dealing with different libraries that have these requirements and limitations.
Intel's solution isn't great, but it's understandable and it works. And, in spite of it, they still delivered a really nice-performing CPU. I think it's great if technically astute users have/retain the option to trade E-cores for AVX-512 (via BIOS), but I think it's kicking a hornets nest to go down the path of having a CPU with asymmetrical capabilities among its cores.
Hopefully, Raptor Lake just adds AVX-512 to the E-cores and we can just let this issue fade into the mists of time, like other missteps Intel & others have made.
SystemsBuilder - Saturday, November 6, 2021 - link
I too believe AVX-512 exclusion in the E cores it is transitory. next gen E cores may include it and the issue goes away for AVX-512 at least (Raptor Lake?). Still there will be other features that P have but E won't have so the scheduler needs to be adjusted for that. This will continue to evolve with every generation of E and P cores - because they are here to stay.I read somewhere a few months ago but right now i do not remember where (maybe on Anandtech not sure) that the AVX-512 transistor budget is quite small (someone measured it on the die) so not really a big issue in terms of area.
AMX is interesting because where AVX-512 are 512 bit vectors, AMX is making that 512x512 bit matrices or tiles as intel calls it. Reading the spec on AMX you have BF16 tiles which is awesome if you're into neural nets. Of course gpus will still perform better with matrix calculations (multiplications) but the benefit with AMX is that you can keep both the general CPU code and the matrix specific code inside the CPU and can mix the code seamlessly and that's gonna be very cool - you cut out the latency between GPU and CPU (and no special GPU API's are needed). but of course you can still use the GPU when needed (sometimes it maybe faster to just do a matrix- matrix add for instance just inside the CPU with the AMX tiles) - more flexibility.
Anyway, I do think we will run into a similar issue with AMX as we have the AVX-512 on Alder Lake and therefore again the scheduler needs to become aware of each cores capabilities and each piece of code need to state what type of core they prefer to run on: AVX2, AVX-512, AMX capable core etc (the compliers job). This way the scheduler can do the best job possible with every thread.
There will be some teething for a while but i think this is the direction it is going.
mode_13h - Sunday, November 7, 2021 - link
The difference is that AMX is new. It's also much more specialized, as you point out. But that means that they can place new hoops for code to jump through, in order to use it.It's very hard to put a cat like AVX-512 back in the bag.
SystemsBuilder - Saturday, November 6, 2021 - link
To be clear, I also want to add that the way code is written today (in my organization) pre Alder Lake code base. Every time we write a code path for AVX512 we need to write a fallback code path incase the CPU is not AVX-512 capable. This is standard (unless you can control the execution H/W 100% - i.e. the servers).Does not mean all code has to be duplicated but the inner loops where the 80%/20% rule (i.e. 20% of the code that consumes 80% of the time, which in my experience often becomes like the 99%/1% rule) comes into play that's where you write two code paths:
1 for AVX-512 in case it CPU is capable and
2 with just AVX2 in case CPU is not capable
mostly this ends up being just as I said the inner most loops, and there are excellent broadly available templates to use for this.
Just from a pure comp sci perspective it is quite interesting to vectorize code and see the benefits - pretty cool actually.