CCX Size

Moving down in node size brings up a number of challenges in the core and beyond. Even disregarding power and frequency, the ability to put structures into silicon and then integrate that silicon into the package, as well as providing power to the right parts of the silicon through the right connections becomes an exercise in itself. AMD gave us some insight into how 7nm changed some of its designs, as well as the packaging challenges therein.

A key metric given up by AMD relates to the core complex: four cores, the associated core structures, and then L2 and L3 caches. With 12nm and the Zen+ core, AMD stated that a single core complex was ~60 square millimeters, which separates into 44mm2 for the cores and 16mm2 for the 8MB of L3 per CCX. Add two of these 60mm2 complexes with a memory controller, PCIe lanes, four IF links, and other IO, and a Zen+ zeppelin die was 213 mm2 in total.

For Zen 2, a single chiplet is 74mm2, of which 31.3 mm2 is a core complex with 16 MB of L3. AMD did not breakdown this 31.3 number into cores and L3, but one might imagine that the L3 might be approaching 50% of that number. The reason the chiplet is so much smaller is that it doesn’t need memory controllers, it only has one IF link, and has no IO, because all of the platform requirements are on the IO die. This allows AMD to make the chiplets extremely compact. However if AMD intends to keep increasing the L3 cache, we might end up with most of the chip as L3.

Overall however, AMD has stated that the CCX (cores plus L3) has decreased in size by 47%. That is showing great scaling, especially if the +15% raw instruction throughput and increased frequency comes into play. Performance per mm2 is going to be a very exciting metric.

Packaging

With Matisse staying in the AM4 socket, and Rome in the EPYC socket, AMD stated that they had to make some bets on its packaging technology in order to maintain compatibility. Invariably some of these bets end up being tradeoffs for continual support, however AMD believes that the extra effort has been worth the continued compatibility.

One of the key points AMD spoke about with relation to packaging is how each of the silicon dies are attached to the package. In order to enable a pin-grid array desktop processor, the silicon has to be affixed to the processor in a BGA fashion. AMD stated that due to the 7nm process, the bump pitch (the distance between the solder balls on the silicon die and package) reduced from 150 microns on 12nm to 130 microns on 7nm. This doesn’t sound like much, however AMD stated that there are only two vendors in the world with technology sufficient to do this. The only alternative would be to have a bigger bit of silicon to support a larger bump pitch, ultimately leading to a lot of empty silicon (or a different design paradigm).

One of the ways in order to enable the tighter bump pitch is to adjust how the bumps are processed on the underside of the die. Normally a solder bump on a package is a blob/ball of lead-free solder, relying on the physics of surface tension and reflow to ensure it is consistent and regular. In order to enable the tighter bump pitches however, AMD had to move to a copper pillar solder bump topology.

In order to enable this feature, copper is epitaxially deposited within a mask in order to create a ‘stand’ on which the reflow solder sits. Due to the diameter of the pillar, less solder mask is needed and it creates a smaller solder radius. AMD also came across another issue, due to its dual die design inside Matisse: if the IO die uses standard solder bump masks, and the chiplets use copper pillars, there needs to be a level of height consistency for integrated heat spreaders. For the smaller copper pillars, this means managing the level of copper pillar growth.

AMD explained that it was actually easier to manage this connection implementation than it would be to build different height heatspreaders, as the stamping process used for heatspreaders would not enable such a low tolerance. AMD expects all of its 7nm designs in the future to use the copper pillar implementation.

Routing

Beyond just putting the silicon onto the organic substrate, that substrate has to manage connections between the die and externally to the die. AMD had to increase the number of substrate layers in the package to 12 for Matisse in order to handle the extra routing (no word on how many layers are required in Rome, perhaps 14). This also becomes somewhat complicated for single core chiplet and dual core chiplet processors, especially when testing the silicon before placing it onto the package.

From the diagram we can clearly see the IF links from the two chiplets going to the IO die, with the IO die also handling the memory controllers and what looks like power plane duties as well. There are no in-package links between the chiplets, in case anyone was still wondering: the chiplets have no way of direct communication – all communication between chiplets is handled through the IO die.

AMD stated that with this layout they also had to be mindful of how the processor was placed in the system, as well as cooling and memory layout. Also, when it comes to faster memory support, or the tighter tolerances of PCIe 4.0, all of this also needs to be taken into consideration as provide the optimal path for signaling without interference from other traces and other routing.

New Instructions: Cache and Memory Bandwidth QoS Control AMD Zen 2 Microarchitecture Overview: The Quick Analysis
POST A COMMENT

217 Comments

View All Comments

  • Teutorix - Tuesday, June 11, 2019 - link

    If TDPs are accurate they should reflect power consumption.

    If a chip needs 95W cooling it's using 95W of power. The heat doesn't come out of nowhere.
    Reply
  • zmatt - Tuesday, June 11, 2019 - link

    I think technically it would be drawing a more than its TDP. The heat generated by electronics is waste due to the inefficiency of semi conductors. If you had a perfect conductor with zero resistance in a perfect world then it shouldn't make any heat. However the TDP cannot exceed power draw as that's where the heat comes from. How much TDP differs from power draw would depend on a lot of things such as what material the semiconductor is made or, silicon, germanium etc. And I'm sure design also factors in a great deal.

    If you read Gamers Nexus, they occasionally measure real power draw on systems, https://www.gamersnexus.net/hwreviews/3066-intel-i...
    And you can see that draw massively exceeds TDP in some cases, especially at the high end. This makes sense, if semiconductors were only 10% efficient then they wouldn't perform nearly as well as they do.
    Reply
  • Teutorix - Tuesday, June 11, 2019 - link

    "I think technically it would be drawing a more than its TDP"

    Yeah, but if a chip is drawing more power than its TDP it is also producing more heat than its TDP. Making the TDP basically a lie.

    "The heat generated by electronics is waste due to the inefficiency of semi conductors. If you had a perfect conductor with zero resistance in a perfect world then it shouldn't make any heat"

    Essentially yes, there is a lower limit on power consumption but its many orders of magnitude below where we are today.

    "How much TDP differs from power draw would depend on a lot of things such as what material the semiconductor is made or, silicon, germanium etc. And I'm sure design also factors in a great deal."

    No. TDP = the "intended" thermal output of the device. The themal output is directly equal to the power input. There's nothing that will ever change that. If your chip is drawing 200W, its outputting 200W of heat, end of story.

    Intel defines TDP at base clocks, but nobody expects a CPU to sit at base clocks even in extended workloads. So when you have a 9900k for example its TDP is 95W, but only when its at 3.6GHz. If you get up to its all core boost of 4.7 its suddenly draining 200W sustained assuming you have enough cooling.

    Speaking of cooling. If you buy a 9900k with a 95W TDP you'd be forgiven for thinking that a hyper 212 with a max capacity of 180W would be more than capable of handling this chip. NOPE. Say goodbye to that 4.7GHz all core boost.

    "If you read Gamers Nexus, they occasionally measure real power draw on systems, https://www.gamersnexus.net/hwreviews/3066-intel-i...
    And you can see that draw massively exceeds TDP in some cases, especially at the high end. This makes sense, if semiconductors were only 10% efficient then they wouldn't perform nearly as well as they do."

    None of that makes any difference. TDP is supposed to represent the cooling capacity needed for the chip. If a "95W" chip can't be sufficiently cooled by a 150W cooler there's a problem.

    Both Intel and AMD need to start quoting TDPs that match the boost frequencies they use to market the chips.
    Reply
  • Cooe - Tuesday, June 11, 2019 - link

    ... AMD DOES include boost in their TDP calculations (unlike Intel), and always have. They make their methodology for this calculation freely available & explicit. Reply
  • Spoelie - Wednesday, June 12, 2019 - link

    Look at these power tables for 2700X
    https://www.anandtech.com/show/12625/amd-second-ge...

    =>You are only hitting 'TDP' figures at close to full loading, so "frequency max" is not limited by TDP but by the silicon.
    =>Slightly lowering frequency *and voltage* really adds up the power savings over many cores. The load table of the 3700 will look on the whole different than for the 3600X. The 3700 will probably lose out in some medium threaded scenarios (not lightly and not heavily threaded)
    Reply
  • Gastec - Wednesday, June 12, 2019 - link

    That's not actually the real power consumption. Most likely you will get a 3700X with 70-75 W (according to the software app indications) but a bit more if tested with a multimeter. Add to that the inefficiency of the PSU, say 85-90%, and you have about 85 W of real power consumption. Somewhat better than my current 110W i7-860 or the 150+W Intel 9000 series ones I would say :) Reply
  • xrror - Monday, June 10, 2019 - link

    funny you say that. AMD TDP and Intel TDP differ. I think.

    HEY IAN, does AMD still measure TDP as "real" (total) dissipation power or Intel's weaksauce "Typical" dissipation power?
    Reply
  • Teutorix - Tuesday, June 11, 2019 - link

    Intel rate TDP at base clocks. AMD do something a little more complex.

    Neither of them reflect real world power consumption for sustained workloads.
    Reply
  • FreckledTrout - Tuesday, June 11, 2019 - link

    In desktops they are simply starting points for the cooling solution needed. They do a lot better in the laptop/tablet space where TDP's make or break designs. Reply
  • Cooe - Tuesday, June 11, 2019 - link

    Yes they do. A 2700X pulls almost exactly 105W under the kind of conditions you describe. Just because Intel's values are completely nonsense doesn't mean they all are. Reply

Log in

Don't have an account? Sign up now