Inside the S822L: Hardware Components

The 2U Rack-mount S822L server contains two IBM POWER8 DCM sockets. Each socket thus contains two cores connected by a 32GBps interconnect. The reason for using a Multi-Chip-Module (MCM) is pretty simple. Smaller five-to-six core dies are a lot cheaper to produce than the massive 650 mm² monolithic 12-core dies. As a result the latter are reserved for IBM's high-end (E880 and a like). So while most POWER8 presentations and news posts on the net talk about the multi-core die below...

... it is actually an MCM with two six core dies like the one below that is challenging the 10 to 18 core Xeons. The massive monolithic 10-12 core dies are in fact reserved for much more expensive IBM servers.

The layout of the S822L is well illustrated by the scheme inside the manual.

Each DCM offers 48 PCIe Gen 3 lanes. 32 of those lanes are directly connected to the processor while 16 connect to PCIe switches. The PCIe switches have "only" 8 lanes upstream to the DCM, but offer 24 lanes to "medium" speed devices downstream. As it unlikely that both your SAS controllers and your network controllers will gobble up the full PCIe x8 bandwidth, this is a very elegant way to offer additional PCIe lanes.

Taking a Closer Look Inside IBM's S822L The L4-cache and Memory Subsystem
Comments Locked

146 Comments

View All Comments

  • FunBunny2 - Friday, November 6, 2015 - link

    "The z10 processor was co-developed with and shares many design traits with the POWER6 processor, such as fabrication technology, logic design, execution unit, floating-point units, bus technology (GX bus) and pipeline design style, i.e., a high frequency, low latency, deep (14 stages in the z10), in-order pipeline." from the Wiki.

    Yes, the z continues the CISC ISA from the 360 (well, sort of) rather than hardware RISC, but as Intel (amongst others) has demonstrated, CISC ISA doesn't have to be in hardware. In fact, the 360/30 (lowest tier) was wholly emulated, as was admitted then. Today, we'd say "micro-instructions". All those billions of transistors could have been used to implement X86 in hardware, but Intel went with emulation, sorry micro-ops.

    What matters is the underlying fab tech. That's not going anywhere.
  • FunBunny2 - Friday, November 6, 2015 - link

    ^^ should have gone to KevinG!!
  • Kevin G - Saturday, November 7, 2015 - link

    The GX bus in the mainframes was indeed shared by POWER chips as that enabled system level component sharing (think chipsets).

    However, attributes like the execution unit and the pipeline depth are different between the POWER6 and z10. At a bird's eye view, they do look similar but the implementation is genuinely different.

    Other features like SMT were introduced with the POWER5 but only the most recent z13 chip has 2 way SMT. Features like out-of-order execution, SMT, SIMD were once considered too exotic to validate in the mainframe market that needed absolute certainty in its hardware states. However, recent zArch chips have implemented these features, sometimes decades after being introduced in POWER.

    The other thing is that IBM has been attempting to get get more and more of the zArch instruction set to be executed by hardware and no microcode. Roughly 75% to 80% of instructions are handled by microcode (there is a bit of a range here as some are conditional to use microcode).
  • JohanAnandtech - Saturday, November 7, 2015 - link

    I believe that benchmark uses about 8 threads and not very well either? Secondly, it is probably very well optimized for SSE/AVX. So you can imagine that the POWER8 will not be very good at it, unless we manually optimize it for Altivec/VSX. And that is beyond my skills :-)
  • UrQuan3 - Monday, December 21, 2015 - link

    I'm sure no one is still reading this as I'm posting over a month later, but...

    I tested handbrake/x264 on a bunch of cross-platform builds including Raspberry Pi 2. I found it would take 24 RPi2s to match a single i5-4670K. That was a gcc compiled handbrake on Raspbian vs the heavily optimized DL copy for Windows. Not too bad really. Also, x264 seems to scale fairly well with the number of cores. Still, POWER8 unoptimized would be interesting, though not a fair test.

    BTW, I'd encourage you to use a more standard Linux version than 6-month experimental little-endian version of Ubuntu. The slides you show advertise support for Ubuntu 14.04 LTS, not 15.04. For something this new, you may need the latest, but that is often not the case.
  • stun - Friday, November 6, 2015 - link

    @Johan You might want to fix "the platform" hyperlink at the bottom of page 4. It is invalid.
  • JohanAnandtech - Friday, November 6, 2015 - link

    Thanks and fixed.
  • Ahkorishaan - Friday, November 6, 2015 - link

    Couldn't read past the graphic on page 1. It's 2015 IBM, time to use a font that doesn't look like a toddler's handwriting.
  • xype - Sunday, November 8, 2015 - link

    To be fair, it seems that the slide is meant for management types… :P
  • Jtaylor1986 - Friday, November 6, 2015 - link

    Using decimals instead of commas to denote thousands is jarring to your North American readers.

Log in

Don't have an account? Sign up now