The OpenPOWER Saga Continues: Can You Get POWER Inside 1U?
by Johan De Gelas on February 24, 2017 8:00 AM ESTPositioning the Tyan GT75
Tyan positions the GT75-BP012 as an HPC and virtualization server. But that is clearly a mistake. Since the POWER architecture has been almost completely absent in the HPC world for years now, the HPC software ecosystem is dominated by x86. The POWER HPC software ecosystem is very small in comparison. That is a fact, not an opinion, as even IBM talked about "a re-entry in the HPC market" at the end of 2015.
But people do not switch to another server ecosystem easily: there has to be a compelling reason, a Unique Selling Proposition (USP). Only the recently launched S822LC can claim such an USP; it has a much faster link (NVLink) to the best NVIDIA GPUs, and as a result can offer much higher performance in some specifically-tailored HPC applications.
The GT75-BP012 has no GPU capable PCIe slot, let alone an NVLink connection. In the cramped 1U space, the CPU is limited to 3.025 GHz, which is a bad trade-off for HPC users. The GT75-BP012 is definitely not an HPC server.
Neither is it a virtualization server, as the virtualization market lives and dies by performance-per-watt.
The only target that is left is the “In-memory computing” market, which ranges from in-memory key value stores ("Redis") to DB2 BLU. In those cases, the limited processing power is – most of the time – less important than the amount of memory that can be installed at a reasonable price. Originally, the server was limited to 512 GB (32 slots, 16 GB per slot) but it should now support up to 1 TB. We say "should" as we were not able to check this.
Still, we do not feel a 1U server is a good match for POWER8. Place that POWER8 CPU on the same Tyan "Habanero" motherboard inside the 2U IBM S812LC (and Tyan's own 2U TN71 servers) and you'll get a much more attractive server. The CPU performs 40% better, the power under load is 20% lower, and you get much more PCIe slots. To reduce it to a car analogy, a turbocharged V8 cannot breathe through a straw.
Tyan's Business Model: Direct Sales Only
As one of the founding fathers of the OpenPOWER foundation, Tyan is in a unique position. It can be the "Open and easily accessible" vendor for buyers who don't care about the different services IBM offers, but just want a fast POWER server for an affordable price. However, the truth is that it is a lot easier to buy a server from IBM than from Tyan.
While the S812LC can be found on IBM's web shop, Tyan follows the more traditional Direct Sales model. So you are meant to buy the POWER8 servers in large volumes, and pricing varies by contract. As a result of this sales model, Tyan does not readily disclose the price of this server.
Consequently, in our discussions with Tyan they were vague about pricing due to the above, which makes it very difficult for us to make any meaningful comparison to other POWER8 offerings or any Xeon-based servers. While it's of course Tyan's choice how they want to do business, we consider this a missed opportunity for the company, as all of the other OpenPOWER vendors are also targeting the customers who want to buy servers in large volumes. As a result, if you want low volumes, IBM is your only choice as far as we know.
Eye Towards the Future: POWER9
Wrapping things up, the announcement of the POWER9 SMT-4 Scale Out processor is the reason why we remain optimistic about the chances of the OpenPOWER ecosystem. A full analysis of POWER9 is beyond the scope of this article, but there is a lot we like about what has been disclosed thus far for the "Linux optimized" POWER9 SO SMT-4:
- It is not an improved POWER8, there are many large improvements
- A better balance between single-threaded performance and throughput: SMT-4 will be combined with a more powerful core
- NVLink 2.0, which offers 7x more bandwidth to the GPU than PCIe 3.0
- No more power-hogging, latency-increasing memory buffers
The POWER9 SO SMT-4 will have up to 8 DDR-4 channels, and up to 2 DIMMs per socket. This means that raw memory bandwidth and memory capacity are halved relative to POWER8, but it is a very good trade-off. The use of direct attached memory (same as the Xeon E5) lowers the latency of DRAM accesses, makes the motherboard design a lot less expensive, and lowers power consumption with 60-80W.
Add 50 to 100% higher performance per socket, and you get formidable competitor for the new Xeon E5 v4 "Skylake EP". Tyan is in a good position to make this powerhouse accessible for the rest of us, so hopefully we will see a more ambitious approach than today.
Facebook
Twitter
RSS
28 Comments
View All Comments
Einy0 - Friday, February 24, 2017 - link
Articles like these make me wonder if some of these companies using IBM eServer iSeries(AS/400) as mid-level servers are wasting their money. I was always under the impression that Power was suppose to be tuned for database heavy workloads and hence have a massive advantage in doing so. I know the iSeries servers run an OS with DB2 built-in and tuned specifically for it but how much of an advantage does that really equate to?FunBunny2 - Friday, February 24, 2017 - link
-- I know the iSeries servers run an OS with DB2 built-in and tuned specifically for it but how much of an advantage does that really equate to?unless IBM has done a complete port recently, AS/400 "integrated database" was built before server versions of DB2 existed. it's/was just a retronym.
kfishy - Friday, February 24, 2017 - link
As ISAs becoming more and more relevant in the post-Moore's law world, where you can't solve a computational problem just by throwing ever more transistors at it, I wonder if this opens up opportunity for POWER to carve out niches left out by Intel's more fixed and general purpose approach.At the same time, POWER will have to contend with a nascent but rising and truly open ISA in RISC-V, where companies can simply implement the subsets of the ISA that they need. The next decade in processor architecture is going to be interesting to watch.
FunBunny2 - Friday, February 24, 2017 - link
-- As ISAs becoming more and more relevant in the post-Moore's law world, where you can't solve a computational problem just by throwing ever more transistors at itgiven that ISA has been reduced to z, ARM, and X86 not counting Power, of course. and ARM might not really qualify as equivalent. for those ancient enough, or well read enough, know that up to and during the "IBM and 7 Dwarves" era, ISA and even base architecture, made a varied ecosystem. not so much anymore. and I doubt anyone will invent a more efficient adder or multiplier or any other subunit of the real CPU. just look at the screen shots of chips over the last couple of decades: the real CPU area of a chip is nearly disappeared. in fact, much (if not most) of the transistor budget for some years has been used for caching, not ISA in hardware. so called micro-architecture is just a RISC CPU, and the rest of the chip is those caches and ever more complicated "decoder". that and integrating what had previously been other-chip functions. IOW, approaching monopoly control of compute.
I expect the next decade to be more of the same: more cache and more off-chip function brought on chip. actual CPU ISA, not so much.
aryonoco - Saturday, February 25, 2017 - link
Thank you Johan. Great article.Not all AnandTech articles live up to the standards set in the days past, but your articles continue your own excellent standards.
Very much looking forward to POWER 9 chips. Hopefully they have also done the work to port the toolchain and important software already to it this time and we won't have to wait another 12 months after release to be able to compile normal Linux programs on it.
Also, 12 fans running at 15,000 rpm in a 1U? What did that sound like?! Wow!
JohanAnandtech - Sunday, February 26, 2017 - link
Thx Aryonoco. Not all of those 12 fans were running at top speed, but imagine a Jumbo jet taking off sound. It clearly show how hard it is to cool IBM's best in a 1U: you have to limit the clockspeed to about 2/3 of what it is capable off and double the number of fans.yuhong - Wednesday, March 1, 2017 - link
"Unfortunately, the latest 8 Gbit based DIMMs are not supported."Micron don't make these chips anymore:
http://media.digikey.com/pdf/PCNs/Micron/PCN_32042...
Interestingly, Crucial is selling 32GB DDR3 quad rank RDIMMs again (but not LR-DIMMs):
http://www.crucial.com/usa/en/ct32g3erslq41339
mystic-pokemon - Sunday, March 5, 2017 - link
For folks who are saying that POWER only looks good on paper. NOT true.I know shit ton of stuff about one of the server Johan listed above. He has a point when he says Power consumption is only so much important.
In short, when you combine all aspects to TCO model: POWER8 server delivers most optimal TCO value
We consider all the following into our TCO model
a) Cost of ownership of the server
b) Warranty (Lesser than conventional server, different model of operations)
c) What it delivers (How many independent threads (SMT8 on POWER8 remember ? 192 hardware threads), how much Memory Bandwidth (230 GBPs), how much total memory capacity in 1 server ( 1 TB with 32 GB)
d) For a public cloud use-case, how many VMs (with x HW threads and x memory cap / bw ) can you deliver on 1 POWER8 server compared to other servers in fleet today ? Based on above stats, a lot .
e) Data center floor lease cost in DC ( 24 of these servers in 1 Rack, much denser. Average the lease over age of server: 3 years ). This includes all DC services like aggers, connectivity and such.
f) Cost per KWH in the specific DC ( 1 Rack has nominal power 750W)
All this combined POWER has good TCO. Its a massively parallel server, what where major advantage comes from. Choose your workload wisely. That's why companies continue to work on it.
I am talking about all this without actually combining with CAPI over PCIe and openCAPI. With POWER9 all this is getting even better. Get it ? POWER is going no where.