The OpenPOWER Saga Continues: Can You Get POWER Inside 1U?
by Johan De Gelas on February 24, 2017 8:00 AM ESTPositioning the Tyan GT75
Tyan positions the GT75-BP012 as an HPC and virtualization server. But that is clearly a mistake. Since the POWER architecture has been almost completely absent in the HPC world for years now, the HPC software ecosystem is dominated by x86. The POWER HPC software ecosystem is very small in comparison. That is a fact, not an opinion, as even IBM talked about "a re-entry in the HPC market" at the end of 2015.
But people do not switch to another server ecosystem easily: there has to be a compelling reason, a Unique Selling Proposition (USP). Only the recently launched S822LC can claim such an USP; it has a much faster link (NVLink) to the best NVIDIA GPUs, and as a result can offer much higher performance in some specifically-tailored HPC applications.
The GT75-BP012 has no GPU capable PCIe slot, let alone an NVLink connection. In the cramped 1U space, the CPU is limited to 3.025 GHz, which is a bad trade-off for HPC users. The GT75-BP012 is definitely not an HPC server.
Neither is it a virtualization server, as the virtualization market lives and dies by performance-per-watt.
The only target that is left is the “In-memory computing” market, which ranges from in-memory key value stores ("Redis") to DB2 BLU. In those cases, the limited processing power is – most of the time – less important than the amount of memory that can be installed at a reasonable price. Originally, the server was limited to 512 GB (32 slots, 16 GB per slot) but it should now support up to 1 TB. We say "should" as we were not able to check this.
Still, we do not feel a 1U server is a good match for POWER8. Place that POWER8 CPU on the same Tyan "Habanero" motherboard inside the 2U IBM S812LC (and Tyan's own 2U TN71 servers) and you'll get a much more attractive server. The CPU performs 40% better, the power under load is 20% lower, and you get much more PCIe slots. To reduce it to a car analogy, a turbocharged V8 cannot breathe through a straw.
Tyan's Business Model: Direct Sales Only
As one of the founding fathers of the OpenPOWER foundation, Tyan is in a unique position. It can be the "Open and easily accessible" vendor for buyers who don't care about the different services IBM offers, but just want a fast POWER server for an affordable price. However, the truth is that it is a lot easier to buy a server from IBM than from Tyan.
While the S812LC can be found on IBM's web shop, Tyan follows the more traditional Direct Sales model. So you are meant to buy the POWER8 servers in large volumes, and pricing varies by contract. As a result of this sales model, Tyan does not readily disclose the price of this server.
Consequently, in our discussions with Tyan they were vague about pricing due to the above, which makes it very difficult for us to make any meaningful comparison to other POWER8 offerings or any Xeon-based servers. While it's of course Tyan's choice how they want to do business, we consider this a missed opportunity for the company, as all of the other OpenPOWER vendors are also targeting the customers who want to buy servers in large volumes. As a result, if you want low volumes, IBM is your only choice as far as we know.
Eye Towards the Future: POWER9
Wrapping things up, the announcement of the POWER9 SMT-4 Scale Out processor is the reason why we remain optimistic about the chances of the OpenPOWER ecosystem. A full analysis of POWER9 is beyond the scope of this article, but there is a lot we like about what has been disclosed thus far for the "Linux optimized" POWER9 SO SMT-4:
- It is not an improved POWER8, there are many large improvements
- A better balance between single-threaded performance and throughput: SMT-4 will be combined with a more powerful core
- NVLink 2.0, which offers 7x more bandwidth to the GPU than PCIe 3.0
- No more power-hogging, latency-increasing memory buffers
The POWER9 SO SMT-4 will have up to 8 DDR-4 channels, and up to 2 DIMMs per socket. This means that raw memory bandwidth and memory capacity are halved relative to POWER8, but it is a very good trade-off. The use of direct attached memory (same as the Xeon E5) lowers the latency of DRAM accesses, makes the motherboard design a lot less expensive, and lowers power consumption with 60-80W.
Add 50 to 100% higher performance per socket, and you get formidable competitor for the new Xeon E5 v4 "Skylake EP". Tyan is in a good position to make this powerhouse accessible for the rest of us, so hopefully we will see a more ambitious approach than today.
Facebook
Twitter
RSS
28 Comments
View All Comments
Amandtec - Friday, February 24, 2017 - link
Only an amateur here but I did read some of your previous articles on this. I thought the big advantage of POWER was it was encouraging 3rd parties to bring ASIC's into their server space, while Intel wan't to own the whole hardware setup? Surely ASIC's is where the whole performance per watt game ends - see Bitcoin mining.ddriver - Friday, February 24, 2017 - link
There is nothing preventing you from building your own accelerators and hooking them to a free PCIE slot.JohanAnandtech - Friday, February 24, 2017 - link
True. But you have only one PCIe 8x and AFAIK it is not an OpenCAPI one, nor an NVME capable.JohanAnandtech - Friday, February 24, 2017 - link
I meant NVLink, not NVME :-)SarahKerrigan - Friday, February 24, 2017 - link
The GT75 is a weird system and I've never been clear on why it exists. The Supermicro 1U (S821LC) at least gets you two CPU's for cheap in 1U... I'm not overly surprised by poor performance here. This is a 130W CPU, not a 170 as Anandtech says, running at low clocks. 170 is only the turbo power. I'm also guessing it's loud as hell with those fans.That being said - Anandtech also has a history of interestingly Intel-centric interpretations of results. Remember when Intel's uarch was "a lot more sophisticated" than P8 according to Anandtech based purely on ST 7zip results, and then when ThunderX did well on 7zip, Anandtech said 7z was a meaningless benchmark irrelevant to server workloads? I've also noticed that since gcc's Power output improved (~4000 MIPS single-thread compression with gcc 6.2 on 3.3GHz P8), 7zip has conveniently vanished from Anandtech's Power reviews. The idea of drawing conclusions about microarchitectures based on a balance of a range of tests seems alien to them. Looked at through that lens, Power8 (well, in systems not named GT75 :P) looks decent at some things, less decent at others, but overall pretty good. Database perf/W, per Anandtech's testing, is better than Haswell. This isn't a bad place for P8 to be, considering it's been shipping since early 2014 and is on the verge of replacement.
Tl;dr - GT75 is a turd but Anandtech sees what they want to see. I have to wonder if Ryzen is going to be reviewed the same way.
JohanAnandtech - Friday, February 24, 2017 - link
7zip is indeed a bad indicator for server performance, but at the time, we only had access to a virtual machine on top of a POWER8 with 2 GB of RAM and we thought it might give us a first glimpse of what the P8 was capable off.As time progressed, we understood that is mostly a TLB/latency sensitive benchmark. So it has no place in a server oriented article, it is mostly interesting to discuss micro arch details.
Ian Cutress - Friday, February 24, 2017 - link
Anandtech has different editors focusing on different areas - Johan on Server, I'm on CPU, Matt on mobile, etc. Feel free to reach out via email if you have suggestions for us.Kevin G - Friday, February 24, 2017 - link
This isn't the best showing of POWER8. Those chips need robust cooling to keep their clock speeds up to be competitive. The 1U form factor does place a lot of constraints on the design.As for the Tyan system itself, I'm kinda of surprised that they went with the Centaur buffer that uses DDR3 as IBM has reportedly been shipping buffers that support DDR4 for nearly half a year now. That'd lower power consumption greatly, though likely not enough to be competitive with Intel on a system level performance/watt metric. Moving to DDR4 would give the system a massive increase in memory capacity as 128 GB LR DDR4 DIMMs are shipping with 256 GB LR DIMMs on the horizon. Using 256 GB DIMMs, a system like this would support 8 TB which is a lot for a 1U server.
Considering the internals of the system I'm not surprised but a secondary PSU option would have been nice, even if it was external. PSU redundancy is remarkably common, even for 1U systems.
I'd also be worried about IO performance on this system with all the networking and SATA ports hanging off of a single PCIe uplink to the CPU.
I do agree with the conclusion that POWER9 looks to be very promising. The nice thing is that IBM is going to be offering both the SMT4 and SMT8 cores in both types of sockets (essentially four different dies!).
ddriver - Saturday, February 25, 2017 - link
Form factor wont change the facts - it is a power, it is too expensive, and it is too slow. Power sounds great on paper, but it doesn't seem to deliver in practice.Kevin G - Tuesday, February 28, 2017 - link
Form factor does matter when it changes the cooling system which in turn can invoke thermal throttling.Even on the x86 side of things, half wide 2U servers are popular as they can use larger fans despite having the same effective density as a full width 1U server. The increased air flow into a chassis is great for keepings cools and helps maintain high turbo levels for performance.