Original Link: http://www.anandtech.com/show/2478
The flood of pictures last week should have given a good picture of what CeBIT is all about, so we'll keep this intro short. In this article, we'll look at a few of the servers that we saw on CeBIT. Almost all our attention goes to the so-called "Tier-2 vendors", as the Tier-1 vendors were mostly focused on the local market and less interested in the international press.
The theme of this year CeBIT was "green IT". It is of course highly disputable whether the incredible amount of energy that datacenters consume can ever be considered "green". Nevertheless, it is clear that current datacenters are grossly inefficient as only a small percentage of the energy is used to do the actual processing of data. One of the latest victims of the continuing search for more power efficiency is the FBDIMM. Every single server vendor was proudly showing Xeon servers with DDR2 DIMMs. FBDIMM based servers were still there of course, but they were much less in the spotlight. Although 1.5V FBDIMMs have lowered the per DIMM power consumption from 15W (1.8V) to ~12W, they are still a lot higher than the typical DDR2 DIMM (6-8W).
The next victim is the massive server power supplies. Just a few years ago, a typical server power supply (e.g. 800W) was about twice as powerful as a desktop power supply (400W). Nowadays, it is almost the opposite as you see in our PSU/Case CeBIT Report. Power supplies from 700 to 1000W are now quite common in desktops, while many dual-socket servers we saw were equipped with 300-500W PSUs. The only 1000W+ PSU we saw was on an eight-socket quad-core system, in a blade chassis of course. However, you can hardly call two 1400W PSUs used to power 10 to 14 (!) servers overkill.
MSI: small but powerful
MSI had some success with their half-length 1U barebones chassis. These typically single-socket systems are popular for web servers and firewalls, but the K2-105-A2M can do more. MSI was able to fit two quad-core Opteron 23xx CPUs into this tiny chassis, together with eight DIMM slots (max. 32GB) and one PCI-E x16 riser card. The only thing left on our wish list is more Ethernet ports, although two Gigabit ports are available.
This kind of server only uses internal storage for the boot device; two hot swappable SATA disks should do that job nicely. Also, note the tiny heatpipe that keeps the PCI-E tunnel and Southbridge cool. The MSI X2-109-A4M features two Xeon sockets connected to Intel's latest 5100 MCH with DDR2 and Xeon 54xx (Harpertown) support.
Unfortunately, Intel cripples this chipset - for strategic reasons - to three DIMMs per channel. That means this server is limited to six DIMMs or 24GB of RAM. Although the chipset can support 48GB (6x8GB), it is not realistic to expect ultra expensive 8GB DIMMs in an entry-level or midrange server.
Tyan: Eight-socket monster, 4U blade, and "Flexblade"
With the current "server consolidation courtesy of virtualization" trend, memory capacity and power consumption is the key to success. Tyan agrees, and shows off their S4989WG2NR: 32 DIMM slots are available on this four-socket motherboard, which can cope with the 16 cores of four Opteron 23xx CPUs. We were quite surprised when Yifang Cheng told us that this board is also targeted at the workstation market. Yes, for those of you that want the ultimate workstation board, you get two PCI-E x16 and two PCI-E x8 slots to build a workstation with SLI workstation cards. You can also connect up to eight SAS disks and six SATA disks to this powerhouse.
For the few that really need more CPU power, there is the Tyan Tank TN27, which uses a S4985G3NR3 board. It seems like a normal quad-socket board, but it features a special PCI-E connector upon which you can stack another quad-socket board.
This allows Tyan to offer an eight-socket server, which can use up to 32 of the newest Barcelona CPUs. The TN27 reminds us of Sun's 2U x4450 (which uses Xeons instead), but the latter is a bit more elegant as it is easier to add DIMMs and CPUs.
Supermicro's Twin was a huge success, especially in the HPC world. The Twin can be found datacenters of countless universities. Tyan, now part of Mitac, decided this market was getting too big for Supermicro alone, so now they offer two alternatives to Supermicro's Twin. The most impressive one is the AMD quad-core based Tyan Tank GT28, which supports two quad-core Opteron 23xx CPUs per node.
A central (1000W) PSU powers both nodes, similar to the way the Supermicro Twin works. Tyan tries to outdo Supermicro by offering 16 (!) DIMMs per node (Supermicro offers eight), four gigabit ports per node (Supermicro has two), and SAS disks are possible (Supermicro only supports SATA). The Intel based "Double 1U" is a bit more modest with only one socket per node. Tyan somehow is able to get round the six DIMM limitation of the 5100 "San Clemente" chipset: the Tyan GT 24 supports eight DDR2 slots per node. However, only two Gigabit Ethernet ports are available.
The goal of this server is to offer a low power web server. This dual 1U only has a 600W PSU - that's only 300W per node.
Tyan also has a few original blade chassis. We heard that Tyan produces some of IBM's blades, so it seems that they have a lot of experience in the market. The same boards that were used in the Double 1U can also be found in a 4U blade form, which find a home in the Tyan FX71 chassis.
The FX71 can contain up to 10 blades. That works out to 80 cores in a 4U package, which is incredibly high density. This could be an interesting solution for a rendering farm. The version with dual Opteron 23xx CPUs only allows 68W CPUs. Considering that each individual blade is only 1.8 inch (or about 4.5 cm) wide, using low power CPUs is a good move. Don't put this rendering farm under your desk though, as it is cooled by 15000 RPM (!) fans. Add some wings and this thing can fly….
If you must have your very own quiet rendering farm and workstation, consider Tyan's Flexblade, also called a "technical workgroup computing platform". This is an evolution of Tyan's famous Personal Super Computer (PSC). You can insert either two quad-socket nodes with a dual-socket node or five dual-socket nodes. The Flexblade allows you to put the blades in a kind of RAID mode. One blade is then a hot spare and will not be used until one of the other four nodes dies.
Each node has a PCI-E x16 slot, supporting one high performance video card per node.
Each blade is connected to the backplane via a PCI-E connector. This connects to a PCI-E Infiniband or network card located in the lower end of the chassis (see the picture above). Each of these PCI-E cards then connects to an uplink card that offers the two external Infiniband and/or Gigabit ports.
Supermicro: the quiet low power blade
Supermicro's Twin got an update: it is now available with the newest Stoakley chipsets and can make use of 1.5V FBDIMMs, which lower the power consumption per DIMM from 15W to 12W. However, the focus was on Supermicro's Blade products. CERN, who is testing its latest and greatest Large Hadron Collider, has chosen to upgrade its datacenter with 210 Supermicro Superblades. The LHC is the largest and highest-energy particle accelerator in the world, with work being done on a "Grand Unified Theory". That is a pretty exciting application for a blade server.
This success has clearly inspired Supermicro as it has already improved its Superblade. The newest blade chassis can take 14 instead of 10 blades and consumes much less power. This was clearly demonstrated by the Superblade running in Supermicro's booth. At relatively light load, the blade server could hardly be heard and the airflow was cool and very light. We made this clearer by holding a napkin at the back of the server as you can see in the picture below. The configuration at CeBIT was a bit unrealistic of course as there were only two blades running under light load, but still most 1U servers we have in the lab produce a much stronger and louder airflow just to power one server. The Supermicro SBE-714D-D28 made an excellent "tuned for low energy consumption" impression on us. Of course, nothing beats accurately measured numbers in our independent lab….
The Superblade SBE-714D-D28 is able to keep power consumption relatively low for a high performance enterprise blade chassis thanks to highly efficient fans, 93% efficient 1400W PSUs, and the Intel 5100 chipset that allows Supermicro to use DDR2. With 10 Blades, the enterprise blade should be quieter than 72 dB. We have to admit, though, that using noise levels to represent power requirements is a very vague way of doing things.
If this high performance enterprise blade is still too loud for you, Supermicro offers the Office Blade that makes less than 50 dB noise. Using 10 blades per chassis and low voltage Xeons allows for even lower power consumption.
Storage for Video editing
AIC/T-win/X-store had a few interesting storage solutions, but the AIC XJ-SA12-212R was the most original one. The SAS JBOD enclosure has a small PCI Express board or target adapter that connects directly to a PCI express card (host adapter) in your server board.
Using a PCI-E x8 card, the storage enclosure should be able to stream up to 2GB per second (or 20 Gbps).
Traditionally, JBODs are connected via an Infiniband connector, which requires quite a bit of conversions from one protocol to another. With the AIC XJ-SA12-212R, latency should be lower as you get end-to-end PCI Express all the way to the drives. The enclosure consists of two 6-disk HDD bricks that are relatively easy to remove and transport. AIC believes that this kind of enclosure is ideal for the video editing people who like to carry their data from the filming location to the editing location. This is the faster, larger, and heavier version of your average portable disk drive.
Other storage tidbits
Promise, which has been focused on the low cost ATA and SATA market, has finally produced its own SAS controllers. Four different SuperTrak controllers have128MB to 256MB data cache onboard and can be equipped with a 72-hour battery backup unit (BBU). Each card has an x8 PCI-E interface and features RAID 5 and RAID 6 support. The black box standing up in the following picture is the BBU.
Seagate's newest Cheetah drives (3.5" 15,000RPM) are now also available as 450GB drives. Atto Technology showcased their 8Gbps Celerity Fibre Channel host bus adapter (HBA). Fibre channel is thus getting ready to double the maximum bandwidth once again.
Thecus - founded only a few years ago (2004) - showed off the N5200Pro, an SMB NAS device that can also be turned into an iSCSI device (running a Linux iSCSI target but of course compatible with MS' iSCSI initiator). You can also combine this storage device with an IP camera.
NAS devices for SMBs have disappointed many people, as most of them are incredibly slow when you want to perform a backup. Despite being tested with a rather sequential access pattern, you get a meager 20-25 MB/s. Considering that these devices are equipped with more than one modern SATA drive, have a gigabit interface that is capable of 110 MB/s, and even our 2.5" LaCie portable USB disk can read data faster than that, this is simply ridiculous. Thecus promises that this should not be the case with the N5200Pro as the Mobile Celeron 1.5 GHz inside should be powerful enough to offer much better data rates. The N5200Pro is capable of RAID 0, 1, 5, 10 and even RAID 6. Hot spares are also supported.
Final words: AMD Shanghai rumors
We heard through the grapevine that AMD's Shanghai has already been benchmarked by a few OEM related people. Those people were very pleased with its performance; the improved IPC and 6 MB L3-cache are probably paying off. Despite the very bad and late start, there was quite a bit of enthusiasm for AMD's Barcelona CPU. Supermicro's and Tyan's fastest blade servers are based on the latest AMD chip for example. The AMD Barcelona seems to have stolen a bit of the quad-socket market share of the Intel Tigerton CPU, which was a lot harder to spot at CeBIT.
The Intel 5100 chipset + Xeon 54xx (Harpertown) combination looks to be another clear winner. Almost every vendor showed a whole range of products based on this low power, high performance combination.
Does anyone remember the XGI Volari? Once touted as an alternative to the midrange of ATI and NVIDIA graphics cards, the XGI Z7 and Z9 are quite popular as low power graphic chips in MSI, Tyan, ASUS, and other servers.