ABOUT BENCH FORUMS PODCAST LOGIN REGISTER
PC Components▼
Smartphones & tablets▼
Systems▼
ENTERPRISE & IT
GUIDES▼
DEALS

Sun Fire V40z: Four Opterons in a 3U

 by Kristopher Kubicki on February 22, 2005 12:05 AM EST
38 Comments | Add A Comment
38 Comments

Chipsets (con't)

We have mentioned this in the past, but Opteron severely differs from Xeon due to the HyperTransport links from processor to processor, and that the memory controller is on the processor die. With each processor sharing a 6.4GB/s link to two other processors, and indirectly their memory banks, a four-way Opteron configuration does not get bottlenecked on a memory controller or Northbridge.

Most high end dual and quad Opteron solutions use two AMD 8131 PCI-X tunnels to control IO off the processors, but they go a step further by daisy-chaining a third 8131 tunnel to the secondary tunnel (which is why the V40z can utilize four 64-bit PCI-X 133MHz slots and still have enough IO for the integrated controllers). The HyperTransport specification details that five devices can be within an HT chain, so having two 8131 PCI-X tunnels daisy-chained is clearly within spec. A brief block diagram of our V40z can be found below.


Click to enlarge.

There is enough headroom on primary PCI-X tunnel for two more 100MHz PCI-X channels; and a third and slowest channel runs in tandem with the gigabit Ethernet and SCSI controller. Browsing through some online documentation revealed that the seventh, 66MHz PCI-X card shares bus resources with the LSI and Broadcom controllers, which means inserting a 33MHz PCI expansion card in the bus will reduce the operating bus of the SCSI and gigabit Ethernet controllers. Considering the 800MB/s headroom on that particular bus, it would seem like a poor choice to install a high bandwidth PCI device in the 66MHz interface anyway. Likewise, using a 66MHz and a 100MHz PCI-X device in tandem on the primary 8131 tunnel will result in both buses slowing to 66MHz.

AMD’s 8111 I/O Hub is linked off the primary 8131 PCI-X tunnel, and from there, most of the basic system functions are controlled including the XGI graphics adaptor. Even though XGI hasn’t been particularly popular on the desktop, Trident’s penetration into the server market solidified XGI’s server market share. The majority of the features on the AMD 8111 remain disabled, like AC’97 audio and the integrated 10/100 Ethernet controller.

Two Broadcom BCM703 chips control the external gigabit Ethernet for the server, but there are also two 10/100 out-of-band Ethernet ports that we will go into more depth later. Winbond provides the rest of the basic functionality of the machine not handled by the AMD 8111 I/O hub. LSI’s 53C1020 Ultra320 SCSI adaptor provides the V40z with the onboard SCSI.


Click to enlarge.

If the soul of our V40z is the Opteron 850, then the temporal lobe would be the Motorola MPC855T Service Processor. The Motorola MPC855T is a particularly interesting chip that we saved an entire page for more detail.

Detailing the Chipsets System Management: Another Linux Success
PRINT THIS ARTICLE
Comments Locked

38 Comments

View All Comments

  • tironside - Thursday, February 24, 2005 - link

    I agree with dwnwrd. the lom part of it is not great for remote console etc. the lom that the hp stuff has is pretty slick, with a java / web interface. The other main problem I have with this is it offers only raid 1 unless you buy a rather expensive add on card to do raid 5, kind of a teaser to put 6 drive bays and only let you do raid 1... It's a good start, but sun needs to make some changes before it can go mission critical. (raid and lom enhancements imho) while I like cli stuff, trying to get junior people to do complicated cli stuff is dangerous...

  • dwnwrd - Thursday, February 24, 2005 - link

    I have some V20s and a V40. The service processor is pretty great except if you try to direct the Linux serial console to it then connect to the "serial over LAN" you'll get a flood of "serial8250: too much work for irq4" and a sleepy system.

    http://supportforum.sun.com/hardware/index.php?t=m...

  • Pontius - Thursday, February 24, 2005 - link

    I am curious what they are using when they benchmark the linux kernel compile times. They use the time command which spits out three times - real, user & sys. Are they using the sum of all these? If not, something is wrong. Because I did the same test, on the same 2.6.4 kernel using -j2 on a dual 2.8GHz Nocona system and I got a "real" time of 147s. That doesn't seem right because the Opterons are way faster at compilation. On the other hand, if I take the sum of the 3 times, I get 420s. Any thoughts?

  • jlee123 - Wednesday, February 23, 2005 - link

    RedHat 9, are you joking!!?? This has got to be a mistake, I can't understand how Sun could be shipping a 64-bit server with a 32-bit OS that's reached End Of Life. It's the equivalent of buying a workstation with Windows ME on it. Also, there was never a official port of RH9 to x86-64, the first x86-64 RedHat was RHEL3, the Fedora team later released FC1 x86-64. If Sun doesn't wish to pay licensing, they'd be better off shipping with FC2, FC3 or CentOS, a free rebuild of RHEL. This hardware isn't even going to begin to be utilized till it's running something more modern like RHEL4 x86-64.

  • JustAnAverageGuy - Wednesday, February 23, 2005 - link

    I could think of one use for these. :)

    http://forums.anandtech.com/categories.aspx?catid=...

  • lauwersw - Wednesday, February 23, 2005 - link

    Standard rule for parallel make is to use 2xnumber of processors available. This gives most optimal results to hide disk latencies and seems to be correct in most cases I've seen.

  • phaxmohdem - Wednesday, February 23, 2005 - link

    Call me what you will, but I would like to see some quad/dual Xeon scores to compare to as well (along with price tages for comparison :) )

    And yes, If I were a rich man who knew what to do with that much computing power, I would have a dozen of these babies in my basement! Who needs women anymore once you have 48 Opteron x50 or x52 cpus humming at your disposal. And drool core? Ahhhhhhhhhhh.

  • JustAnAverageGuy - Tuesday, February 22, 2005 - link

    That thing is a BEAST.

    I have no idea what I'd do with a computer like that.

  • MrEMan - Tuesday, February 22, 2005 - link

    Kristopher,

    Thanks for the clarification about the reduced media tag.

    E

  • KristopherKubicki - Tuesday, February 22, 2005 - link

    RyanVM: The system used 850s.

    Kristopher
Copyright © 2024. All rights reserved.
BENCH
TOPICS
FOLLOW
ABOUT

Log in

Don't have an account? Sign up now