For a special occasion, and with what looked like a pricing error, I decided to splash out on a 10GBase-T switch for my testing lab. Coming in at almost £800, reduced from £1900, this beast was not cheap but surprisingly below my personal cost-per-port to get into the 10-gigabit game. Rather than review the switch (how do you review a switch anyway? ), I just want to go through what this thing is and what I can do with it. Plus some rough point-to-point bandwidth speeds.

The Quest for 10G on Copper

One of my personal crusades in recent years has been to push 10-gigabit networking – specifically Ethernet over copper (10GBase-T) – into a price range that is more amenable to home users. For a long time, this technology has been priced for commercial and enterprise: upwards of $100 per port for the switch and $100-$200 per port for the add-in cards. This is partly because the technology has a lot of enterprise bells and whistles, such as QoS, but also there has never been a big drive for more than gigabit Ethernet in the home.

Recently this changed somewhat. After a decade of Intel’s 10G silicon on the shelves, Aquantia came in and started offering add-in cards below $100 – and not only for 10G but also the new 2.5G and 5G standards as well. Their idea is to expand the market for this technology, given that they’ve been in the backhaul and networking backbone markets for a while. They had a 2 year lead over others on the 2.5G/5G silicon, but the key issue (as I explained to them over a year ago) was that in order to make it happen in the home it would require switches. These switches could either be managed or unmanaged, but really there needs to be a $50/port or $30/port series of switches for multi-gigabit to really take off. I make an online poll just for this.

Out of 137 voters at the time, about 10% said they would jump on the technology at $80 per port. Around a third said $50 per port, and 60% or so said $30 per port. To be honest, these results were around what I expected. Personally, I think a $250 5-port switch would be a great point to enter the market.

All that being said, and as much as the good folks at Aquantia agree with me, they don’t make the switches – it’s up to the Netgears, the D-Links, the TP-Links, and such to actually build them. I don’t have contacts with any of them to say what their thoughts are, but they haven’t been as quick as I hoped. One thing is that, I guess, they don’t want to build cheap 10G switches which might pull business away from the high margin enterprise hardware.

The State of 10GBase-T

A while back, before Aquantia burst into the scene, we did a piece about every consumer motherboard with 10GBase-T built in. This article saw insane traffic for a short piece, but it also showed every motherboard that was using Intel’s X540-T2 controller chip. For these boards, the chip was expensive (adding ~$250 to the board retail price), power hungry, and it required a good number of PCIe lanes. The upside was that most of these boards were dual port.

Since then, we’ve seen boards with Aquatia AQC107 (and AQC108) chips on board, which raise the price of the board by $70-$100 for a single port, but this is still a far more accessible way of enabling anything better than gigabit Ethernet on a PC. Then there's the range of 10GbE PCIe cards available, running at around $100.

As for the switches, the only options were a number of managed 8-port models from the likes of Netgear, such as the Netgear XS708E, which was around the $750 mark. Shelling out almost $80-$100 per port (after taxes), as we saw in the poll above, is a little insane for a home network and doesn’t appeal to very many users.

In the last year or so, there have been a number of switches that have hit the market offering two 10GBase-T ports and eight 1G ports. This includes switches such as the ASUS XG-U2008, which has been on sale for $250-$300 or so, the Netgear gaming-focused GS810EMX at $250-$300, and the Netgear GS110EMX which is a non-gaming version for slightly less. The problem with these switches is that they only have two ports – there’s no way to make a ‘tree’ from them, it essentially becomes expensive point-to-point connection, given the cheap cost of gigabit switches.


The Netgear Gaming 2x10G + 8x1G managed switch

So as of this week, the state of play was this for 10G offerings:

The offerings are still pretty abysmal for anyone looking for a ‘quick fix’ to enable 10GBase-T in the home.

It Was A Misprice or Something

So this week, when a family member asked me what I wanted for my birthday, I idly flicked through some switch listings. Thinking I might just splurge for a 2-port, I was hoping that an 8-port had come down in price. What I found, without too much trouble, was the Netgear XS724EM, a 24-port 10GBase-T switch. My search hadn’t been for that many ports – I assumed it would automatically be too expensive.

The XS724EM had an RRP of £1700. The price in front of me was £782. After a quick rant on Twitter, it was a no brainer (ed: I still think you're insane). At £782 / $858, this was a 55% discount, and comes in at just under $36 per port. I expected at some point that the cost of the switches would come down in price, although I didn’t anticipate the first one to do so would be a super-large one. Not only that, but it supports both 5G and 2.5G as well, so it's still beneficial with existing Cat5e runs.

If you go to the page today, you will see that this might have been a misprice.

The unit is currently up for £1280, almost £500 more than what I paid for it. Bargain. Prime delivery too.

Unboxing the XS724EM

After showing the box to the resident feline population, it was time to see what we had. On the side it gives a lot of pertinent information. This unit weighs 3.72kg / 8.21 lbs, which will be a key point for some users.

In the box, the unit is well packaged with foam blocks, although there is little space above and below it should the box be punctured.

Aside from the manual, the box came with two power cords (one UK, one EU), along with rubber feet for users putting the switch on a desk somewhere, and brackets to extend the unit to a standard 19-inch rack. Some of the comments online state that in a rack, using the screws, it actually ends up very rear heavy, putting a lot of torque on the screws if the unit isn’t directly above a server. In this case, it might be good to invest in rails.

The manual gives examples of how to connect the switch to multiple devices. Interestingly it thinks that gaming laptops with 2.5G connections are somewhat ubiquitous – I think someone should tell Netgear this is not the case.

There is also an app for the smartphone to help with additional management.

The cables for the switch are designed to be put in the front, and we get 24x 1G/2.5G/5G/10GBase-T ports for RJ45 cables. There is also two 10G SFP+ ports on the right, muxed with the final two 10GBase-T ports so only one pair can be used at once.

The lights on a normal gigabit Ethernet port are both orange and flicker with data. In this case, to discern 2.5G, 5G, and 10G, the LEDs go green and will have different patterns based on the connectivity.

There’s a Kensington Lock on the rear for physical security.

Airflow through the unit is provided by three fans near the outtake, with the intake on the other side.

Opening the chassis takes two screws on either side and three on the rear. It slides off like a standard server chassis, keeping the front panel.

At the rear of the chassis, covered in a shroud, in the built in power supply. The main PCB has several big heatsinks on it, which we’ll get to in a bit.

The fans in the chassis are Delta AFB0412SHB brushless fans, and these can kick up quite a noise at full blast. Luckily the only time I’ve heard them on full is when turning the unit on.

On the PCB are the controllers covered in aluminium heatsinks. These heatsinks are big and heavy, and there’s even a metal plate on top of the main switching fabric.

I actually tried to take this plate off to see the controllers underneath, but that was a no-go. The heatsinks actually use additional thermal pads to keep the plate attached and to conduct the heat energy through the unit. As I bought this unit personally for my use, rather than AnandTech’s money or a review sample, I wasn’t willing to potentially break things. Sorry.

After fitting it all back together, and putting the rubber feet on, it was time to hook it up to my home network.

This switch is going to sit at the crossroads of my five main test beds, along with a steam cache server (to enable quicker downloads), a local NAS, and a few other devices. For sure, I’ll be doing some office rearrangement soon to make the most out of the switch.

Using the Switch

This is a managed switch, which means there is the opportunity to go in and organise all of the settings. However, for users who just want to use it as a switch, it is almost as easy as plug and play. In fact, it was plug and play to begin with – in order to make the process a bit easier, I went into the web interface for the switch and disabled DHCP to make it perfectly clear (DHCP is handled by my router).

Logging in was straightforward (IP and password are on the bottom of the switch, and the password default is password), and the management control seems suitable for what it was designed for. Users can tell which ports are connected at what speeds, and also limit connectivity per port, and set up VLANs. In my case I’m not going to be using much of any of this, but the VLAN and QoS options are going to be key for office users.

Performance

As it turns out, testing networking hardware is difficult. If you really want to get a detailed overview of a switch, it requires the best part of 12-16 systems hitting it hard, aggregating the results for latency and bandwidth, and also keeping track of power, temperature, and noise. Unfortunately I have neither the time nor the facilities to do that, but a quick blast of iperf for peak-to-peak speeds is what we have at hand.

For our testing systems, on one end I have an AMD Ryzen Pro 2400GE (35W) APU system with an Intel X540T2 PCIe card equipped, and the other end is an X170 motherboard with a Core i3-7100T (35W) and an Aquantia AQC107 PCIe card equipped. Both systems were running Win10 x64 Enterprise 1803. I ran installed cards and drivers, made no other settings changes, and ran iperf by varying the number of parallel connections.

The default settings in iperf and on the two systems showed that we could, in theory, reach transfer rates of around 9.3 Gbps. The cards could also be the limiting factor here – the dangers of testing networking is that typically a 10G card is connected to a 10G switch is connected to another 10G card; either one of those three parts could be the bottleneck. I did note that iperf very easily used 85% of one thread on each system, so it could be that we need a faster CPU for better performance as well.

But a more initial concern when buying a switch like this is noise. This is a switch designed for the hubbub of a small office, or a server rack – not necessarily a home office where I might be recording audio. However in my initial use, the only time the fans have come on is when the machine is turned on (like some motherboards turn all fans to full until the startup sequence finishes). After that initial 15 second startup, the fans go to silent. When testing point-to-point peak speeds over several minutes, the unit is still silent. It naturally gets warm to touch, but in my setup it is out of the way on a desk. I’m sure I can find a place for the cats to sit on it and enjoy.

The Final Word

As mentioned, I forked over my own cash for this hardware. At $36 a port, I’m still amazed that the first one that crossed my $50 line was a massive 24-port switch, so now I have overkill for whatever I have planned (ed: it may involve CPUs and motherboards). The key thing here, for me, will be my testing – every new testbed requires 100GB of CPU tests and 800G+ of gaming tests, so copying these over takes time. On a gigabit network, using my new Steam Cache means that at a speed of 70MB/s a big game like GTA5 can still take 13 minutes. I’m hoping that with 10G, if can push that transfer speed to SATA limits, that the total time will be down to around two minutes. There's also the possibility of doing some network card testing in the future now.

 

Related Reading

POST A COMMENT

55 Comments

View All Comments

  • praeses - Friday, September 28, 2018 - link

    "how do you review a switch anyway?"

    I can tell you how I review a switch (for work, then if it passes the test there I'll get a small one for home) by going through a checklist on whether it's viable in the first place:
    1) Does it have some sort of OOB management (serial, usb, dedicated management interface, or removable memory?)
    2) Does it leak any information without logging in? (welcome pages, etc.)
    3) Can it be recovered from a bad configuration (someone miss clicking a setting) via some sort of test before apply, undo, or reasonable reset configuration?
    4) How well does it handle firmware updates? (simple process or unnecessarily convoluted, how many reboots)
    5) How long does it take to reboot? 1 min, 5 min, (10 or more is an instant-fail)
    6) Can the configuration be exported/imported in some sort of text file (if not it's usually an instant-fail)
    7) Does it have SSH access? (If not it's usually an instant-fail)
    8) Can all functionality be changed by both SSH and the GUI?
    9) Are the commands contiguous or do different sections of the configuration use weird syntax compared to the rest like how some integrate features from acquired companies but have not yet standardized the language?
    10) How well does it handle multiple roles/permission levels/accounts?
    11) Can I have another tech work on it without knowing what some brand-specific lingo means?
    12) Does it automatically correct issues such as not allowing the default VLAN ID to bet set when a port is not a member of that VLAN?
    13) How many menus (command-line or gui) does it require to make changes such as VLANs?
    14) Does the GUI require any software to be downloaded (including web browser add-ons)? Does the software work with all platforms I use? Typically if it's not https/HTML5 works on all current browsers it's an instant-fail.
    15) How does it handle certificates for the web interface?
    16) What is the track record for the company to support a model line like this one for security/reliability updates?
    17) Does it require some crappy vendor locked in SFP module or DAC? Can it get proper SFP media link status and other statistics?

    Other things like SNMP, statistics, inventory, etc. are big bonuses too. Stuff like that I go through before doing performance testing. Performance testing seems to be wire-speed in most cases unless doing some layer 3 stuff so I tend to look at the management side of things first unless it's specifically for some SAN or similar.

    I think in a month or two PoE IEEE 802.3bz is supposed to be finalized so I'm mostly waiting on products supporting that.
    Reply
  • atomt - Saturday, September 29, 2018 - link

    18) Delivers packets in the correct order.
    Had a Mikrotik CSS326 fail on this during testing the other day. Seems that if a flow hits the port buffers packets gets dequeued in the wrong order.

    19) Advertised features like IGMP/MLD multicast snooping, DHCP guard/snooping/binding, IPv6 RA-Guard, Energy Efficient Ethernet actually works and not a huge pain to set up.

    I have a switch here that fucks up LLDP in a way that makes all link partners disable EEE if more than one EEE-supporting device is connected.

    20) Has sufficiently good buffer management to handle mismatched port speeds and other congestion properly. While not suffering from bufferbloat.
    Reply
  • praeses - Saturday, September 29, 2018 - link

    ooph.. yeah I have ran into 20 before and 18 I could see being a huge headache Reply
  • jumbo_ - Sunday, September 30, 2018 - link

    agreed. You see, reviewing switches is as complex as reviewing any kind of other IT gear. It is less about "speed", since switches use ASICs for forwarding the actual traffic which bypasses the control plane CPU and more about actual "experience". To add more on top of the above:
    21) will the vendor provide software/firmware upgrades 3/6/12/24/36 months after?
    22) port to port latency (typically in micro-seconds)
    23) does it crash every 2 weeks?
    24) verify performance claims - 20Gbps (in + out) in 70byte frames is something completely different than 20Gbps of 1500 byte frames
    24a) similar to above, just differrent traffic patterns - one to many and many to one
    25) verify 10/100/1000 Gbps support, half duplex, full duplex
    26) can you get basic operation information from the CLI/GUI that will allow you actually troubleshoot. This is surprisingly the biggest problem with SoHO switches.
    26a) current information about interface up/down, pps in, pps out, ouput buffer drops, L1 errors
    26b) historical information about 26a, CPU %
    26c) feature status, (spanning tree, snooping)
    Reply
  • BillyONeal - Monday, October 01, 2018 - link

    Most of that stuff is about *managed* switches that are wholly irrelevant in a small home / home office scenario like this. Reply
  • npz - Friday, September 28, 2018 - link

    I find that hard to believe unless there was some other variable involved. Maybe 2-3x but 10x?
    Did you compare back-to-back fibre with back-to-back (auto-xover) native 10Gbase-T RJ45 copper and NOT SFP copper, which adds delay in the transceiver?

    Or did you go through a switch for copper when comparing your direct attach fiber? If so was the switch a simple dump switch or a managed switch with some management enabled e.g. VLANs, QoS, etc?
    Reply
  • bcronce - Saturday, September 29, 2018 - link

    Was just looking at transceivers for SPF+ and DAC was 0.3us while 10Gbase-T was 2.3us. That's 8x right there. Reply
  • npz - Friday, September 28, 2018 - link

    edit: sorry that was meant to be a reply to @daviderickson
    https://www.anandtech.com/comments/13397/in-the-la...
    Reply
  • mode_13h - Saturday, September 29, 2018 - link

    Nice mini-review. I was surprised that one connection only saturated 1/3rd of the end-to-end bandwidth. Consider experimenting with jumbo frames, next.

    > I’m sure I can find a place for the cats to sit on it and enjoy.

    And get it clogged with cat fur? That will surely get the fans to start spinning...
    Reply
  • atomt - Saturday, September 29, 2018 - link

    *strokes the 32 port 40Gbe Arista switch currenly powering my home while breathing heavily*

    "ed: I still think you're insane"

    You can take the insanity further. I believe in you.
    Reply

Log in

Don't have an account? Sign up now