Pete Lumbis is a Technical Marketing Engineer at Cumulus Networks.
The change to 100G is happening at a rapid pace, and soon it will be the go-to speed for networks (opens in new tab). It is, of course, no secret that speed makes a noticeable difference to your network. As a result, 100G is soon going to be a necessity for many organisations going forward.
Before you jump headfirst into changing your business's networking (opens in new tab) capabilities though, there are some things your need to consider, because increasing network speed is more than just pushing more bits across a wire. Below are outlined some top tips to make sure the switch is as effortless as possible.
Decide what you really need
First things first – you need to decide what speed you really need for your future network (opens in new tab). A single 100G connection can be broken out into four channels or lanes of 25G each. This is why 100G networking is sometimes called 25G/100G. Normally this will mean that you run 100G connections between your switches, and 25G to your servers (opens in new tab).
If you’re thinking of jumping from 10G to 40G to the server, 25G is probably the better choice. A single 100G port can be broken up into 2x50G links, giving you the option to connect two servers at 50G each. This way, you can connect the same number of servers, get more bandwidth, and use fewer ports. It’s a win-win! This is compared to running 40G to two servers instead. You’d have to consume two switch ports instead of one, and you’d still be getting less bandwidth.
What you should also be aware of is being able to precisely clock synchronize between systems, which is especially important when moving from 10G to either 40G or 100G. When getting switches from different vendors (which is touched upon in more detail below) having any clocking issues can increase latency and packet loss.
The bottom line: use 25G/100G where you need more bandwidth and more flexibility in the rack. Stick with 10G/40G where you have the capacity to run additional connections without having to buy more switches.
Know your reach
The distances at which you can get 100G don’t differ too much from 10G/40G. But there’s more to keep in mind than just distance. If you need long reach optics, some switches limit you to specific ports (opens in new tab). Also, some switches will limit how many 100G ports you can break out into 4x25G or 2x50G connections. These limitations can affect how much density you can realistically achieve.
The reach of the optics is a big consideration to make, because essentially it could lead to re-wiring of your network depending on what you already have in place. Of course, that would be expensive, and disruptive to your business. Because 100G is a whole new ballgame, you will have a new generation of optics and cables that you will have to deal with.
Making sure you understand what you need to change, or what you already have in place can make a big difference to not only the time it takes to set up the new network, but also how far you will have to dig in your pockets to get the infrastructure right.
Making sure your switches are compatible
To support your 10G/40G connections, your new leaf switches will need to support your existing QSFP+ (quad (4-channel) small form-factor pluggable) and SFP+ ( small form-factor pluggable transceiver) modules.
When shopping for new leaf switches – which are used in data centers to aggregate traffic from server nodes - look for switches that accept the QSFP28 form factor, which is physically capable of taking existing QSFP+ modules. If you have SFP+ modules you want to use, make sure that your new switches also accept the SFP28 form factor.
You need to make sure your switches have plenty of QSFP28 and SFP48 and can do 10G/40G before you make the change.
The problem with early models
There is a potential bump in the road, and what you need to know is that many 100G switches came out before the current 25G/100G standards were ratified in 2016. Because of this, the switches don’t always work together, and the point above on same vendor switches comes in to play here. This means that for any hardware refresh down the road (opens in new tab), you may find that the early models aren’t compatible with newer switches.
The best advice here is to stay away from switches that were released before the current 25G/100G standards were adopted to save yourself any potential headaches later on.
Ultimately when you are thinking of changing to 100G make sure you know what you really need before overhauling your whole system. As with any networking changes, there are some pre-checks you need to do: do you have the capability, and what reach to do you have (and what do you need); and finally make sure your switches are compatible with any changes. Following these steps will help you effortlessly switch across to 100G and get your network up to speed - both figuratively and literally.
Pete Lumbis is a Technical Marketing Engineer at Cumulus Networks (opens in new tab).