Tag Archives: 800GbE

The Heat is On

Posted on by
Reply

One of the things I like to do in my twenty-eight minutes of spare time per week is play Battletech. It’s a table top wargame that involves big robots and lots of weapons. Some of them are familiar, like missiles and artillery. Because it’s science fiction there are also lasers and other crazy stuff. It’s a game of resource allocation. Can my ammunition last through this fight? You might be asking yourself “why not just carry lots of lasers?” After all, they don’t need ammo. Except the game designers thought of that too. Lasers produce heat. And heat, like ammunition, must be managed. Generate too much and you will shut down. Or boil your pilot alive in the cockpit. Rewind a thousand years and the modern network in a data center is facing a similar issue.

Watt Are You Talking About?

The average AI rack is expected to consume 600 kilowatts of power by next year. GPUs and CPUs are hungry beasts. They need to be fed as much power as possible in order to do whatever math makes AI happen. They have to come up with creative ways to cool those devices as well. We’re quickly reaching the limits of air cooling, with new designs using liquid cooling and even immersion in mineral oil to keep these machines from frying everything.

Networking is no slouch in the energy consumption department either. The latest generation of networking devices are ramping up to 800 GbE and consuming a significant amount of energy to keep the bandwidth flowing. One of the biggest consumers of power in these racks is the digital signal processor (DSP) that are clustered inside fiber optic modules. You have to have a lot of DSPs to condition the light that allows for operations at the top end of the scale. Additionally, the CPUs and ASICs themselves inside the units are running at peak performance to provide the resources AI clusters need to return value.

Worse yet, the very design of the network switch works against traditional air cooling. Servers have pretty face plates with air channels that can pull in air and vent it out the back. Nothing in the way except for maybe a USB stick when you’re loading the operating system. Switches, on the other hand, plug in all the cables up front. Those cables and modules impede air flow and reduce the amount of cooling the switch is capable of producing because of simple physics. If you’ve ever put your hand in front of a modern switch you know that it can pull in a lot of air. But every connection reduces that and creates a feedback loop.

Chill Out

Just like with the AI cluster servers, we have to deal with the heat generation. Unlike the AI servers, we can’t just dunk them in oil or build a bigger chassis with more fans to bleed it off. I mean, we technically could if necessary. But when is the last time you saw a top-of-rack (ToR) switch bigger than 2U? That’s because the real value in the rack isn’t the network. It’s the server that the network connects. Every rack unit of space you give to the networking gear or anything that isn’t a server is a wasted RU that could have been generating revenue.

Modern networking devices have hit 800 GbE but AI wants more. There are designs that are looking to move to 1.6 TbE in the near future. But can they handle the load of DSPs that will be required to condition the laser light? Will they be able to reduce the size of this switches down to accommodate rack economics? Can they keep the whole mess cool enough that it won’t melt anyone that walks into the hot aisle to check on a cable before getting roasted by a flame throwing?

Plug It In

One of the solutions that looks to reduce this is Linear Pluggable Optics (LPO). LPO works to solve these issues by moving the DSPs out of the pluggable module itself and into the switch ASIC. This does do a good job of reducing the power draw of the optical module itself while moving the complexity of signal conditioning into the switch. The net result is less power consumption but the tradeoff is more complexity. To quote Battletech YouTuber Mechanical Frog, “Opportunity cost spares no one.”

Likewise, a competing solution like Co-packed Optics (CPO) trades the module itself for wiring the optical portion of connection right into the switch. This eliminates the need for DSPs to condition the signal and significantly reduces power and heat generation but at the cost of the flexibility of using modules in the first place. I’ll have more to say on CPO in the future.

Tom’s Take

The thing to think about is that the technology we’re using creates the bounds that we have to work within for other technology to build on. We have to deliver networking that doesn’t stop for AI to function the way the designers want. To them, the value of AI is not being more power efficient or generating less heat. Instead, it’s about crunching numbers faster or making better pictures that people want to generate. Data centers have an unlimited power budget and heat removal capabilities as far as AI companies are concerned. Reality dictates that nothing is unlimited but it’s up to the manufacturers to scale those limits before someone hits them. Otherwise, the network is just going to get blamed again. And the heat really gets turned up.

Networking Is Fast Enough

Posted on by
2

Without looking up the specs, can you tell me the PHY differences between Gigabit Ethernet and 10GbE? How about 40GbE and 800GbE? Other than the numbers being different do you know how things change? Do you honestly care? Likewise for Wi-Fi 6, 6E, and 7. Can you tell me how the spectrum changes affect you or why the QAM changes are so important? Or do you want those technologies simply because the numbers are bigger?

The more time I spend in the networking space the more I realize that we’ve come to a comfortable point with our technology. You could call it a wall but that provides negative connotations to things. Most of our end-user Ethernet connectivity is gigabit. Sure, there are the occasional 10GbE cards for desktop workstations that do lots of heavy lifting for video editing or more specialized workflows like medical imaging. The rest of the world has old fashioned 1000Mb connections based on 802.3z ratified in 1998.

Wireless is similar. You’re probably running on a Wi-Fi 5 (802.11ac) or Wi-Fi 6 (802.11ax) access point right now. If you’re running on 11ac you might even be connected using Wi-Fi 4 (802.11n) if you’re running in 2.4GHz. Those technologies, while not quite as old as GigE, are still prevalent. Wi-Fi 6E isn’t really shipping in quantity right now due to FCC restrictions on outdoor use and Wi-Fi 7 is a twinkle in hardware manufacturers’ eye right now. Why aren’t we clamoring for more, faster, better, stronger all the time?

Speedometers

How fast can your car go? You might say you’ve had it up to 100 mph or above. You might take a look at your speedometer and say that it can go as high as 150 mph. But do you know for sure? Have you really driven it that fast? Or are you guessing? Would you be shocked to learn that even in Germany, where the Autobahn has an effectively unlimited speed limit, that cars are often limited to 155 mph?. Even though the speedometer may go higher the cars are limited through an agreement for safety reasons. Many US vehicles are also speed limited between 110 and 140 mph.

Why are we restricting the speeds for these vehicles? Safety is almost always the primary concern, driven by the desire for insurance companies to limit claims and reduce accidents. However, another good reason is also why the Autobahn has a higher effective speed limit: road conditions. My car may go 100 mph but there are very few roads in my part of the US that I would feel comfortable going that fast on. The Autobahn is a much better road surface for driving fast compared to some of the two-lane highways around here. Even if the limit was higher I would probably drive slower for safety reasons. The roads aren’t built for screaming speeds.

That same analogy applies to networking. Sure, you may have a 10GbE connection to your Mac Mini and you may be moving gigs of files back and forth between machines in your local network. What happens if you need to upload it to Youtube or back it up to cloud storage? Are you going to see those 10GbE speeds? Or are you going to be limited to your ISP’s data rates? The fastest engine can only go as fast the pathways will permit. In essence, that hot little car is speed limited because of the pathway the data takes to the destination.

There’s been a lot of discussion in the space about ever-increasing connectivity from 400GbE to 800GbE and soon even into the terabit range. But most of it is specialized for AI workloads or other massive elephant flows that are delivered via a fabric. I doubt an ISP is going to put in an 800GbE cross connect to increase bandwidth for consumers any time soon. They won’t do it because they don’t need to. No consumer is going to be running quite that fast.

Likewise, increasing speeds on wireless APs to more than gigabit speeds is silly unless you want to run multiple cables or install expensive 10GbE cards that will require new expensive switches. Forgetting Multigig stuff for now you’re not going to be able to plug in a 10GbE AP to an older switch and get the same performance levels. And most companies aren’t making 10GbE campus switches. They’re still making 1GbE devices. Clients aren’t topping out their transfer rates over wireless. And even if they did they aren’t going to be going faster than the cable that plugs the AP into the rest of the network.

Innovation Idling

It’s silly, right? Why can’t we make things go faster?!? We need to use these super fast connections to make everything better. Yet somehow our world works just fine today. We’ve learned to work with the system we have. Streaming movies wouldn’t work on a dial-up connection but adding 10GbE connections to the home won’t make Netflix work any faster than it does today. That’s because the system is optimized to deliver content just fast enough to keep your attention. If the caching servers or the network degrades to the point where you have to buffer your experience is poor. But so long as the client is getting streaming data ahead of you consuming it you never know the difference, right?

Our networks are optimized to deliver data to clients running on 1GbE. Without a massive change in the way that workloads are done in the coming years we’re never going to be faster than that. Our software programs might be more optimized to deliver content within that framework but I wouldn’t expect to see 10GbE become a huge demand in client devices. Frankly, we don’t need that much speed. We don’t need to run flat out all the time. Just like a car engine we’re more comfortable running at a certain safe speed that preserves our safety and the life of the equipment.

Tom’s Take

Be honest with yourself. Do you want 10GbE or Wi-Fi 7 because you actually need the performance? Or do you just want to say you have the latest and greatest? Would you pay extra for a v12 engine in a sports car that you never drive over 80 mph? Just to say you have it? Ironically enough, this is the same issue that cloud migrations face today. We buy more than we need and never use it because we don’t know what our workloads require. Instead, we buy the fastest biggest thing we can afford and complain that something is holding it back. Rather than rushing out to upgrade your Wi-Fi or Ethernet, ask yourself what you need, not what you want. I think you’ll realize the network is fast enough for the foreseeable future.