Is Bandwidth A Precious Resource?

During a recent episode of the Packet Pushers Podcast, Greg and Drew talked about the fact that bandwidth just keeps increasing and we live in a world where the solution to most problems is to just increase the pipeline to the data center or to the Internet. I came into networking after the heady days of ISDN lines everywhere and trying to do traffic shaping on slow frame relay links. But I also believe that we’re going to quickly find ourselves in a pickle when it comes to bandwidth.

Too Depressing

My grandparents were alive during the Great Depression. They remember what it was like to have to struggle to find food or make ends meet. That one singular experience transformed the way they lived their lives. If you have a relative or know of someone that lived through that time, you probably have noticed they have some interesting habits. They may keep lots of cash on hand stored in various places around the house. They may do things like peel labels from jelly jars and use them as cups. They may even go to great lengths to preserve as much as they can for reuse later “just in case”.

It’s not uncommon for this to happen in the IT world as well. People that have been marked by crazy circumstance develop defense mechanisms against. Maybe it’s making a second commit to a configuration to ensure it’s correct before being deployed. Maybe it’s always taking a text backup of a switchport before shutting it down in case an old bug wipes it clean. Or, as it relates to the topic above, maybe it’s a network engineer that grew up on slow ISDN circuits trying to optimize links for traffic when there is absolutely no need to do so.

People will work with what they’re familiar with. If they treat every link as slow and prone to congestion they’ll configure they QoS policies and other shaping features like they were necessary to keep a 128k link alive with a massive traffic load. Even if the link has so much bandwidth that it will never even trigger the congestion features.

Rational Exuberance

The flip side of the Great Depression grandparent is the relative that grew up during a time when everything was perfect and there was nothing to worry about. The term coined by Alan Greenspan to define this phenomenon was Irrational Exuberance, which is the idea that the stock market of 1996 was overvalued. It also has the connotation of meaning that people will believe that everything is perfectly fine and dandy when all is well right up to the point when the rug gets pulled out from underneath them.

Going back to our bandwidth example, think about a network engineer that has only ever known a world like we have today where bandwidth is plentiful and easily available. I can remember installing phone systems for a school that had gigabit fiber connectivity between sites. QoS policies were non-existent for them because they weren’t needed. When you have all the pipeline you can use you don’t worry about restraining yourself. You have a plethora of bandwidth capabilities.

However, you also have an issue with budgeting. Turns out that there’s no such thing as truly unlimited bandwidth. You’re always going to hit a cap somewhere. It could be the uplink from the server to the switch. Maybe it’s the uplink between switches on the leaf-spine fabric. It could even be the WAN circuit that connects you to the Internet and the public cloud. You’re going to hit a roadblock somewhere. And if you haven’t planned for that you’re going to be in trouble. Because you’re going to realize that you should have been planning for the day when your options ran out.

Building For Today

If you’re looking at a modern enterprise, you need to understand some truths.

  1. Bandwidth is plentiful. Until it isn’t. You can always buy bigger switches. Run more fiber. Create cross-connects to increase bandwidth between east-west traffic. But once you hit the wall of running out of switch ports or places to pull fiber you’re going to be done no matter what.
  2. No Matter How Much You Have, It Won’t Be Enough. I learned this one at IBM back in 2001. We had a T3 that ran the entire campus in Minnesota. They were starting to get constrained on bandwidth so they paid a ridiculous amount of money to have another one installed to increase the bandwidth for users. We saturated it in just a couple of months. No matter how big the circuit or how many you install, you’re eventually going to run out of room. And if you don’t plan for that you’re going to be in a world of trouble.
  3. Plan For A Rainy Day. If you read the above, you know you’re going to need to have a plan in place when the day comes that you run out of unlimited bandwidth. You need to have QoS policies ready to go. Application inspection engines can be deployed in monitor mode and ready at a moment’s notice to be enabled in hopes of restricting usage and prioritizing important traffic. Remember that QoS doesn’t magically create bandwidth from nothing. Instead, it optimizes what you have and ensures that it can be used properly by the applications that need it. So you have to know what’s critical and what can be left to best effort. That means you have to do the groundwork ahead of time so there are no surprises. You have to be vigilant too. Who would have expected last year that video conference traffic would be as important as it is today?

Tom’s Take

Bandwidth is just like any other resource. It’s not infinite. It’s not unlimited. It only appears that way because we haven’t found a way to fill up that pipe yet. For every protocol that tries to be a good steward and not waste bandwidth like OSPF, you have a newer protocol like Open/R that has never known the harsh tundra of an ISDN line. We can make the bandwidth look effectively limitless but only by virtue of putting smart limits in place early and understanding how to make things work more smoothly when the time comes. Bandwidth is precious and you can make it work for you with the right outlook.

How Much Is Unlimited Anyway?


The big news today came down from the Microsoft MVP Summit that OneDrive is not going to support “unlimited” cloud storage going forward. This came as a blow to folks that were hoping to store as much data as possible for the foreseeable future. The conversations have already started about how Microsoft pulled a bait-and-switch or how storage isn’t really free or unlimited. I see a lot of parallels in the networking space to this problem as well.

All The Bandwidth You Can Buy

I remember sitting in a real estate class in college talking to my professor, who was a commercial real estate agent. He told us, “The happiest day of your real estate career is the day you buy an apartment complex. The second happiest day of your career is when you sell it to the next sucker.” People are in love with the idea of charging for a service, whether it be an apartment or cloud storage and compute. They think they can raise the price every year and continue to reap the profits of ever-increasing rent. What they don’t realize is that those increases are designed to cover increased operating costs, not increased money in your pocket.

Think about someone like Amazon. They are making money hand over fist in the cloud game. What do you think they are doing with it? Are they piling it up in a storage locker and sitting on it like a throne? Or lighting cigars with $100 bills? The most likely answer is that they are plowing those profits back into increasing capacity and offerings to attract new customers. That’s what customers want. Amazon can take some profit from the business but if they stop expanding customers will leave to find another service that meets their needs.

Bandwidth in networks is no different. I worked for IBM as in intern many years ago. Our site upgraded their Internet connection to a T3 to support the site. We were informed that just a few months after the upgrade that all the extra bandwidth we’d installed was being utilized at more than 90%. It took almost no time for the users to find out there was more headroom available and consume it.

The situation with bandwidth today is no different. Application developers assume that storage and bandwidth are unlimited or significant. They create huge application packages that load every conceivable library or function for the sake of execution speed. Networking and storage pays the price to make things faster. Apps take up lots of space and take forever to download even a simple update. The situation keeps getting worse with every release.

Slimming the Bandwidth Pipeline

Some companies are trying to take a look at how to keep this bloat from exploding. Facebook has instituted a policy that restricts bandwidth on Tuesdays to show developers what browsing at low speeds really feels like. They realize that not everyone in the world has access to ultra-fast LTE or wireless.

Likewise, Amazon realizes that on-boarding data to AWS can be painful if there are hundreds of gigabytes or even a few terabytes to migrate. They created Snowball, which is essentially a 1 petabyte storage array that you load up on-site and return to Amazon to store. It’s a decidedly low tech solution to a growing problem.

Networking professionals know that bandwidth isn’t unlimited. Upgrades and additional capacity cost money. Service providers have the same limitations as regular networks. If you want more bandwidth than they can provide, you are out of luck. If you’re willing to pay through the nose providers are happy to build out solutions for you. You’re providing the capital investment for their expansion. Everything costs money somehow.

Tom’s Take

“Unlimited” is a marketing lie. Whether it’s unlimited nights and weekends, unlimited mobile data, or unlimited storage, nothing is truly infinite. Companies want you to take advantage of their offerings to sell you something else. Free services are supported by advertising or upset opportunities. Providers continue to be shocked when they offer something with no reasonable limit and find that a small percentage of the user base is going to take advantage of their mistake.

Rather than offering false promises of unlimited things, providers should be up front. They should have plans that offer large storage amounts with conditions that make it clear that large consumers of those services will face restriction. People that want to push the limit and download hundreds of gigabytes of mobile data or store hundreds of terabytes of data in the cloud should know up front that they will be singled out for special treatment. Believable terms for services beat the lies of no limits every time.