Thoughts on Cisco Live 2015

Cisco Live 2015 Twitter Pic

We’ve secretly replaced Tom with Mike Rowe. Let’s see if anyone notices…

Cisco Live 2015 is in the books. A great return to San Diego. A farewell from John Chambers. A greeting from Chuck Robbins (@ChuckRobbins). And a few other things.

The Community is Strong, But Concerned

The absolute best part of Cisco Live is the community that has grown from the social media attendees. More than once I heard during the week “I can’t believe this used to be 20-30 people!”. The social community continues to grow and change. Some people move on. Others return from absence. Still others are coming for the first time.

The Cisco Live social community is as inclusive as any I have seen. From the Sunday night Tweetup to the various interactions throughout the week, I’m proud to be a part of a community that strives to make everyone feel like they are part of a greater whole. I met so many new people this year and marveled at the way the Social Media Hub and Meetup Area were both packed at all hours of the day.

That being said, the community does have some concerns. Some of them are around institutionalized community. There was worry that bringing so many people into the Champions community threatened to marginalize the organic community that had grown up in the past six years. While some of that worry was quieted by the end of the show, I think the major concerns are still present and valid to a certain degree. I think a discussion about the direction of the Champion program and how it will interact with other organic communities is definitely in order sooner rather than later.

Gamification Continues, And I’m Not A Fan

Many of the activities at Cisco Live revovled around prizes and giveaways for interaction. As we’ve seen throughout the years, any time a prize is awarded for a game there is going to be some trying to work the system. I even mentioned it here:

I’m all for having fun. But the reward for a well-played game should be in the game itself. When things have to be modified and changed and curated to ensure no one is taking advantage, it stops being fun and starts being a competition. Competitions cause hurt feelings and bad blood. I think it’s time to look at what the result of this gamification is and whether it’s worth it.

Power Transitions And Telling The Story Right

As expected, John Chambers gave his farewell as CEO and introduced Chuck Robbins to the Cisco Live community. By all accounts, it was an orderly transfer of power and a great way to reassure the investors and press that things are going to proceed as usual. I was a bit interested in the talk from Chambers about how this transition plan has been in place for at least ten months. Given the discussion in the tech press (and more than a couple private comments), the succession wasn’t a smooth as John lets on. Maybe it’s better that the general Cisco public not know how crazy the behind-the-scenes politics really were.

Chuck finds himself in a very precarious position. He’s the person that follows the legend. Love him or hate him, Chambers has been the face of Cisco forever. He is the legend in the networking community. How do you step into his shoes? It’s better that John stepped down on his own terms instead of being forced out by the board. Chuck has also done a great job of rolling out his executive team and making some smart moves to solidify his position at the top.

The key is going to be how Chuck decides to solidify the businesses inside of Cisco. Things that were critical even two years ago are shrinking in the face of market movement. John’s speech was very pointed: there is another tranisition coming that can’t be missed. Chuck has a hard road ahead trying to stabilize Cisco’s position in the market. A cheeky example:

Cisco has missed transitions, SDN being the most recent. They need to concentrate on what’s important and remove the barriers to agile movement. A start would be cutting back on the crazy amounts of business units (BUs) competing for face time with the CEO. You could easily consolidate 50% of the organizations inside Cisco and still have more than anyone else in networking. A racecar that goes 200 mph is still unstable if it isn’t streamlined. Chuck needs to cut Cisco down to fighting weight to make the story sound right.

Cisco Finally Understands Social, But They Don’t Quite Get It (Yet)

I applaud the people inside of Cisco and Cisco Live that have fought tooth and nail for the past few years to highlight the importance of social. Turning a ship the size of Cisco can’t be easy, but it’s finally starting to sink in how powerful social media can be. I can promise you that Cisco understands it better than companies like IBM or Oracle. That’s not to say that Cisco embraces social like it should.

Cisco is still in the uncomfortable mode of using social as a broadcast platform rather than an interaction tool. There are some inside of Cisco that realize the need to focus on the audience rather than the message. But those are exceptions to the general rule of being “on message”.

Social media is a powerful tool to build visibility of personalities. The messenger is often more important than the message. Just ask Pheidippides. Allow your people the freedom to develop a voice and be themselves will win you more converts than having a force of robots parroting the same platitudes on a scheduled basis.

Cisco has some great people invovled in the community. Folks like J Metz (@DrJMetz), Rob Novak (@Gallifreyan), and Lauren Friedman (@Lauren) how how dedicated people can make a name for themselves separate from their employer. Cisco would do well to follow the example of these folks (and many others) and let the messengers make the audience they key.


Tom’s Take

Thanks to Tech Field Day, I go to a lot of industry events now. But Cisco Live is still my favorite. The people make it wonderful. The atmosphere is as electric as any I’ve been a part of. This was my tenth Cisco Live. I can’t imagine not being a part of the event.

Yes, I have concerns about some of the things going on, but it’s the kind of concern that you have for a loved one or dear friend. I want people to understand the challenges of keeping Cisco Live relevant and important to attendees and find a way to fix the issues before they become problems. What I don’t want to see is a conference devoid of personality and wonderful people going through the motions. That would not only destroy the event, but the communities that have sprung from it as well.

Cisco Live 2016 will be intensely personal for me. It’s the first return to Las Vegas since 2011. It’s also the fifth anniversary of Tom’s Corner. I want to make the next Cisco Live as important as Cisco Live 2011 was for me. I hope you will all join me there and be a part of the community that has changed my life for the better.

 

There’s No Such Thing As Free Wireless

Wireless

If you’ve watched any of the recent Wireless Field Day presentations, you know that free wireless is a big hot button issue. The delegates believe that wireless is something akin to a public utility that should be available without reservation. But can it every really be free?

No Free Lunches

Let’s take a look at other “free” offerings you get in restaurants. If you eat at popular Mexican restaurants, you often get free tortilla chips and salsa, often called a “setup”. A large number of bars will have bowls of salty snacks waiting for patrons to enjoy between beers or other drinks. These appetizers are free so wireless should be free as well, right?

The funny thing about those “free” appetizers is that they aren’t really free. They serve as a means to an end. The salty snacks on the bar are there to make you thirsty and cause you to order more drinks to quench that thirst. The cost of offering those snacks is balanced by the amount of extra alcohol you consume. The “free” chips and salsa at the restaurant serve as much to control food costs as they do to whet your appetite. By offering cheap food up front, people are less likely to order larger food dishes that cost more to make. And if you don’t want to enjoy food from the menu, most restaurants will charge you a “nominal” fee to recoup their costs.

These “free” items serve to increase sales for the business. Business don’t mind giving things away as long as they can profit from them. The value proposition of a free service has to be balanced with some additional revenue source. In that sense, nothing is really and truly free from an altruistic point of view.

Anal(ytics) Retentive

The path to offering “free” wifi seems to be headed down the road of collecting information about users in order to offer services to recoup costs. Whether it be through a loyalty programs or social wiereless logins, companies are willing to give you access to wireless in exchange for some information about you.

The tradeoff is reasonable in the eyes of the business. They have to upgrade their infratructure to support transient guest users. It’s one thing to offer guest wireless to employees who are on the payrool and being productive. It’s something else entirely to offer it to people who will potentially use it and not your services. You have to have a way to get that investment back.

For a large percentage of the population, giving away information is something they dont’ care about. It’s something freely available on social media, right? If everyone can find out about it, might as well give it to someone in exchange for free wireless, right?

Despite what people have said as of late, the real issues with social login and data analytics have nothing to do with offering the data. Storing the data somewhere is of little consquence in the long run. So long as a compnay doesn’t attempt to use that data against you in some way then data collection is benign.

Yes, storing that data could be problematic thanks to the ever-shrinking timeline for exposing large databases inside companies. Data sitting around in a database has a siren call to companies to either do something with it or sell it to a third party in an effort to capitalize on the gold mine they are sitting on. But the idea that most people have is that won’t happen. That makes it tolerable to give away something meaningless in exchange for a necessary service.


Tom’s Take

The price of freedom is vigilance. The price of free wireless is a little less than that. Business owners need value to offer additional services. Cost with no return gives no value. Whether that value comes from increased insight into customer bases or reselling that data to someone that wants to provide analytics services to businesses is a moot point. Wireless will never truly be free so long as there is something that can be traded for its value.

Can Community Be Institutionalized?

CommunityPlanning

As technology grows at a faster pace, companies are relying more and more on their users to help spread the word about what they are doing. Why pay exorbitant amounts for marketing when there is a group of folks that will do it for little to nothing? These communities of users develop around any product or company with significant traction in the market. But can they be organized, built, and managed in a traditional manner?

Little Pink Houses

Communities develop when users start talking to each other. They exist in numerous different forms. Whether it be forum posters or sanctioned user groups or even unofficial meetups, people want to get together to talk about things. These communities are built from the idea that knowledge should be shared. Anecdotes, guides, and cautionary tales abound when you put enough people into a room and get them talking about a product.

That’s not to say that all communities can be positive ones. Some communities are even built around the idea of a negative reaction. Look at these groups that formed around simple ideas like getting their old Facebook page back or getting their old MySpace layout returned to them. Imagine the reaction that you get when you have a enterprise product that makes changes that users don’t like. That’s how communities get started too.

Whether they are positive or negative, communites exist to give people a way to interact with other like-minded individuals. Community is a refuge that allows members to talk freely and develop the community to suit their needs.

Community Planning

What happens when the community needs more direction? Some communities are completely sanctioned and sponsored by their subjects, like the VMware User Group (VMUG). Others are independent but tend to track along with the parent, such as the Cisco User Groups that have developed over the years. These tend to be very well organized versus other more informal communities.

With the advent of social media, many ad hoc communities have formed quickly around the idea of sharing online. Social media makes meeting new members of the community quick and easy. But it’s also difficult to control social communities. They grow and change so rapidly that even monitoring is a challenge.

The wild and unpredictable nature of social communities has led to a new form of sponsored community – the influencer outreach program. These programs have different names depending on the company, but the idea is still roughly the same: reach out to influencers and social media users in the immediate community and invite them into a new community that offers incentives like insider information or activities outside of those regularly available to everyone.

Influencer outreach programs are like a recipe. You must have the right mix in the correct proportions to make everything work. If you have too much of something or not enough of another, the whole construct can fall apart. Too many members leads to a feeling of non-exclusiveness. Too few members-only briefings leads to a sense that the program doesn’t offer anything over and above “normal” community membership.

The Meringue Problem

One of the most important things that influencer outreach communities need to understand is something I call the “Meringue Problem”. If you’ve ever made meringue for a dessert, you know that you have to whip the egg whites and sugar until it forms soft peaks. That’s what makes meringue light and fluffy. It’s a lot of work but it pays off if done right. However, if you whip the mixture too hard or too long, those soft peaks fall apart into a mess that must be thrown out.

The Meringue Problem in influencer outreach communities comes when the program organizers and directors (chefs) get too involved in directing the community. They try to direct things too much or try to refocus the community away towards an end that the community may or may not support wholeheartedly. That ends up creating animosity among the members and a feeling that things would be better if everyone “would just back off for a bit”. There are a hundred different reasons why this overinvolvement happens, but the results are always the same: a fractured community and a sense of disappointment.

The First Rule

If you want a textbook method for building a community, take a page from one of my favorite movies – Fight Club. Tyler and the Narrator start a community dedicated to working out agression through physical expression. They don’t tell everyone in the bar to come outside and start fighting. They just do their thing on their own. When others want to be invovled, they are welcomed with open arms (and closed fists).

Later, the whole idea of Fight Club takes on a life of it’s own. It becomes a living, breathing thing that no one person can really direct anymore. In the movie, it is mentioned that the leader moves among the crowd, with the only important thing being the people fighting in the ring. But it’s never exclusionary. They’ll let anyone join. Just ask Lou.

Tyler finally decides that he needs something more from Fight Club. So what does he do? Does he try to refocus the community to a new end? How can you control something like that? Instead, he creates a new community from a subset of the Fight Club members. Project Mayhem is still very much a part of Fight Club, as the space monkeys are still Fight Club members. But Project Mayhem is a different community with different goals. It’s not better or worse. Just…different.


Tom’s Take

I’m a proud member of several communities. Some of them are large and distinguished. Others are small and intimate. In some, I’m a quiet member in the back. In others I help organize and direct things. But no matter who I am there or what I’m doing, I remember the importace of letting the community develop. Communities will find their way if you let them. A guiding hand sometimes does help the community accomplish great things and transcend barriers. But that hand must guide. It should never force or meddle. When that line is cross, the community ceases being a collection of great people and starts taking on attributes that make it more important thant the members. And that kind of institutionalization isn’t a community at all.

Special thanks to Jeff Fry (@FryGuy_PA) and Stephen Foskett (@SFoskett) for helping me collect my thoughts for this post.

Just. Write.

955951_28854808

Somewhere, someone is thinking about writing. They are confused where to start. Maybe they think they can’t write well at all? Perhaps they even think they’ll run out of things to say? Guess what?

Just. Write.

Why A Blog?

Social media has taken over as the primary form of communication for a great majority of the population. Status updates, wall posts, and picture montages are the way we tell everyone what we’re up to. But this kind of communication is fast and ephemeral. Can you recall tweets you made seven months ago? Unless you can remember a keyword, Twitter and Google do a horrible job of searching for anything past a few days old.

Blogs represent something different. They are the long form record of what we know. They expand beyond a status or point-in-time posting. Blogs can exist for months or years past their original post date. They can be indexed and shared and amplifed. Blogs are how we leave our mark on the world.

I’ve been fielding questions recently from a lot of people about how to get started in blogging. I’m a firm believer that everyone has at least one good blog post in them. One story about a network problem solved or a cautionary tale that they’ve run into and wish to save others from. Everyone knows one thing that few others do. Sharing that one thought is what sets you apart from others.

A lot of blogs start off as a collection of notes or repository of knowledge that is unique to the writer. This makes it easy to share that knowledge with others. As people find that knowledge, they want to share it with others. As they share it, you become more well known in the community. As people learn who you are, they want to share with you. That’s how a simple post can start an avalanche of opportunity to learn more and more.

How To Start

This is actually much easier than it appears. Almost everyone has a first “Starting A Blog” post. It’s a way to announce to the world that your site is more than a parking page. That post is easy to write. And it will hardly ever be read.

The next step is to tell the world your one thing. Create pictures if it helps. Craft a story. Lay out all the information. Make sure to break it up into sections. It doesn’t have to be fancy, just readable. Once you’ve gotten all the information out of your head and onto virtual paper, it’s time to tell the world about it.

Publicize your work through those social media channels. Link to your post on Facebook, Twitter, LinkedIn, and anywhere else you can. The more eyeballs you get on your post, the more feedback you will get. You will also get people that share your post with others. That’s the amplication effect that helps you reach even further.

Now, Keep Going

Okay, you’ve gotten that out of your system. Guess what? You need to keep going. Momentum is a wonderful thing. Now that you’ve learned how to craft a post, you can write down more thoughts. Other stories you want to tell. Maybe you had a hard time configuring a switch or learning what a command does? Those are great posts to write and share. The key is to keep going.

You don’t need a set schedule. Some write every week to keep on track. Others write once a month to sum up things they are working on. The key is to find a schedule that works for you. Maybe you only do something interesting every two weeks? Maybe your job is so exicting that you can fill a whole week’s worth of posts?

The worst thing in the world is to have a rhythm going and then stop. Real life does get in the way more than you think. Jobs run long. Deadlines come and go. Missing a post becomes two. Two becomes three, and before you know it you haven’t posted for six months or more.

The way to fix the momentum problem is to keep writing things down. It doesn’t have to be a formal post. It can easily be bullet points in a draft. Maybe it’s a developing issue that you’re documenting? Just jot down the important things and when you’ve wrapped up all the hard work, you have all the beginnings of a great blog post. Every support case has notes that make for great blog subjects.


Tom’s Take

I still consider blogging to be one of the most important ways to share with people in our community. It’s very easy to write down tweets and status updates. It’s also very hard to find them again. Blogs are like living resumes for us. Everything we’ve done, everything we know is contained in several thousand characters of text that can be searched, indexed, and shared.

If you are sitting in a chair reading this thinking that you can’t blog, stop. Open a text editor and just start writing. Write about screwing up a VLAN config and how you learned not to do it again. Write down an interesting support case. Maybe it was a late-night migration gone wrong. Just write it down. When you finish telling your story, you’ve got the best start to blog you could possibly hope for. The key is to just write.

 

Open Choices In Networking

neo-architect

I had an interesting time at the spring meeting of the Open Networking User Group (@ONUG_) this past week. There were lots of discussions about networking, DevOps, and other assorted topics. One that caught me by surprise was some of the talk around openness. These tweets from Lisa Caywood (@RealLisaC) were especially telling:

After some discussion with other attendees, I think I’ve figured it out. People don’t want an open network. They want choice.

Flexible? Or Predictable?

Traditional networking marries software and hardware together. You want a Cisco switch? It runs IOS or NX-OS. Running Juniper? You can have any flavor of OS you want…as long as it’s Junos. That has been the accepted order of things for decades. Flexibility is traded for predictability. Traditional networking vendors give you many of the tools you need. If you need something different, you have to find the right mix of platform and software to get your goal accomplished. Mixing and matching is almost impossible.

This sounds an awful lot like the old IBM PC days. The same environment that gave rise to whitebox computers. We have a whitebox switching movement today as well for almost the same reasons – being able to run a different OS on cheaper hardware to the same end as the traditional integrated system. In return, you gain back that flexibility that you lost. There are some tradeoffs, however.

In theory, a whitebox switch is marginally harder to troubleshoot than a traditional platform. Which combiantion of OS and hardware are you running? How do those things interact to create bugs? Anyone that has ever tried to install USB device drivers on Windows knows that kind of pain. Getting everything to work right can be rough.

In practice, the support difference is negligible. Traditional vendors have a limited list of hardware, but the numerous versions of software (including engineering special code) interacting with those platforms can cause unforseen consequences. Likewise, most third party switch OS vendors have a tight hardware compatibility list (HCL) to ensure that everything works well together.

People do like flexibility. Giving them options means they can build systems to their liking. But that’s only a part of the puzzle.

The Problem Is Choice

Many of the ONUG attendees I talked to liked the idea of whitebox switching. They weren’t entirely enamoured, however. When I pressed a bit deeper, a pattern started to emerge. It sounded an awful lot like this:

I don’t want to run Vendor X Linux on my switch. I want to run my Linux on a switch!

That issue highlighted the real issue. Open networking proponents don’t want systems that offer open source networking enhancing the work of all. What they want is a flexible network that is capable of letting them run what they want on things.

The people that attend conferences like ONUG don’t like rigid choice options. Telling them they can run IOS or Junos is like picking the lesser of two evils. These people want to have a custom OS with the bare minimum needed to support a role in the network. They are used to solving problems outside the normal support chain. They chafe at the idea of being forced into a binary decision.

That goes back to Lisa’s tweets. People don’t want a totally open network running Quagga and other open source solutions. They want an open architecture that lets them rip and replace solutions based on who is cheaper that week or who upset them at the last account team meeting. They want the freedom to use their network as leverage to get better deals.

It’s a whole lot easier to get a better discount when you can legitimately threaten to have the incumbent thrown out and replaced relatively easily. Even if you have no intentions of doing so. Likewise, new advances in whitebox switching give you leverage to replace sections of the network and have feature parity with traditional vendors in all but a few corner cases. It seems to be yet another redefinition of open.


Tom’s Take

Maybe I’m just a cynic. I support development of software that makes the whole world better. My idea of open involves people working together to make everything better. It’s not about using strategies to make just my life easier. Enterprises are big consumers of open technologies with very little reciprocity outside of a few forward thinkers.

Maybe the problem is that we’ve overloaded open to mean so many other things that we have congnitive dissonance when we try to marry the various open ideas together? Open network architecture is easy as long as you stick to OSPF and other “standard” protocols. Perhaps the problem of choice is being shortsighted enough to make the wrong one.

 

The Light On The Fiber Mountain

MountainRoad

Fabric switching systems have been a popular solution for many companies in the past few years. Juniper has QFabric and Brocade has VCS. For those not invested in fabrics, the trend has been to collapse the traditional three tier network model down into a spine-leaf architecture to optimize east-west traffic flows. One must wonder how much more optimized that solution can be. As it turns out, there is a bit more that can be coaxed out of it.

Shine A Light On Me

During Interop, I had a chance to speak with the folks over at Fiber Mountain (@FiberMountain) about what they’ve been up to in their solution space. I had heard about their revolutionary SDN offering for fiber. At first, I was a bit doubtful. SDN gets thrown around a lot on new technology as a way to sell it to people that buy buzzwords. I wondered how a fiber networking solution could even take advantage of software.

My chat with M. H. Raza started out with a prop. He showed me one of the new Multifiber Push On (MPO) connectors that represent the new wave of high-density fiber. Each cable, which is roughly the size and shape of a SATA cable, contains 12 or 24 fiber connections. These are very small and pre-configured in a standardized connector. This connector can plug into a server network card and provide several light paths to a server. This connector and the fibers it terminates are the building block for Fiber Mountain’s solution.

With so many fibers running a server, Fiber Mountain can use their software intelligence to start doing interesting things. They can begin to build dedicated traffic lanes for applications and other traffic by isolating that traffic onto fibers already terminated on a server. The connectivity already exists on the server. Fiber Mountain just takes advantage of it. It feels very simliar to the way we add in additional gigabit network ports when we need to expand things like vKernel ports or dedicated traffic lanes for other data.

Quilting Circle

Where this solution starts looking more like a fabric is what happens when you put Fiber Mountain Optical Exchange devices in the middle. These switching devices act like aggregation ports in the “spine” of the network. They can aggregate fibers from top-of-rack switches or from individual servers. These exchanges tag each incoming fiber and add them to the Alpine Orchestration System (AOS), which keeps track of the connections just like the interconnections in a fabric.

Once AOS knows about all the connections in the system, you can use it to start building pathways between east-west traffic flows. You can ensure that traffic between a web server and backend database has dedicated connectivity. You can add additional resources between systems that are currently engaged in heavy processing. You can also dedicated traffic lanes for backup jobs. You can do quite a bit from the AOS console.

Now you have a layer 1 switching fabric without any additional pieces in the middle. The exchanges function almost like a passthrough device. The brains of the system exist in AOS. Remember when Ivan Pepelnjak (@IOSHints) spent all his time pulling QFabric apart to find out what made it tick? The Fiber Mountain solution doesn’t use BGP or MPLS or any other magic protocol sauce. It runs at layer 1. The light paths are programmed by AOS and the packets are swtiched across the dense fiber connections. It’s almost elegant in the simplicity.

Future Illumination

The Fiber Mountain solution has some great promise. Today, most of the operations of the system require manual intervention. You must build out the light paths between servers based on educated guesses. You must manually add additional light paths when extra bandwidth is needed.

Where they can really improve their offering in the future is to add intelligence to AOS to automatically make those decisions based on thresholds and inputs that are predefined. If the system can detect bigger “elephant” traffic flows and automatically provision more bandwidth or isolate these high volume packet generators it will go a long way toward making things much easier on network admins. It would also be great to provide a way to interface that “top talker” data into other systems to alert network admins when traffic flows get high and need additional resources.


Tom’s Take

I like the Fiber Mountain solution. They’ve built a layer 1 fabric that performs similarly to the ones from Juniper and Brocade. They are taking full advantage of the resources provided by the MPO fiber connectors. By adding a new network card to a server, you can test this system without impacting other traffic flows. Fiber Mountain even told me that they are looking at trial installations for customers to bring their technology in at lower costs as a project to show the value to decision makers.

Fiber Moutain has a great start on building a low latency fiber fabric with intelligence. I’ll be keeping a close eye on where the technolgy goes in the future to see how it integrates into the entire network and brings SDN features we all need in our networks.

 

Could IPv6 Drown My Wireless Network?

IPv6WiFi

By now, the transition to adopt IPv6 networks is in full swing. Registrars are running out of prefixes and new users overseas are getting v6-only allocations for new circuits. Mobile providers are going v6-only and transition mechanisms are in place to ease the migration. You can hear about some of these topics in this recent roundtable recorded at Interop last week:

One of the converstaions that I had with Ed Horley (@EHorley) during Interop opened my eyes to another problem that we will soon be facing with IPv6 and legacy technology. Only this time, it’s not because of a numbering scheme. It’s because of old hardware.

Rate Limited

Technology always marches on. Things that seemed magical to us just five years ago are now antiquated and slow. That’s the problem with the original 802.11 specification. It supported wireless data rates at a paltry 1 Mbps and 2 Mbps. When 802.11b was released, it raised the rates to 5.5 Mbps and 11 Mbps. Those faster data rates, combined with a larger coverage area, helped 802.11b become commercially successful.

Now, we have 802.11n with data rates in the hundreds of Mbps. We also have 802.11ac right around the corner with rates approaching 1 Gbps. It’s a very fast wireless world. But thanks to the need to be backwards compatible with existing technology, even those fast new 802.11n access points still support the old 1 & 2 Mbps data rates of 802.11. This is great if you happen to have a wireless device from the turn of the millenium. It’s not so great if you are a wireless engineer supporting such an installation.

Wireless LAN professionals have been talking for the past couple of years about how important it is to disable the 1, 2, and 5.5 Mbps data rates in your wireless networks. Modern equipment will only utilize those data rates when far away from the access point and modern design methodology ensures you won’t be far from an access point. Removing support for those devices forces the hardware to connect at a higher data rate and preserve the overall air quality. Even one 802.11b device connecting to your wireless network can cause the whole network to be dragged down to slow data rates. How important is it to disable these settings? Meraki’s dashboard allows you to do it with one click:

MerakiDataRates

Flood Detected

How does this all apply to IPv6? Well, it turns out that that multicast has an interesting behavior on wireless networks. It seeks out the lowest data rate to send traffic. This ensures that all recievers get the packet. I asked Matthew Gast (@MatthewSGast) of Aerohive about this recently. He said that it’s up to the controller manufacturer to decide how multicast is handled. When I gave him an inquisitive look, he admitted that many vendors leave it up to the lowest common denominator, which is usually the 1 Mbps or 2 Mbps data rate.

This isn’t generally a problem. IPv4 multicast tends to be sporadic and short-lived at best. Most controllers have mechanisms in place for dealing with this, either by converting those multicasts to unicasts or by turning off mulitcast completely. A bit of extra traffic on the low data rates isn’t noticeable.

IPv6 has a much higher usage of multicast, however. Router Advertisements (RAs) and Multicast Listener Discovery (MLD) are crictical to the operation of IPv6. So critical, in fact, that turning off Global Multicast on a Cisco wireless controller doesn’t disable RAs and MLD from happening. You must have multicast running for IPv6.

What happens when all that multicast traffic from IPv6 hits a controller with the lower data rates enable? Gridlock. Without vendor intervention the MLD and RA packets will hop down to the lowest data rate and start flooding the network. Listeners will respond on the same low data rate and drag the network down to an almost-unusable speed. You can’t turn off the multicast to fix it either.

The solution is to prevent this all in the first place. You need to turn off the 802.11b low data rates on your controller. 1 Mbps, 2 Mbps, and 5.5 Mbps should all be disabled, both as a way to prevent older, slower clients from connecting to your wireless network and to keep newer clients running IPv6 from swamping it with multicast traffic.

There may still be some older clients out there that absolutely require 802.11b data rates, like medical equipment, but the best way to deal with these problematic devices is isolation. These devices likely won’t be running IPv6 any time in the future. Isolating them onto a separate SSID running the 802.11b data rates is the best way to ensure they don’t impact your other traffic. Make sure you read up on how to safely disable data rates and do it during a testing window to ensure you don’t break everything in the world. But you’ll find your network much more healthy when you do.


Tom’s Take

Legacy technology support is critical for continued operation. We can’t just drop something because we don’t want to deal with it any more. Anyone who has ever called a technical support line feels that pain. However, when the new technology doesn’t feasably support working with older tech, it’s time to pull the plug. Whether it be 802.11b data rates or something software related, like dropping PowerPC app support in OS X, we have to keep marching forward to make new devices run at peak performance.

IPv6 has already exposed limitations of older technologies like DHCP and NAT. Wireless thankfully has a much easier way to support transitions. If you’re still running 802.11b data rates, turn them off. You’ll find your IPv6 transition will be much less painful if you do. And you can spend more time working with tech and less time trying to tread water.