SD-WAN and Technical Debt

Back during Networking Field Day 22, I was having a fun conversation with Phil Gervasi (@Network_Phil) and Carl Fugate (@CarlFugate) about SD-WAN and innovation. I mentioned that it was fascinating to see how SD-WAN companies kept innovating but that bigger, more established companies that had bought into SD-WAN seemed to be having issues catching up. As our conversation continued I realized that technical debt plays a huge role in startup culture in all factors, not just with SD-WAN. However, SD-WAN is a great example of technical debt to talk about here.

Any Color You Want In Black

Big companies have investments in supply chains. They have products that are designed in a certain way because it’s the least expensive way to develop the project or it involves using technology developed by the company that gives them a competitive advantage. Think about something like the Cisco Nexus 9000-series switches that launched with Cisco ACI. Every one of them came with the Insieme ASIC that was built to accelerate the policy component of ACI. Whether or not you wanted to use ACI or Insieme in your deployment, you were getting the ASIC in the switch.

Policies like this lead to unintentional constraints in development. Think back five years to Cisco’s IWAN solution. It was very much the precursor to SD-WAN. It was a collection of technologies like Performance Routing (PfR), Application Visibility Control (AVC), Policy Based Routing (PBR), and Network Based Application Recognition (NBAR). If that alphabet soup of acronyms makes you break in hives, you’re not alone. Cisco IWAN was a platform very much market by potential and complexity.

Let’s step back and ask ourselves an important question: “Why?” Why was IWAN so complicated? Why was IWAN hard to deploy? Why did IWAN fail to capture a lot of market share and ride the wave that eventually became SD-WAN? Looking back, a lot of the choices that were made that eventually doomed IWAN can come down to existing technical debt. Cisco is a company that makes design decisions based on what they’ve been doing for a while.

I’m sure that the design criteria for IWAN came down to two points:

  1. It needs to run on IOS.
  2. It needs to be an ISR router.

That doesn’t sound like much. But imagine the constraints you run into with just those two limitations. You have a hardware platform that may not be suited for the kind of work you want to do. Maybe you want to take advantage of x86 chipset acceleration. Too bad. You have to run what’s in the ISR. Which means it could be underpowered. Or incapable of doing things like crypto acceleration for VPNs, which is important for building a mesh of encrypted tunnels. Or maybe you need some flexibility to build a better detection platform for applications. Except you have to use IOS. Which uses NBAR. And anything you write to extend NBAR has to run on their platforms going forward. Which means you need to account for every possible permutation of hardware that IOS runs on. Which is problematic at best.

See how technical debt can creep in from the most simplistic of sources? All we wanted to do was build a platform to connect WANs together easily. Now we’re mired in a years-old hardware choice and an aging software platform that can’t help us do what needs to be done. Is it any wonder why IWAN didn’t succeed in the original form? Or why so many people involved with the first generation of SD-WAN startups were involved with IWAN, even if just tangentially?

Debt-Free Development

Now, let’s look at a startup like CloudGenix, who was a presenter at Networking Field Day 22 and was recently acquired by Palo Alto Networks. They started off on a different path when they founded the startup. They knew what they wanted to accomplish. They had a vision for what would later be called SD-WAN. But instead of shoehorning it into an existing platform, they had the freedom to build what they wanted.

No need to keep the ISR platform? Great. That means you can build on x86 hardware to make your software more universally deployable on a variety of boxes. Speaking of boxes, using commercial off-the-shelf (COTS) equipment means you can buy some very small devices to run the software. You don’t need a system designed to use ATM modules or T1 connections. If all you little system is ever going to use is Ethernet there’s no reason to include expansion at all. Maybe USB for something like a 4G/LTE modem. But those USB ports are baked into the board already.

A little side note here that came from Olivier Huynh Van of Gluware. You know the USB capabilities on a Cisco ISR? Yeah, the ISR chipset didn’t support USB natively. And it’s almost impossible to find USB that isn’t baked into an x86 board today. So Cisco had to add it to the ISR in a way that wasn’t 100% spec-supported. It’s essentially emulated in the OS. Which is why not every USB drive works in an ISR. Take that for what’s it’s worth.

Back to CloudGenix. Okay, so you have a platform you can build on. And you can build software that can run on any x86 device with Ethernet ports and USB devices. That means your software doesn’t need to do complicated things. It also means there are a lot of methods already out there for programming network operating systems for x86 hardware, such as Intel’s Data Plane Development Kit (DPDK). However CloudGenix chose to build their OS, they didn’t need to build everything completely from scratch. Even if they chose to do it there are still a ton of resources out there to help them get started. Which means you don’t have to restart your development every time you need to add a feature.

Also, the focus on building the functions you want into an OS you can bend to your needs means you don’t need to rely on other teams to build pieces of it. You can build your own GUI. You can make it look however you want. You can also make it operate in a manner that is easiest for your customer base. You don’t need to include every knob or button or bell and whistle. You can expose or hide functions as you wish. Don’t want customers to have tons of control over VPN creation or certificate authentication? You don’t need to worry about the GUI team exposing it without your permission. Simple and easy.

One other benefit of developing on platforms without technical debt? It’s easy to port your software from physical to virtual. CloudGenix was already successful in porting their software to run from physical hardware to the cloud thanks to CloudBlades. Could you imagine trying to get the original Cisco IWAN running in a cloud package for AWS or Azure? If those hives aren’t going crazy right now I’m sure you must have nerves or steel.


Tom’s Take

Technical debt is no joke. Every decision you make has consequences. And they may not be apparent for this generation of products. People you may never meet may have to live with your decisions as they try to build their vision. Sometimes you can work with those constraints. But more often than not brilliant people are going to jump ship and do it on their own. Not everyone is going to succeed. But for those that have the vision and drive and turn out something that works the rewards are legion. And that’s more than enough to pay off any debts, technical or not.

Magical Mechanics

If you’re a fan of this blog, you’ve probably read my last post about the new SD-WAN magic quadrant that’s been making the rounds and generating discussion. Some people are smiling that this report places Cisco in an area other than leadership in the SD-WAN space. Others are decrying the report as being unfair and contradictory. I wanted to take another look at it given some new information and some additional thoughts on the results.

Fair and Square

The first thing I wanted to do is make sure that I was completely transparent with the way the Gartner Magic Quadrant (MQ) works. I have a very good idea thanks to a conversation with Andrew Lerner (@Fast_Lerner), who is the Research VP of Networking at Gartner. Andrew was nice enough to clarify my understanding of the MQ and accompanying documentation. I’ll quote him here to make sure I don’t get anything wrong:

In an MQ, we assess the overall vendors’ behavior and offering in the market. Product, service/support sales, marketing, innovation, etc. if a vendor has multiple products in a market and sells them regularly to the enterprise, they are part of the MQ assessment. Viable products are not “excluded”.

As you can see from Andrew’s explanation, the MQ takes into account all the aspects of a company. It’s not just a product. It’s the sales, marketing, and other aspects of the company that give the overall score for a company. So how does Gartner figure out how products and services? That’s where their Critical Capabilities documents come into play. They are focused exclusively on products and services. They don’t take marketing or sales or anything else into account.

According to Andrew, when Gartner did their Critical Capabilities document on Cisco, they looked at Meraki MX and IOS-XE only. Viptela vEdge was not examined. So, the CC documents give us the Gartner overview of technology behind the MQ analysis. While the CC documents are focused solely on the Meraki MX and IOS-XD SD-WAN technology, they are components of the overall analysis in the MQ that was published.

What does that all mean in the long run?

Axis and Allies

In order to break this down a bit further, let’s ignore the actual quadrants in the MQ for the moment. Instead, let’s think about this picture as a graph. One axis of the graph is “Ability to Execute,” or in other words can the company do what they say they’re going to do? The other axis is “Completeness of Vision.” This is a question of how the company has rounded out their understanding of the market forces and direction. I want to use this sample MQ with some labels thrown in to help my readers understand how each axis can affect the ranking a company gets:

So, let’s look at where Cisco was situated on those graphs. They are not in the upper right part of the graph, which is the “good” part according to what most people will tell you when they glance at it. Cisco was ranked almost directly in the middle of the graph. Why would that be?

Let’s look at the Execution axis. Why would Cisco have some issues with execution on SD-WAN? Well, the biggest one is probably the shift to IOS-XE and the issues with code quality. Almost everyone involved deploying IOS-XE has told me that Cisco had significant issues in the early releases. Daniel Dib (@DanielDibSwe) had a great conversation with me about the breakdown between the router code and the controller code a week ago. Here’s the first tweet in the chain:

So, there were issues that have been addressed. But is the code completely stable right now? I can’t say, since I haven’t deployed it. But ask around and see what the common wisdom is. I would be genuinely interested to hear how much better things have gotten in the last six months. But, those code quality issues from months ago are going to be a concern in the report. And you can’t guarantee that every box is going to be running on the latest code. Having issues with stable code is going to impact your ability to execute on your vision.

Now, let’s look at the Completeness of Vision axis. If you reference the above picture you’ll see that the Challengers square represents companies that don’t yet understand the market direction. Given that this was the location that Cisco was placed in (barely), let’s examine why that might be. Let’s start by asking “which Cisco product is best for my SD-WAN solution?” Do you know for sure? Which one are you going to be offered?

In my last post, I said that Cisco had been deemphasizing vEdge deployments in favor of IOS-XE. But I completely forgot about Meraki as an additional offering for SMBs and smaller deployments. Where does Meraki make the most sense? And how big does your network need to be before it outgrows a Meraki deployment? Are you chasing a feature? Or maybe you need a service that isn’t offered natively on the Meraki platform? All of these questions need to be answered when you look at what you’re going to do with SD-WAN.

The other companies that Cisco will tell you are their biggest competitors are probably VMware and Silver Peak. How many SD-WAN platforms do they sell? How about companies ranked closer to Cisco in the MQ like Citrix, CloudGenix, or Versa Networks? How many SD-WAN solutions do they offer? How about HPE, Juniper and Aryaka?

In almost every case, the answer is “one”. Each of these companies have settled on a single solution for SD-WAN or SD-Branch. They don’t split those deployments across different product lines. They may have different sized boxes for things, but they all run the same common software. Can you integrate an IOS-XE SD-WAN appliance into a Meraki deployment? Can you take a Meraki MX and make it work with vEdge?

You may be starting to see that the completeness of Cisco’s vision isn’t lacking in SD-WAN but instead how they’re going to accomplish it. Rather than having one solution that can be scaled to fit all needs, Cisco is choosing to offer two different solutions for SMBs and enterprises. And if you count vEdge as a separate product from IOS-XE, as Cisco has suggested in some of their internal reports, then you have three products! I’m not saying that Cisco doesn’t have a vision. But it really looks like that vision is hazier than it should be.

If Cisco had a unified vision with stable code that was integrated up and down the stack I have no doubts they would have been rated higher. When Cisco was deploying Viptela vEdge as the sole SD-WAN offering they had it was much easier to figure out how everything was going to integrate together. But, just like all transitions, the devil is in the details here as Cisco tries to move to IOS-XE. Code quality is going to be a potential source of problems no matter what. But if you are staking your reputation on moving everyone to a single code base from a different more stable one you had better get it right quickly. Otherwise you’re going to get it counted against you.


Tom’s Take

I really appreciate Andrew Lerner for reaching out regarding my analysis of the MQ. I will admit I wasn’t exactly spot on with the differences between the MQ and the Critical Capabilities documents. But the results are close. The CC analyzes Cisco’s big platforms and tells people how they work. The MQ takes everything as a whole and gives Cisco a ranking of where they stand in the market. Sales numbers aside, do you think Cisco should be a leader in the SD-WAN space? Do you feel that where they are today with IOS-XE and Meraki MX is a complete vision? If you do then you likely won’t care about the MQ either way. But for those that have questions about execution and vision, maybe it’s time to figure out what might have caused Cisco to be right in the middle this time around.

SD-WAN Squares and Perplexing Planes

The latest arcane polygon is out in the SD-WAN space. Normally, my fortune telling skills don’t involve geometry. I like to talk to real people about their concerns and their successes. Yes, I know that the gardening people do that too. It’s just that no one really bothers to read their reports and instead makes all their decisions based on boring wall art.

Speaking of which, I’m going to summarize that particular piece of art here. Note this isn’t the same for copyright reasons but close enough for you to get the point:

4D8DB810-3618-44EA-8AA2-99EB7EAA3E45

So, if you can’t tell by the colors here, the big news is that Cisco has slipped out of the top Good part of the polygon and is now in the bottom Bad part (denoted by the red) and is in danger of going out of business and being the laughing stock of the networking community. Well, no, not so much that last part. But their implementation has slipped into the lower part of the quadrant where first-stage startups and cash-strapped companies live and wish they could build something.

Cisco released a report rebutting those claims and it talks about how Viptela is a huge part of their deployments and that they have mindshare and marketshare. But after talking with some colleagues in the networking industry and looking a bit deeper into the issue, I think Cisco has the same problem that Boeing had this year. Their assurances are inconsistent with reality.

A WANderful World

When you deploy Cisco SD-WAN, what exactly are you deploying? In the past, the answer was IWAN. IWAN was very much an initial attempt to create SD-WAN. It wasn’t perfect and it was soon surpassed by other technologies, many of which were built by IWAN team members that left to found their own startups. But Cisco quickly realized they needed to jump in the fray and get some help. That’s when the bought Viptela.

Here’s where things start getting murky. The integration of Viptela SD-WAN has been problematic. The initial sales model after the acquisition was to continue to sell Viptela vEdge to the customer. The plan was then to integrate the software on a Cisco ISR, then to integrate the Viptela software into IOS-XE. They’ve been selling vEdge boxes for a long while and are supporting them still. But the transition plan to IOS-XE is in full effect. The handwriting on the wall is that soon Cisco will only offer IOS-XE SD-WAN for sale, not vEdge.

Flash forward to 2019. A report is released about the impact and forward outlook for SD-WAN. It has a big analyst name attached to it. In that report, they leave out the references to vEdge and instead grade Cisco on their IOS-XE offering which isn’t as mature as vEdge. Or as deployed as vEdge. Or, as stated by even Cisco, as stable as vEdge, at least at first. That means that Cisco is getting graded on their newest offering. Which, for Cisco, means they can’t talk about how widely deployed and stable vEdge is. Why would this particular analyst firm do that?

Same is Same

The common wisdom is that Gartner graded Cisco on the curve that the sales message is that IOS-XE is the way and the future of SD-WAN at Cisco. Why talk about what came before when the new hot thing is going to be IOS-XE? Don’t buy this old vEdge when this new ISR is the best thing ever.

Don’t believe me? Go call your Cisco account team and try to buy a vEdge. I bet you get “upsold” to an ISR. The path forward is to let vEdge fade away. So it only makes sense that you should be grading Cisco on what their plans are, not on what they’ve already sold. I’ll admit that I don’t put together analyst reports with graphics very often, so maybe my thinking is flawed. But if I call your company and they won’t sell me the product that you want me to grade you on, I’m going to guess I shouldn’t grade you on it.

So Cisco is mad that Gartner isn’t grading them on their old solution and is instead looking at their new solution in a vacuum. And since they can’t draw on the sales numbers or the stability of their existing solution that you aren’t going to be able to buy without a fight, they slipped down in the square to a place that doesn’t show them as the 800-lb gorilla. Where have a heard something like this before?

Plane to See

The situation that immediately sprung to mind was the issue with Boeing’s 737-MAX airliner. In a nutshell, Boeing introduced a new airliner with a different engine and configuration that changed its flight profile. Rather than try to get the airframe recertified, which could take months or years, they claimed it was the same as the old one and just updated training manuals. They also didn’t disclose there was a new software program that tried to override the new flight characteristics and that particular “flaw” caused the tragic crashes of two flights full of people.

I’m not trying to compare analyst reports to tragedies in any way. But I do find it curious that companies want their new stuff to be treated just like their old stuff with regards to approvals and regulation and analysis. They know that the new product is going to have issues and concerns but they don’t want you to count those because remember how awesome our old thing was?

Likewise, they don’t want you to count the old problems against them. Like the Pinto or the Edsel from Ford, they don’t want you to think their old issues should matter. Just look at how far we’ve come! But that’s not how these things work. We can’t take the old product into account when trying to figure out how the new one works. We can’t assume the new solution is the same as the old one without testing, no matter how much people would like us to do that. It’s like claiming the Space Shuttle was as good as the Saturn V rocket because it went into space and came from NASA.

If your platform has bugs in the first version, those count against you too. You have to work it all out and realize people are going to remember that instability when they grade your product going forward. Remember that common wisdom says not to install an operating system update until the first service patch. You have to consider that reputation every time you release a patch or an update.


Tom’s Take

The SD-WAN MQ tells me that Cisco isn’t getting any more favors from the analysts. The marketing message not lining up with the install base is the heart of a transition away from hardware and software that Cisco doesn’t own. The hope that they could just smile and shrug their shoulders and hope that no one caught on has been dashed. Instead, Cisco now has to realize their going to have to earn that spot back through good code releases and consistent support and licensing. No one is going to give them any slack with SD-WAN like they have with switches and unified communications. If Cisco thinks that they’re just going to be able to bluff their way through this product transition, that idea just won’t fly.

The Development of DevNet’s Future

You’re probably familiar with Cisco DevNet. If not, DevNet is the place Cisco has embraced outreach to the developer community building for software-defined networking (SDN). Though initially cautious in getting into the software developer community, Cisco has embraced their new role and really opened up to help networking professionals embrace the new software normal in networking. But where is DevNet going to go from here?

Humble Beginnings

DevNet wasn’t always the darling of Cisco’s offerings. I can remember sitting in on some of the first discussions around Cisco OnePK and thinking to myself, “This is never going to work.”

My hesitation with Cisco’s first attempts to focus on software platforms came from two places. The first was what I saw as Cisco trying to figure out how to extend the platforms to include some programmability. It was more about saying they could do software and less about making that software easy to use or program against. The second place was actually the lack of a place to store all of this software knowledge. Programmers and developers are fickle lot and you have to have a repository where they can get access to the pieces they needed.

DevNet was that place that Cisco should have built from the start. It was a way to get people excited and involved in the process. But it wasn’t for everyone at first. If you don’t speak developer you’re going to feel lost. Even if you are completely fluent in networking and you know what you want to accomplish, just not how to get there. DevNet started off as the place to let the curious learn how to combine networking and programming.

The Ascent

DevNet really came into their own about 3 years ago. I use that timeline because that’s when I first heard that people were wanting to spend more time at Cisco Live in the DevNet Zone learning programming and other techniques and less time in traditional sessions. Considering the long history of Cisco Live that’s an impressive feat.

More importantly, DevNet changed the conversation for professionals. Instead of just being for the curious, DevNet became a place where anyone could go and find the information they needed. It became a resource. Not just a playground. Instead of poking around and playing with things it became a place to go and figure things out. Or a place to learn more about a new technology that you wanted to implement, like automation. If the regular sessions at Cisco Live were what you had to learn, DevNet is where you wanted to go and learn.

Susie Wee (@SusieWee) deserves all the credit in the world here. She has seen what the developer community needs to thrive inside of Cisco and she’s delivered it. She’s the kind of ambassador that can go between the various Cisco business units (BUs) and foster the kind of attitudes that people need to have to succeed. It’s no longer about turf wars or fiefdoms. Instead, it’s about leveraging a common platform for developers and networkers alike to find a common ground to build from. But even that’s not enough to complete the vision.

Narrow of Purpose, Wide of Vision

During Cisco Live 2019, I talked quite a bit with Susie and her team. And one of things that struck me from our conversations was not how DevNet was an open and amazing place. Or how they were adding sessions as fast as they could find instructors. It was that so many people weren’t taking advantage of it. That’s when I realized that DevNet needs to shift their focus. Instead of just providing a place for networking people to learn, they’re going to have to go on the offensive.

DevNet needs to enhance and increase their outreach programs. Being a static resource is fine when your audience is eager to learn and looking for answers. But those people have already flocked to the DevNet banner. For things to grow, DevNet needs to pull the laggards along. The people who think automation is just a fad. Or that SDN is in the Trough of Disillusionment from a pretty Gartner graphic. DevNet has momentum, and soon will have the certification program needed to help networking people show off their transformation to developers.


Tom’s Take

For DevNet to really succeed, they need to be grabbing people by the collar and dragging them to the new reality of networking. It’s not enough to give people a place to do research on nice-to-have projects. You’re going to have get the people engaged and motivated. That means committing resources to entry-level outreach. Maybe even building a DevNet Academy similar to the Cisco Academy. But it has to happen. Because the people that aren’t already at DevNet aren’t going to get there on their own. They need a push (or a pull) to find out what they don’t know that they don’t know.

Cisco Live 2019 – Rededicating Community

The 2019 Cisco Live Sign Photo

Another Cisco Live is in the books for me. I was a bit shocked to realize this was my 14th event in a row. I’ve been going to Cisco Live half of the time it’s been around! This year was back in San Diego, which has good and bad points. I’d like to discuss a few of them there and get the thoughts of the community.

Good: The Social Media Hub Has Been Freed! – After last year’s issues with the Social Media Hub being locked behind the World of Solutions, someone at Cisco woke up and realized that social people don’t keep the same hours as the show floor people. So, the Hub was located in a breezeway between the Sails Pavilion and the rest of the convention center. And it was great. People congregated. Couches were used. Discussions were had. And the community was able to come together again. Not during the hours when it was convenient. But a long time. This picture of the big meeting on Thursday just solidifies in my mind why the Social Media Hub has to be in a common area:

You don’t get this kind of interaction anywhere else!

Good: Community Leaders Step Forward – Not gonna lie. I feel disconnected sometimes. My job at Tech Field Day takes me away from the action. I spend more time in special sessions than I do in the social media hub. For any other place that could spell disaster. But not for Cisco Live. When the community needs a leader, someone steps forward to fill the role. This year, I was happy to see my good friend Denise Fishburne filling that role. The session above was filled with people paying rapt attention to Fish’s stories and her bringing people into the community. She’s a master at this kind of interaction. I was even proud to sit on the edge and watch her work her craft.

Fish is the d’Artagnan of the group. She may be part of the Musketeers of Social Media but Fish is undoubtedly the leader. A community should hope to have a leader that is as passionate and involved as she is, especially given her prominent role in Cisco. I feel like she can be the director of what the people in the Social Media Hub need. And I’m happy to call her my friend.

Bad: Passes Still Suck – You don’t have to do the math to figure out that $700 is bigger than $200. And that $600/night is worse than $200/night. And yet, for some reason we find ourselves in San Diego, where the Gaslamp hotels are beyond insane, wondering what exactly we’re getting with our $700 event pass. Sessions? Nope. Lunch? Well, sort of. Access to the show floor? Only when it’s open for the random times during the week. Compelling content? That’s the most subjective piece of all. And yet Cisco is still trying to tell us that the idea of a $200 social-only pass doesn’t make sense.

Fine. I get it. Cisco wants to keep the budgets for Cisco Live high. They got the Foo Fighters after all, right? They also don’t have to worry about policing the snacks and food everywhere. Or at least not ordering the lowest line items on the menu. Which means less fussing about piddly things inside the convention center. And for the next two years it’s going to work out just great in Las Vegas. Because Vegas is affordable with the right setup. People are already booking rooms at the surrounding hotels. You can stay at the Luxor or the Excalibur for nothing. But if the pass situation is still $700 (or more) in a couple of years you’re going to see a lot of people dropping out. Because….

Bad: WTF?!? San Francisco?!? – I’ve covered this before. My distaste for Moscone is documented. I thought we were going to avoid it this time around. And yet, I found out we’re going back to SF in 2022.

WHY?!?!?!?

Moscone isn’t any bigger. We didn’t magically find seating for 10,000 extra people. More importantly, the hotel situation in San Fran is worse than ever before. You seriously can’t find a good room this year for VMworld. People are paying upwards of $500/night for a non-air conditioned shoe box! And why would you do this to yourself Cisco?

Sure, it’s cheap. Your employees don’t need hotel rooms. You can truck everything up. But your costs savings are being passed along to the customer. Because you would rather them pay through the nose instead of footing the bill yourself. And Moscone still won’t hold the whole conference. We’ll be spilled over into 8 different hotels and walking from who knows where to get to the slightly nicer shack of a convention center.

I’m not saying that Cisco Live needs to be in Vegas every year. But it’s time for Cisco to start understanding that their conference needs a real convention center. And Moscone ain’t it.

Better: Going Back to Orlando – As you can see above, I’ve edited this post to include new information about Cisco Live 2022. I have been informed by multiple people, including internal Cisco folks, that Live 2022 is going to Orlando and not SF. My original discussion about Cisco Live in SF came from other sources with no hard confirmation. I believe now it was floated as a trial balloon to see how the community would respond. Which means all my statements above still stand regarding SF. Now it just means that there’s a different date attached to it.

Orlando is a better town for conventions than SF. It’s on-par with San Diego with the benefit that hotels are way cheaper for people because of the large amount of tourism. I think it’s time that Cisco did some serious soul searching to find a new venue that isn’t in California or Florida for Cisco Live. Because if all we’re going to do is bounce back and forth between San Diego and Orlando and Vegas over and over again, maybe it’s time to just move Cisco Live to Vegas and be done with the moving.


Tom’s Take

Cisco Live is something important to me. It has been for years, especially with the community that’s been created. There’s nothing like it anywhere else. Sure, there have been some questionable decisions and changes here and there. But the community survives because it rededicates itself every year to being about the people. I wasn’t kidding when I tweeted this:

Because the real heart of the community is each and every one of the people that get on a plane and make the choice time and again to be a part of something special. That kind of dedication makes us all better in every possible way.

Cisco’s Catalyst for Change

You’ve probably heard by now of the big launch of Cisco’s new 802.11ax (neé Wi-Fi 6) portfolio of devices. Cisco did a special roundtable with a group of influencers from the community called Just The Tech. Here’s a video from that event covering the APs that were released, the 9120:

Fred always does a great job of explaining the technical bits behind the APs. But one thing that caught my eye here is the name of the AP – Catalyst. Cisco has been using Aironet for their AP line since they purchased Aironet Wireless Communications back in 1999. The name was practically synonymous with wireless technologies for many people in the industry that worked exclusively with Cisco technologies.

So, is the name change something we should be concerned about?

A Rose Is a Rose Is An AP

Cisco moving toward a unified naming convention for their edge solutions makes a lot of sense. Ten years ago, wireless was still primarily 802.11g-based with 802.11n still a few months away from being proposed and ratified. Connectivity hadn’t quite yet reached the ubiquitous levels of wireless that we see today. The iPhone was only about to be on its third revision.

Cisco Catalyst devices were still the primary method of getting users connected to the network. Even laptop users hunted for Ethernet ports everywhere instead of just connecting to wireless. Ethernet was more reliable and faster than 54Mbps (at best) and fighting contention with all the other devices around. Catalyst stood for reliability.

In the time since, wireless has become the new edge device connectivity. No longer do we hunt for Ethernet ports unless we have a specific need for one. Laptops don’t come with dedicated wired networking options any longer. In 2019, wireless is king. And Aironet is the wireless name that Cisco has built. So why the change?

In short, because edge connectivity isn’t wired versus wireless any longer. Instead, it’s unified. Whether it was because of the idiotic decisions made by Gartner to required wired switching for their wireless Magic Quadrant (TM) or because people stopped thinking about Ethernet except to power wireless access points, the fact is that the edge no longer has wires. For Cisco, this means that Catalyst switches aren’t the edge any longer. So the name doesn’t have the same power as it once did.

However, the Aironet name has also lost its luster. Why? Because Aironet is a remnant of Cisco’s pre-controller AP past. The line of APs that most people are likely using in their office right now aren’t from the Aironet heritage. Instead, they are based on technology acquired by Cisco from Airespace that Cisco bought in 2005 to add controller-based technology to their portfolio. And, aside from references to Airespace in the code of the Wireless LAN Controllers (WLC), the line never really had a brand like Catalyst or Aironet.

Today, Cisco has started the move away from using Airespace technology in their controllers. As this video from 2018 shows, Cisco has begun to migrate their controller OS to a more modern platform instead of relying on modifying the old Airespace code again and again. This means that development going forward should be more rapid and less resultant on the whims of keeping everything running properly on a codebase over a decade old.

Branding New

So, that explains the reasons why Cisco might want to refresh everything. But why the naming of the APs? Why not just rely on Aironet and keep that branding going forward?

Well, because they want to make end users believe that the network is the same no matter if it’s wired or wireless. They want buyers to believe that Catalyst stands for edge connectivity, no matter where that edge might be. And, unless they really screw up and start making us think these new APs are switches they’ll be able to pull off this branding exercise fairly well.

That’s because users have stopped caring about the wired versus wireless debate. Instead, they only care about speed and reliability. 802.11ax will help on both fronts, and Cisco wants to capitalize on that by making these new APs feel different. And the best way to do that is by rebranding them.

Wireless professionals don’t care about the name. Most of the time they just refer to the model number anyway. And while Cisco’s model numbering strategies seem to be getting a bit crowded in the 9000-level of things, this makes a lot of sense to distance themselves from their past. The old 802.11ac APs are still very viable and will likely be useful all they way until the end of their life. But when the time comes to pull them out, you’ll be retiring Aironet and Airespace along with them. Even if you didn’t realize those were the branding names of those APs.


Tom’s Take

Branding matters. Or it doesn’t. Either you love the name of the thing you’ve been using or you couldn’t care less. Whether it’s an iPhone or a car or an access point, everything has a name and a number attached to it. Cisco has decided, for better or worse, to unify the edge under the Catalyst name. Maybe it will stick and reduce confusion with customers. Maybe it will be hated enough that they’ll bring back the Aironet name in a couple of cycles to “get back to basics” as it were. But for now, the catalyst for change at Cisco leads to a unified edge solution.

What Makes a Security Company?

When you think of a “security” company, what comes to mind? Is it a software house making leaps in technology to save us from DDoS attacks or malicious actors? Maybe it’s a company that makes firewalls or intrusion detection systems that stand guard to keep the bad people out of places they aren’t supposed to be. Or maybe it’s something else entirely.

Tradition Since Twenty Minutes Ago

What comes to mind when you think of a traditional security company? What kinds of technology do they make? Maybe it’s a firewall. Maybe it’s an anti-virus program. Or maybe it’s something else that you’ve never thought of.

Is a lock company like Schlage a security company? Perhaps they aren’t a “traditional” IT security company but you can guarantee that you’ve seen their products protecting data centers and IDF closets. What about a Halon system manufacturer? They may not be a first thought for security, but you can believe that a fire in your data center is going cause security issues. Also, I remember that I learned more about Halon and wet/dry pipe fire sprinkler systems from my CISSP study than anywhere else.

The problem with classifying security companies as “traditional” or “non-traditional” is that it doesn’t reflect the ways that security can move and change over the course of time. Even for something as cut-and-dried as anti-virus, tradition doesn’t mean a lot. Symantec is a traditional AV vendor according to most people. But the product that used to be called Norton Antivirus and the product suite that now includes is are worlds apart in functionality. Even though Symantec is “traditional”, what they do isn’t. And when you look at companies that are doing more advanced threat protection mechanisms like deception-based security or using AI and ML to detect patterns, the lines blur considerably.

But, it doesn’t obviate the fact that Symantec is a security company. Likewise, a company can be a security company even if they security isn’t their main focus. Like the Schlage example above, you can have security aspects to your business model without being totally and completely focused on security. And there’s no bigger example of this than a company like Cisco.

A Bridge Not Far Enough?

Cisco is a networking company right? Or are they a server company now? Maybe they’re a wireless company? Or do they do cloud now? There are many aspects to their business models, but very few people think of them as a security company. Even though they have firewalls, identity management, mobile security, Malware protection, VPN products, Email and Web Security, DNS Protection, and even Threat Detection. Does that mean they aren’t really a security company?

It could be rightfully pointed out that Cisco isn’t a security company because many of these technologies they have were purchased over the years from other companies. But does that mean that their solutions aren’t useful or maintained? As I was a doing research for this point, a friend pointed out the story of Cisco MARS and how it was purchased and ultimately retired by Cisco. However, the Cisco acquisition of Protego that netted them MARS happened in 2004. The EOL announcement was in 2011, and the final end-of-support was in 2016. Twelve years is a pretty decent lifetime for any security product.

The other argument is that Cisco doesn’t have a solid security portfolio because they have trouble integrating their products together. A common criticism of large companies like Cisco or Dell EMC is that it is too difficult to integrate their products together. This is especially true in situations where the technologies were acquired over time, just like Cisco.

However, is the converse true? Are standalone products easier to integrate? Is is more simple to take solutions from six different companies and integrate them together in some fashion? I’d be willing to be that outside of robust API support, most people will find that integrating security products from different vendors is as difficult (if not more so) than integrating products from one vendor. Does Cisco have a perfect integration solution? No, they don’t. But why should they? Why should it be expected that companies that acquire solutions immediate burn cycles to make everything integrate seamlessly. Sure, that’s on the roadmap. But integrations with other products is on everyone’s road map.

The last argument that I heard in my research is that Cisco isn’t a security company because they don’t focus on it. They’re a networking (or wireless or server) company. Yet, when you look at the number of people that Cisco has working in a specific business unit on a product, it can often be higher headcount that some independent firms have working on their solutions. Does that mean that Cisco doesn’t know what they’re doing? Or does it mean that individual organizations can have multiple focuses? That’s a question for the customers to answer.


Tom’s Take

I take issue with a definition of “traditional” versus non-traditional. For the reason that Apple is a traditional computer company and so is Wang Computers. Guess which one is still making computers? And even in the case of Apple, you could argue that their main line-of-business is mobile devices now. But, does anyone dispute Apple’s ability to make a laptop? Would a company that does nothing but make laptops be a “better” computer company? The trap of labels like that is that it ignores a significant amount of investment in business at the expense of a quick and easy label. What makes a company a computer company or a security company isn’t how they label themselves. It’s what they do with the technology they have.

Cisco and the Two-Factor Two-Step

In case you missed the news, Cisco announced yesterday that they are buying Duo Security. This is a great move on Cisco’s part. They need to beef up their security portfolio to compete against not only Palo Alto Networks but also against all the up-and-coming startups that are trying to solve problems that are largely being ignored by large enterprise security vendors. But how does an authentication vendor help Cisco?

Who Are You?

The world relies on passwords to run. Banks, email, and even your mobile device has some kind of passcode. We memorize them, write them down, or sometimes just use a password manager (like 1Password) to keep them safe. But passwords can be guessed. Trivial passwords are especially vulnerable. And when you factor in things like rainbow tables, it gets even scarier.

The most secure systems require you to have some additional form of authentication. You may have heard this termed as Two Factor Authentication (2FA). 2FA makes sure that no one is just going to be able to guess your password. The most commonly accepted forms of multi-factor authentication are:

  • Something You Know – Password, PIN, etc
  • Something You Have – Credit Card, Auth token, etc
  • Something You Are – Biometrics

You need at least two of these in order to successfully log into a system. Not having an additional form means you’re locked out. And that also means that the individual components of the scheme are useless in isolation. Knowing someone’s password without having their security token means little. Stealing a token without having their fingerprint is worthless.

But, people are starting to get more and more sophisticated with their attacks. One of the most popular forms of 2FA is the SMS authentication. It combines What You Know, in this case you password for your account, with Something You Have, which is a phone capable of receiving an SMS text message. When you log in, the authentication system sends an SMS to the authorized number and you have to type in the short-lived code to get into the system.

Ask Reddit how that worked out for them recently. A hacker (or group) was able to intercept the 2FA SMS codes for certain accounts and use both factors to log in and gather account data. It’s actually not as trivial as one might think to intercept SMS codes. It’s much, much harder to crack the algorithm of something like a security token. You’d need access to the source code and months to download everything. Like exactly what happened in 2011 to RSA.

In order for 2FA to work effectively, it needs to be something like an app on your mobile device that can be updated and changed when necessary to validate new algorithms and expire old credentials. It needs to be modern. It needs to be something that people don’t think twice about. That’s what Duo Security is all about. And, from their customer base and the fact that Cisco payed about $2.3 billion for them, they must do it well.

Won’t Get Fooled Again

How does Duo help Cisco? Well, first and foremost I hope that Duo puts an end to telnet access to routers forever. Telnet is the lazy way we enable remote access to devices. SSH is ten times better and a thousand times more secure. But setting it up properly to authenticate with certificate authentication is a huge pain. People want it to work when they need it to work. And tying it to a specific machine or location isn’t the easiest or more convenient thing.

Duo can give Cisco the ability to introduce real 2FA login security to their devices. IOS could be modified to require Duo Security app login authentication. That means that only users authorized to log into that device would get the login codes. No more guessed remote passwords!

Think about integrating Duo with Cisco ISE. That could be a huge boon for systems that need additional security. You could have groups of system that need 2FA and others that don’t. You could easily manage those lists and move systems in and out as needed. Or, you could start a policy that all systems needs 2FA and phase in the requirements over time to make people understand how important it is and give them time to download the app and get it set up. The ISE possibilities are endless.

One caveat is that Duo is a program that works with a large number of third party programs right now. Including integrations with Juniper Networks. As you can imagine, that list might change once Cisco takes control of the company. Some organizations that use Duo will probably see a price increase and will continue to offer the service to their users. Others, possibly Juniper as an example, may be frozen out as Cisco tries to keep the best parts of the company for their own use. If Cisco is smart, they’ll keep Duo available for any third party that wants to use the platform or integrate. It’s the best solution out there for solving this problem and everyone deserves to have good security.


Tom’s Take

Cisco buying a security company is no shock. They need the horsepower to compete in a world where firewalls are impediments at best and hackers have long since figured out how to get around static defenses. They need to get involved in software too. Security isn’t fought in silicon any more. It’s all in code and beefing up the software side of the equation. Duo gives them a component to compete in the broader authentication market. And the acquisition strategy is straight out of the Chambers playbook.

A plea to Cisco: Don’t lock everyone out of the best parts of Duo because you want to bundle them with recurring Cisco software revenue. Let people integrate. Take a page from the Samsung playbook. Just because you compete with Apple doesn’t mean you can’t make chips for them. Keep your competitors close and make they use your software and you’ll make more money than freezing everyone out and claiming your software is the best and least used of the bunch.

Back In The Saddle Of A Horse Of A Different Color

I’ve been asked a few times in the past year if I missed being behind a CLI screen or I ever got a hankering to configure some networking gear. The answer is a guarded “yes”, but not for the reason that you think.

Type Casting

CCIEs are keyboard jockeys. Well, the R&S folks are for sure. Every exam has quirks, but the R&S folks have quirky QWERTY keyboard madness. We spend a lot of time not just learning commands but learning how to input them quickly without typos. So we spend a lot of time with keys and a lot less time with the mouse poking around in a GUI.

However, the trend in networking has been to move away from these kinds of input methods. Take the new Aruba 8400, for instance. The ArubaOS-CX platform that runs it seems to have been built to require the least amount of keyboard input possible. The whole system runs with an API backend and presents a GUI that is a series of API calls. There is a CLI, but anything that you can do there can easily be replicated elsewhere by some other function.

Why would a company do this? To eliminate wasted effort. Think to yourself how many times you’ve typed the same series of commands into a switch. VLAN configuration, vty configs, PortFast settings. The list goes on and on. Most of us even have some kind of notepad that we keep the skeleton configs in so we can paste them into a console port to get a switch up and running quickly. That’s what Puppet was designed to replace!

By using APIs and other input methods, Aruba and other companies are hoping that we can build tools that either accept the minimum input necessary to configure switches or that we can eliminate a large portion of the retyping necessary to build them in the first place. It’s not the first command you type into a switch that kills you. It’s the 45th time you paste the command in. It’s the 68th time you get bored typing the same set of arguments from a remote terminal and accidentally mess this one up that requires a physical presence on site to reset your mistake.

Typing is boring, error prone, and costs significant time for little gain. Building scripts, programs, and platforms that take care of all that messy input for us makes us more productive. But it also changes the way we look at systems.

Bird’s Eye Views

The other reason why my fondness for keyboard jockeying isn’t as great as it could be is because of the way that my perspective has shifted thanks to the new aspects of networking technology that I focus on. I tell people that I’m less of an engineer now and more of an architect. I see how the technologies fit together. I see why they need to complement each other. I may not be able to configure a virtual link without documentation or turn up a storage LUN like I used to, but I understand why flash SSDs are important and how APIs are going to change things.

This goes all they way back to my conversations at VMunderground years ago about shifting the focus of networking and where people will go. You remember? The “ditch digger” discussion?

 

This is more true now than ever before. There are always going to be people racking and stacking. Or doing basic types of configuration. These folks are usually trained with basic knowledge of their task with no vision outside of their job role. Networking apprentices or journeymen as the case may be. Maybe one out of ten or one out of twenty of them are going to want to move up to something bigger or better.

But for the people that read blogs like this regularly the shift has happened. We don’t think in single switches or routers. We don’t worry about a single access point in a closet. We think in terms of systems. We configure routing protocols across multiple systems. We don’t worry about a single port VLAN issue. Instead, we’re trying to configure layer 2 DCI extensions or bring racks and pods online at the same time. Our visibility matters more than our typing skills.

That’s why the next wave of devices like the Aruba 8400 and the Software Defined Access things coming from Cisco are more important than simple checkboxes on a feature sheet. They remove the visibility of protocols and products and instead give us platforms that need to be configured for maximum effect. The gap between the people that “rack and stack” and those that build the architecture that runs the organization has grown, but only because the middle ground of administration is changing so fast that it’s tough to keep up.


Tom’s Take

If I were to change jobs tomorrow I’m sure that I could get back in the saddle with a couple of weeks of hard study. But the question I keep asking myself is “Why would I want to?” I’ve learned that my value doesn’t come from my typing speed or my encyclopedia of networking command arguments any more. It comes from a greater knowledge of making networking work better and integrate more tightly into the organization. I’m a resource, not a reactionary. And so when I look to what I would end up doing in a new role I see myself learning more and more about Python and automation and less about what new features were added in the latest OSPF release on Cisco IOS. Because knowing how to integrate technology at a high level is more valuable to everyone than just knowing the commands to type to turn the lights on.

Not The Cisco of John Chambers Anymore

I just got back from Cisco Live 2017 last night and I had a blast at the show. There was a lot of discussion about new architectures, new licensing models, and of course, Tech Field Day Extra. However, one of the most interesting topics went largely under the radar. I think we’re fully in the transition of Cisco away from being the Company of John Chambers.

Steering A Tall Ship

John Chambers wasn’t the first CEO of Cisco. But he’s the one that most people would recognize. He transformed the company into the juggernaut that it is today. He watched Cisco ascend to the leader in the networking space and helped it transform into a company that embraced voice, security, and even servers and compute as new business models.

John’s Cisco is a very unique animal. It’s not a single company. It’s a collection of many independent companies with their own structures and goals all competing with each other for resources. If John decided that UCS was more important to his goals this quarter, he shifted some of the support assets to focus on that business unit. It was a featured product, complete with healthy discounts to encourage user adoption.

Product lines that didn’t perform as well were usually shown the door or swept under the rug. Even larger, well-publicized acquisitions tended to disappear under the spotlight of harsh criticism. Flip Video, Cius, and even Umi are not only lackluster products, but I bet you even forgot about one or two of them. John didn’t like highlighting failures any more than any of us, but the failures were often highlighted in spite of their stellar up-front marketing and sudden disappearance.

You can’t run the ship forever, though. Eventually, John knew he would need to step down. He had courted many, many heirs apparent in his time at Cisco. There were literally a dozen or more people inside the organization that saw themselves as the next CEO of the company. And when the time came to name his successor, Chuck Robbins was not the first name on a lot of lists. But his ascension to the throne of the networking powerhouse is turning heads.

Turning The Tall Ship

By all accounts, Cisco is a company in transition. Beset on all sides by cheaper merchant silicon, an industry shift to software-focused architecture, and several upstart companies featuring the best and brightest Cisco talent from years past. Cisco is facing multiple challenges that would have been singularly laughable a decade ago.

Part of this challenge comes from the reliance on the hardware model that John Chambers so proudly touted. John loves hardware. There’s margin in hardware. Hardware occupies space. It reminds people of the importance of things. And hardware eventually needs to be replaced. These all speak to the model of a company like the old IBM run by Tom Watson.

But Chuck Robbins sees Cisco differently. His push toward software is turning the ship away from dwindling hardware margins. The Intuitive Network architecture is setting Cisco up to rely more on software innovation than ever before. These are the kinds of organizational shifts that we’ve seen IBM go through as they focused on becoming more aligned with the direction of the industry. But these massive changes aren’t the only things that show how Cisco has transformed.

John Chambers loved the idea of having many, many business units. They were like sworn vassals pledging their loyalty to a distant king. The more voices showing the allegiance, the better. And those vassals could be courted as the possible successor to the throne should the prove worthy. So, when Chuck ascended to the head of the table, he showed his distaste for the vassal approach. He quietly allowed his competitors for the top job to exit gracefully on their terms. That’s not uncommon in situations where the throne is hotly contested.

Chuck also started collapsing those dozens of business units into organizational structure that makes sense. Not marketing wrappers, but real changes. Where before Networking and Security were two ships passing the night, they now run under the control of one person, David Goeckeler. The old Cisco system would have had two or more people reporting back to Chambers. Now, Robbins has one person to talk to about the direction of both of these key pillars of Cisco’s product lines.

A curious appearance of the shift in organizational focus was visible at Cisco Live 2017. In years past, a vice president has served as “host” for the event. They introduce the keynotes and give statistics about the attendance and other key facts. They also did the “interview” of the celebrity keynote speaker on Thursday. This year, there was no host. Chuck came on stage for his keynote without introduction. He did his speech and closed the session without anyone else on stage aside from his guests. On Thursday, he was the one interviewing the celebrity speaker, Brian Cranston.

It may not sound like much, but all of these little things add up to a very interesting change in Cisco’s organization. Chuck Robbins is going to take a much different role than Chambers. He’s going to be closer to the products. He’s going to be more involved in decisions. He’s going to be the one driving the ship rather than waiting for someone to execute decisions he’s suggested. Will that be enough to help Cisco keep their position in the networking space? Only time will tell.


Tom’s Take

I’ve said before that in the sports world, you never want to be the coach that follows the legend. Everything you do will be scrutinized through their lens and compared negatively. Some very good people can emerge from the shadow of their predecessor, but most are doomed to spend very good years being compared unfairly to the myth of the past.

At first, it looked like Chuck Robbins was headed down the same path. But with the major internal changes, the focus on software instead of hardware, and his more hands-on approach to management, I think we’re quickly going to find ourselves speaking of Cisco in the same way we refer to IBM today as “Not Tom Watson’s IBM”. I hope that the Cisco of Chuck Robbins succeeds and thrives so that in the future people will refer not to Chuck Robbins as the successor of John Chambers but instead refer to John Chambers as the guy who came before the Great Chuck Robbins.