Author Archives: networkingnerd

Unknown's avatar

About networkingnerd

Tom Hollingsworth, CCIE #29213, is a former network engineer and current organizer for Tech Field Day. Tom has been in the IT industry since 2002, and has been a nerd since he first drew breath.

More Bang For Your Budget With Whitebox

Posted on by
1

white-box-sdn-nfv

As whitebox switching starts coming to the forefront of the next buying cycle for enterprises, decision makers are naturally wondering about the advantages of buying cheaper hardware. Is a whitebox switch going to provide more value for me than buying something from an established vendor? Where are the real savings? Is whitebox really for me? One of the answers to this puzzle comes not from the savings in whitebox purchases, but the capability inherent in rapid deployment.

Ten Thousand Spoons

When users are looking at the acquisition cost advantages of buying whitebox switches, they typically don’t see what they would like to see. Ridiculously cheap hardware isn’t the norm. Instead, you see a switch that can be bought for a decent discount. That does take into account that most vendors will give substantial one-time discounts to customers to entice them into more lucrative options like advanced support or professional services.

The purchasing advantage of whitebox doesn’t just come from reduced costs. It comes from additional unit purchases. Purchasing budgets don’t typically spell out that you are allowed to buy ten switches and three firewalls. They more often state that you are allowed to spend a certain dollar amount on devices of a specific type. Savvy shoppers will find deals or discounts to get more for their dollar. The real world of purchasing budgets means that every dollar will be spent, lest the available dollars get reduced next year.

With whitebox, that purchasing power translates into additional units for the same budget amount. If I could buy three switches from Vendor X or five switches from Whitebox Vendor Y, ceteris paribus I would buy the whitebox switches. If the purpose of the purchase was to connect 144 ports, then that means I have two extra switches lying around. Which does seem a bit wasteful.

However, the option of having spares on the shelf becomes very appealing. Networks are supposed to be built in a way to minimize or eliminate downtime because of failure. The network must continue to run if a switch dies. But what happens to the dead switch? In most current cases, the switch must be sent in for warranty replacement. Services contracts with large networking vendors give you the option for 4-hour, overnight, or next business day replacements. These vendors will even cross-ship you the part. But you are still down the dead switch. If the other part of the redundant pair goes down, you are going to be dead in the water.

With an extra whitebox switch on the shelf you can have a ready replacement. Just slip it into place and let your orchestration and provisioning software do the rest. While the replacement is shipping, you still have redundancy. It also saves you from needing to buy a hugely expensive (and wildly profitable) advanced support contract.

All You Need Is A Knife

Suppose for a moment that we do have these switches sitting around on a shelf doing nothing but waiting for the inevitable failure in the network. From a cost perspective, it’s neutral. I spent the same budget either way, so an unutilized switch is costing me nothing. However, what if I could do something with that switch?

The real advantage of whitebox in this scenario comes from the ability to use non-switching OSes on the hardware. Think for a moment about something like a network packet monitor. In the past, we’ve needed to download specialized software and slip a probing device into the network just for the purposes of packet collection. What if that could be done by a switch? What if the same hardware that is forwarding packets through the network could also be used to monitor them as well?

Imagine creating an operating system that runs on top of something like ONIE for the purpose of being a network tap. Now, instead of specialized hardware for that purpose you only need to go and use one of the switches you have lying around on the shelf and repurpose it into a sensor. And when it’s served that purpose, you put it back on the shelf and wait until there is a failure before going back to push it into production as a replacement. With Chef or Puppet, you could even have the switch boot into a sensor identity for a few days and then provision it back to being a data forwarding switch afterwards. No need for messy complicated software images or clever hacks.

Now, extend those ideas beyond sensors. Think about generic hardware that could be repurposed for any function. A switch could boot up as an inline firewall. That firewall could be repurposed into a load balancer for the end of the quarter. It could then become a passive IDS during an attack. All without moving. The only limitation is the imagination of the people writing code for the device. It may not ever top the performance of a device running purely for the purpose of a given function, but the flexibility of having a device that can serve multiple functions without massive reconfiguration would win out in the long run for many applications. Flexibility is more key than overwhelming performance.

Tom’s Take

Whitebox is still finding a purpose in the enterprise. It’s been embraced by webscale, but the value to the enterprise is not found in massive capabilities like that. Instead, the additional purchasing power that can be derived from additional unit purchases for the same dollar amount leads to reduced support contract costs and even new functionality increases from existing hardware lying around that can be made to do so many other things. Who could have imagined that a simple switch could be made to do the job of many other purpose-built devices in the data center? Isn’t it ironic, don’t you think?

 

Making Your Wireless Guest Friendly

Posted on by
2

Wireless

During the recent Virtualization Field Day 4, I was located at a vendor building and jumped on their guest wireless network. There are a few things that I need to get accomplished before the magic happens at a Tech Field Day event, so I’m always on the guest network quickly. It’s only after I take care of a few website related items that I settle down into a routine of catching up on email and other items. That’s when I discovered that this particular location blocked access to IMAP on their guest network. My mail client stalled out when trying to fetch messages and clear my outbox. I could log into Gmail just fine and send and receive while I was on-site. But my workflow depends on my mail client. That made me think about guest WiFi and usability.

Be Our (Limited) Guest

Guest WiFi is a huge deal for visitors to an office. We live in a society where ever-present connectivity is necessary. Email notifications, social media updates, and the capability to look up necessary information instantly have pervaded our lives. For those of us fortunate enough to still have an unlimited cellular data plan, our connectivity craving can be satisfied by good 3G/LTE coverage. But for those devices lacking a cellular modem, or the bandwidth to exercise it, we’re forced to relay of good old 802.11a/b/g/n/ac to get online.

Most companies have moved toward a model of providing guest connectivity for visitors. This is far cry from years ago when snaking an Ethernet cable across the conference room was necessary. I can still remember the “best practice” of disabling the passthrough port on a conference room IP phone to prevent people from piggybacking onto it. Our formerly restrictive connectivity model has improved drastically. But while we can get connected, there are still some things that we limit through software.

Guest network restrictions are nothing new. Many guest networks block malicious traffic or traffic generally deemed “unwanted” in a corporate environment, such as Bittorrent or peer-to-peer file sharing protocols. Other companies take this a step further and start filtering out bandwidth consumers and the site associated with them, such as streaming Internet radio and streaming video, like YouTube and Vimeo. It’s not crucial to work (unless you need your cat videos) and most people just accept it and move on.

The third category happens, for the most part, at large companies or institutions. Protocols are blocked that might provide covert communications channels. IMAP is a good example. The popular thought is that by blocking access to mail clients, guests cannot exfiltrate data through that communications channel. Forcing users onto webmail gives the organization an extra line of defense through web filters and data loss prevention (DLP) devices that constantly look for data leakage. Another protocol that is added in this category is IPSec or SSL VPN connections. In these restrictive environments, any VPN use is generally blocked and discouraged.

Overstaying Your Welcome

Should companies police guest wireless networks for things like mail and VPN clients? That depends on what you think the purpose of a guest wireless network is for. For people like me, guest wireless is critical to the operation of my business. I need access to websites and email and occasionally things like SSH. I can only accomplish my job if I have connectivity. My preference would be to have a guest network as open as possible to my needs.

Companies, on the other hand, generally look at guest wireless connectivity as a convenience provided to guests. It’s more like the phone in the lobby by the reception desk. In most cases, that phone has very restricted dialing options. In some companies, it can only dial internal extensions or a central switchboard. In others, it has some capability to dial local numbers. Almost no one gives that phone the ability to dial long distance or international calls. To the company, giving wireless connectivity to guests serves the purpose of giving them web browsing access. Anything more is unnecessary, right?

It’s a classic standoff. How do we give the users the connectivity they need while protecting the network? Some companies create a totally alien guest network with no access to the inside and route all traffic through it. That’s almost a requirement to avoid unnecessary regulatory issues. Others use a separate WAN connection to avoid having the guest network potentially cause congestion with the company’s primary connection.

The answers to this conundrum aren’t going to come easily. But regardless of this users need to know what works and what doesn’t. Companies need to be protected against guest users doing things they aren’t supposed to. How can we meet in the middle?

A Heaping Helping of Our Hospitality

The answer lies in the hospitality industry. Specifically in those organizations that offer tiered access for their customers. Most hotels will give you the option of a free or reduced rate connection that is rate limited or has blocks in place. You can upgrade to the premium tier and unlock a faster data cap and access to things like VPN connections or even public addresses for things like video conferencing. It’s a two-tier plan that works well for the users.

Corporate wireless should follow the same plan. Users can be notified that their basic connectivity has access to web browsing and other essential items, perhaps at a rate limit to protect the corporate network. For those users (like me) that need access to faster network speeds or uncommon protocols like IMAP, you could setup a “premium” guest network that has more restrictive terms of use and perhaps gathers more information about the user before allowing them onto the network. This is also a good solution for vendors or contractors that need access to more of the network that a simple guest solution can afford them. They can use the premium tier with more restrictions and the knowledge that they will be contacted in the event of data exfiltration. You could even monitor this connection more stringently to insure nothing malicious is going on.

Tom’s Take

Guest wireless access is always going be an exercise is balance. You need to give your guests all the access you can without giving them the keys to the kingdom. Companies providing guest access need to adopt a tiered model like that of the hospitality industry to provide the connectivity needed for power users while still offering solutions that work for the majority of visitors. At the very least, companies need to notify users on the splash page / captive portal which services are disabled. This is the best way to let your guests know what’s in store for them.

Rules Shouldn’t Have Exceptions

Posted on by
2

MerkurRazor

On my way to Virtualization Field Day 4, I ran into a bit of a snafu at the airport that made me think about policy and application. When I put my carry-on luggage through the X-ray, the officer took it to the back and gave it a thorough screening. During that process, I was informed that my double-edged safety razor would not be able to make the trip (or the blade at least). I was vexed, as this razor had flown with me for at least a whole year with nary a peep from security. When I related as much to the officer, the response was “I’m sorry no one caught it before.”

Everyone Is The Same, Except For Me

This incident made me start thinking about polices in networking and security and how often they are arbitrarily enforced. We see it every day. The IT staff comes up with a new plan to reduce mailbox sizes or reduce congestion by enforcing quality of service (QoS). Everyone is all for the plan during the discussion stages. When the time comes to implement the idea, the exceptions start happening. Upper management won’t have mailbox limitations. The accounting department is exempt from the QoS policy. The list goes on and on until it’s larger than the policy itself.

Why does this happen? How can a perfect policy go from planning to implementation before it falls apart? Do people sit around making up rules they know they’ll never follow? That does happen in some cases, but more often it happens that the folks that the policy will end up impacting the most have no representation in the planning process.

Take mailboxes for example. The IT department, being diligent technology users, strive for inbox zero every day. They process and deal with messages. They archive old mail. They keep their mailbox a barren wasteland of in-process things and shuffle everything else off to the static archive. Now, imagine an executive. These people are usually overwhelmed by email. They process what they can but the wave will always overtake them. In addition, they have no archive. Their read mail sits around in folders for easy searching and quick access when a years-old issue becomes present again.

In modern IT, any policies limiting mailbox sizes would be decided by the IT staff based on their mailbox size. Sure, a 1 GB limit sounds great. Then, when the policy is implemented the executive staff pushes back with their 5 GB (or larger) mailboxes and says that the policy does not apply to them. IT relents and finds a way to make the executives exempt.

In a perfect world, the executive team would have been consulted or had representation on the planning team prior to the decision. The idea of having huge mailboxes would have been figured out in the planning stage and dealt with early instead of making exceptions after the fact. Maybe the IT staff needed to communicate more. Perhaps the executive team needed to be more involved. Those are problems that happen every day. So how do we fix them?

Exceptions Are NOT The Rule

The way to increase buy-in for changes and increase communication between stakeholders is easy but not without pain. When policies are implemented, no deviations are allowed. It sounds harsh. People are going to get mad at you. But you can’t budge an inch. If a policy exception is not documented in the policy it will get lost somewhere. People will continue to be uninvolved in the process as long as they think they can negotiate a reprieve after the fact.

IT needs to communicate up front exactly what’s going into the change before the the implementation. People need to know how they will be impacted. Ideally, that will mean that people have talked about the change up front so there are no surprises. But we all know that doesn’t happen. So making a “no exceptions” policy or rule change will get them involved. Because not being able to get out of a rule means you want to be there when the rules get decided so you can make your position clear and ensure the needs of you and your department are met.

Tom’s Take

As I said yesterday on Twitter, people don’t mind rules and polices. They don’t even mind harsh or restrictive rules. What they have a problem with is when those rules are applied in an arbitrary fashion. If the corporate email policy says that mailboxes are supposed to be no more than 1 GB in size then people in the organization will have a problem if someone has a 20 GB mailbox. The rules must apply to everyone equally to be universally adopted. Likewise, rules must encompass as many outlying cases as possible in order to prevent one-off exceptions for almost everyone. Planning and communication are more important than ever when planning those rules.

Time For A Data Diet?

Posted on by
1
Embed from Getty Images

I’m running out of drive space. Not just on my laptop SSD or my desktop HDD. But everywhere. The amount of data that I’m storing now is climbing at an alarming rate. What’s worse is that I often forget I have some of it until I go spelunking back through my drive to figure out what’s taking up all that room. And it’s a problem that the industry is facing too.

The Data Junkyard

Data is accumulating. You can’t deny that. Two factors have lead to this. The first is that we now log more data from things than ever before. In this recent post from Chris Evans (@ChrisMEvans), he mentions that Virgin Atlantic 787s are generating 500GB of data per flight. I’m sure that includes telemetry, aircraft performance, and other debugging information that someone at some point deemed crucial. In another recent article from Jacques Mattheij (@JMattheij), he mentions that app developers left the debug logging turned on, generating enormous data files as the system was in operation.

Years ago we didn’t have the space to store that much data. We had to be very specific about what needed to be capture and stored for long periods of time. I can remember having a 100MB hard drive in my first computer. I can also remember uninstalling and deleting several things in order to put a new program on. Now there is so much storage space that we don’t worry about running out unless a new application makes outrageous demands.

You Want To Use The Data?

The worst part about all this data accumulation is that once it’s been stored, no one ever looks at it again. This isn’t something that’s specific to electronic data, though. I can remember seeing legal offices with storage closets dedicated to boxes full of files. Hospitals have services that deal with medical record storage. In the old days, casinos hired vans to shuffle video tapes back and forth between vaults and security offices. All that infrastructure just on the off-chance that you might need the data one day.

With Big Data being a huge funding target and buzzword source today, you can imagine that every other startup in the market is offering to give you some insight into all that data that you’re storing. I’ve talked before about the drive for analysis of data. It’s the end result of companies trying to make sense of the chaos. But what about the stored data?

Odds are good that it’s going to just sit there in perpetuity. Once the analysis is done on all this data, it will either collect dust in a virtual file box until it is needed again (perhaps) in the far future or it will survive until the next SAN outage and never be reconstructed from backup. The funny thing about this collected data cruft is that no one misses it until the subpoena comes.

Getting Back To Fighting Weight

The solution to the problem isn’t doing more analysis on data. Instead, we need to start being careful about what data we’re storing in the first place. When you look at personal systems like Getting Things Done, they focus on stemming the flow of data quickly to give people more time to look at the important things. In much the same way, instead of capturing every bit coming from a data source and deciding later what to do with it, the decision needs to be made right away. Data Scientists need to start thinking like they’re on a storage budget, not like they’ve been handed the keys to the SAN kingdom.

I would be willing to bet that a few discrete decisions in the data collection process about what to keep and what to throw away would significantly cut down on the amount data we need to store and process. Less time spent querying and searching through that mess would optimize data retrieval systems and make our infrastructure run much faster. Think of it like spring cleaning for the data garage.

Tom’s Take

I remember a presentation at Networking Field Day a few years ago when Statseeker told us that they could scan data points from years in the past down to the minute. The room collectively gasped. How could you look that far back? How big are the drives in your appliance? The answer was easy: they don’t store every little piece of data coming from the system. They instead look at very specific things that tell them about the network and then record those with an eye to retrieval in the future. They optimize at storage time to help the impact of lookup in the future.

Rather than collecting everything in the world in the hopes that it might be useful, we need to get away from the data hoarding mentality and trim down to something more agile. It’s the only way our data growth problem is going to get better in the near future.

If you’d like to hear some more thoughts on the growing data problem, be sure to check out the Tech Talk sponsored by Fusion-io.

 

A Bright And Happy 2015 Ahead

Posted on by
3

Welcome to a new year finally divisible by five! This is a year devoid of extra February days, Olympics, or anything else. It’s a chance for us to take a look at technology and make things better and easier for users and IT staff. It’s also probably going to be called the year of VDI, NFV, and SDN. Again.

Rather than writing a wrap up post for the end of 2014 like so many other sites, I like to look at what I said I was going to do 365 days ago and see if I followed through on them. It’s a way to keep myself honest and also to see how the year transformed around me and my goals.

Looking at 2014

Thankfully, my goals for 2014 were modest. I wanted to get more involved with the people in the IT industry. And I did that in a big way. I went to a ton of conferences and events through the year. Cisco Live, VMworld, and HP Discover Barcelona were all on my list this year, as well as all of the Tech Field Day events I took part in as an organizer. It was a grand opportunity to meets lots of people in the technology space. I got to interact with the old guard and see the rise of new stars. Jobs changed. People sought out new careers. And through it all I got a real sense that the people that are going to change the world in technology are passionate about what they do.

Passion is the key to making sense out of what we do. I’m not saying that you have to be so in love with your job that you are blinded to the world. What I mean is that you need to have passion about the things that matter to you. For me, it’s about seeing new technology and exposing people to it. I love Tech Field Day. It warms my heart when people come to me during and after the event and tell me that they were able to see so much more than they imagined. When a delegate tells me they finally had a chance to meet one of their tech idols or had a game changing conversation during the limo ride between presenters I genuinely smile. Those are the kinds of moments that make everything worth it for me.

What’s In Store For 2015?

For now, the major things aren’t going to change any time soon. My Bruce Wayne job is still going to be Tech Field Day. My Batman job is going to be writing on this blog. But I’m going to try a few new things and see how they work out.

Markdown

I’ve played around with the idea of writing in Markdown for a while now. It’s a simple language that turns thoughts into HTML with out needing to remember some of the more irritating code sections. I’ve never really committed to it before, looking at it more as a hobby or a thing I would eventually get to. Well, for 2015 I’m going to commit to writing all of my posts in Markdown. There’s no better way to learn than a trial by fire. I don’t think the regular posts are going to be a big deal, but the 2015 Cisco Live Twitter List could be fun.

If you’d like to see a great reference sheet for Markdown, check out Greg Ferro’s (@EtherealMind) page on Markdown Reference.

Blog Themes

I wanted to retheme my blog for 2015. I investigated several options and ultimately abandoned all of them because I could never find the right combination. I’m picky about many things I work with every day, including my blog theme, my backpack/messenger bag, and my computer desk. Since I’m hosted on WordPress.com, I can’t just install any theme I want or make modifications to it as I would like. I’m going to keep investigating some ideas and may try them out now and then. Just don’t be surprised if things look slightly different one day in the near future.

Cisco Live Managmement

One of the ideas that I’m going to float out here six months early for Cisco Live is a poll/form for picking the best time to take the Twitter photo. Every year for the last four years we’ve taken a huge photo with all the social media crew at Cisco Live. In the past couple of years we’ve had some issues getting everyone in the picture due to scheduling. This year, Jeff Fry (@FryGuy_PA) and I want to make sure that no one is left out that wants to be in the big photo because of their schedule. I’m going to put up a poll in the next couple of months to pick the best possible time for the photo. And we’ll make sure to publish the results and work with the Cisco Live Social Media staff to get the photographer for that time.

I’m also looking at creating some other spreadsheets to keep track of other information during the event, so if you get a random email from me about it keep in mind that I’m trying to keep myself sane this year.

Tom’s Take

I’m excited for 2015. There’s going be a lot of technology to write about. Tech Field Day will be in Austin, Boston, and Silicon Valley. We’re going to be talking about wireless, networking, storage, and event Big Data! I’m also looking forward to reconnecting with my friends and peers this year and meeting new and exciting people. Through it all, I’m going to be writing away here as well to put my thoughts down about trends and ideas in the industry. There may be the occasional technical piece now and then, since explanation of complex tech subjects is something I think there needs to be more of.

To my readers, thanks for helping me realize how important blogging is the community. Keep posting comments and sharing my thoughts with the world. And in 2015 we’ll have more fun that we’ve had in a long while.

Q And A Should Include The E

Posted on by
3
Embed from Getty Images

The IT world is cyclical for sure. I’ve seen trends and topics repeating themselves over and over again in my relatively short time here. I find it interesting that we keep solving similar problems over and over again. I also find it fascinating that this particular issue leads to the reason why blogs are so important.

Any Questions?

Questions abound in IT. It’s the nature of the industry. However, it’s not just new questions that we create when technology leaps past us. We keep asking the same questions over and over again. This is the field of study that created the FAQ, remember?

In recent memory, I find the same questions being asked over and over again:

  • What is SDN?
  • How can SDN help me?
  • What makes this different from what we’ve done before?

You’ve probably asked those very same questions. Perhaps you found the answers you were looking for. Perhaps you’re still trying to figure it out. The problem is that those questions are still being asked. The industry should have evolved to the point where the simple questions have been answered with simple answers. Complex questions, or those questions that need more in-depth discussion, should be treated as such. Yes, the question of what SDN really is would take more than a cursory paragraph on a blog, but we should be able to at least answer it with enough specificity to make the user not feel like they been slighted.

Questions will never stop coming in IT. But how should we handle them?

Any Answers?

Questions may abound in IT, but the answers drive IT. People make a career out of being the person with the answers. It’s in all the marketing jargon. It’s why we create blogs. Even though most of my writing in the last year has been focused on industry trends or non-technical focused posts, the top three posts on my blog are still answers to simple questions:

  1. When Is A Trunk Not A Trunk?
  2. Switchport Voice VLAN – What Does It Do?
  3. Why Is My SFP Not Working?

These posts are far and away the most popular. I even saw this a few months ago and it made me smile:

This would make it seem like people are in need of answers. Any blogger can look at the incoming search terms for their blog and see all the things that brought readers to them. People want answers and they will keep looking until they find them. But why?

Explain It

I never understood why people kept searching for answers until I thought about satisfaction. I think Randall Munroe summed up the satisfaction (or lack thereof) angle here:

Who are you, DenverCoder9?!? (Thanks XKCD)
Who are you, DenverCoder9?!? (Thanks XKCD)

People can find answers easily. But they won’t stop looking until they are satisfied with the answer. It’s easy to find people saying things like “That’s not supported” or “RTFM” when you’re looking for an answer to a particularly difficult problem. And if you’ve ever called a tech support line, you know how unfulfilling the unsupported answer can feel.

That’s when explanation comes into play for me. First, an admission: I’m a chronic explainer. If you’ve ever met me and had a conversation with me for more than three minutes, you know I explain things. I talk about comic books and movies and technical topics in more depth than I should. That’s because I want things explained to me. Explaining how OSPF area calculations are done is as important as explaining how Captain America ended up wielding Mjolnir.

Think about the following answers:

This is unsupported.

or

This is unsupported on that platform because the CPU doesn’t have enough horsepower to process the packets in real time. We tried cutting down on the processing time but it just overwhelmed the unit no matter how much we tried. So rather than dealing with poor performance, we marked it as unsupported.

Both answers are technically correct. But the second is much more satisfying because the explanation is there instead of just the distilled answer.

The IT world needs more explanation. We need to know why things work the way they do instead of just getting a response of a few words. The explanation has the keys to understanding the answer to the question in its totality. It prevents us from asking the same questions over and over again. It leaves us fulfilled and ready to seek out the next question that needs to be asked.

Expiring The Internet

Posted on by
Reply
Embed from Getty Images

An article came out this week that really made me sigh.  The title was “Six Aging Protocols That Could Cripple The Internet“.  I dove right in, expecting to see how things like Finger were old and needed to be disabled and removed.  Imagine my surprise when I saw things like BGP4 and SMTP on the list.  I really tried not to smack my own forehead as I flipped through the slideshow of how the foundation of the Internet is old and is at risk of meltdown.

If It Ain’t Broke

Engineers love the old adage “If it ain’t broke, don’t fix it!”.  We spend our careers planning and implementing.  We also spend a lot of time not touching things afterwards in order to prevent it from collapsing in a big heap.  Once something is put in place, it tends to stay that way until something necessitates a change.

BGP is a perfect example.  The basics of BGP remain largely the same from when it was first implemented years ago.  BGP4 has been in use since 1994 even though RFC 4271 didn’t officially formalize it until 2006.  It remains a critical part of how the Internet operates.  According to the article, BGP is fundamentally flawed because it’s insecure and trust based.  BGP hijacking has been occurring with more frequency, even as resources to combat it are being hotly debated.  Is BGP to blame for the issue?  Or is it something more deeply rooted?

Don’t Fix It

The issues with BGP and other protocols mentioned in the article, including IPv6, aren’t due to the way the protocol was constructed.  It is due in large part to the humans that implement those protocols.  BGP is still in use in the current insecure form because it works.  And because no one has proposed a simple replacement that accomplishes the goal of fixing all the problems.

Look at IPv6.  It solves the address exhaustion issue.  It solves hierarchical addressing issues.  It restores end-to-end connectivity on the Internet.  And yet adoption numbers still languish in the single digit percentage.  Why?  Is it because IPv6 isn’t technically superior? Or because people don’t want to spend the time to implement it?  It’s expensive.  It’s difficult to learn.  Reconfiguring infrastructures to support new protocols takes time and effort.  Things that are better spent on answering user problems or taking on additional tasks as directed by management that doesn’t care about BGP insecurity until the Internet goes down.

It Hurts When I Do This

Instead of complaining about how protocols are insecure, the solution to the problem should be two fold: First, we need to start building security into protocols and expiring their older, insecure versions.  POODLE exploited SSLv3, an older version that served as a fallback to TLS.  While some old browsers still used SSLv3, the simple easy solution was to disable SSL and force people to upgrade to TLS-capable clients.  In much the same way, protocols like NTP and BGP can be modified to use more security.  Instead of suggesting that people use those versions, architects and engineers need to implement those versions and discourage use of the old insecure protocols by disabling them.  It’s not going to be easy at first.  But as the movement gains momentum, the solution will work.

The next step in the process is to build easy-to-configure replacements.  Bolting security onto a protocol after the fact does stop the bleeding.  But to fix the underlying symptoms, the security needs to be baked into the protocol from the beginning.  But doing this with an entirely new protocol that has no backwards compatibility will be the death of that new protocol.  Just look at how horrible the transition to IPv6 has been.  Lack of an easy transition coupled with no monetary incentive and lack of an imminent problem caused the migration to drag out until the eleventh hour.  And even then there is significant pushback against an issue that can no longer be ignored.

Building the next generation of secure Internet protocols is going to take time and transparent effort.  People need to see what’s going into something to understand why it’s important.  The next generation of engineers needs to understand why things are being built the way they are.  We’re lucky in that many of the people responsible for building the modern Internet are still around.  When asked about limitations in protocols the answer remains remarkably the same – “We never thought it would be around this long.”

The longevity of quick fixes seems to be the real issue.  When the next generation of Internet protocols is built there needs to be a built-in expiration date.  A point-of-no-return beyond which the protocol will cease to function.  And there should be no method for extending the shelf life of a protocol to forestall it’s demise.  In order to ensure that security can’t be compromised we have to resign ourselves to the fact that old things need to be put out to pasture.  And the best way to ensure that new things are put in place to supplant them is to make sure the old things go away on time.

Tom’s Take

The Internet isn’t fundamentally broken.  It’s a collection of things that work well in their roles that maybe have been continued a little longer than necessary.  The probability of an exploit being created for something rises with every passing day it is still in use.  We can solve the issues of the current Internet with some security engineering.  But to make sure the problem never comes back again, we have to make a hard choice to expire protocols on a regular basis.  It will mean work.  It will create strife.  And in the end we’ll all be better for it.

Cisco Just Killed The CLI

Posted on by
10

DeadCLI

Gallons of virtual ink have been committed to virtual paper in the last few days with regards to Cisco’s lawsuit against Arista Networks.  Some of it is speculating on the posturing by both companies.  Other writers talk about the old market vs. the new market.  Still others look at SDN as a driver.

I didn’t just want to talk about the lawsuit.  Given that Arista has marketed EOS as a “better IOS than IOS” for a while now, I figured Cisco finally decided to bite back.  They are fiercely protective of IOS and they have to be because of the way the trademark laws in the US work.  If you don’t go after people that infringe you lose your standing to do so and invite others to do it as well.  Is Cisco’s timing suspect? One does have to wonder.  Is this about knocking out a competitor? It’s tough to say.  But one thing is sure to me.  Cisco has effectively killed the command line interface (CLI).

“Industry Standards”

EOS is certainly IOS-like.  While it does introduce some unique features (see the NFD3 video here), the command syntax is very much IOS.  That is purposeful.  There are two broad categories of CLIs in the market:

  • IOS-like – EOS, HP Procurve, Brocade, FTOS, etc
  • Not IOS-like – Junos, FortiOS, D-Link OS, etc

What’s funny is that the IOS-like interfaces have always been marketed as such.  Sure, there’s the famous “industry standard” CLI comment, followed by a wink and a nudge.  Everyone knows what OS is being discussed.  It is a plus point for both sides.

The non-Cisco vendors can sell to networking teams by saying that their CLI won’t change.  Everything will be just as easy to configure with just a few minor syntax changes.  Almost like speaking a different dialect of a language.  Cisco gains because more and more engineers become familiar with the IOS syntax.  Down the line, those engineers may choose to buy Cisco based on familiarity with the product.

If you don’t believe that being IOS-like is a strong selling point, take a look PIX and Airespace.  The old PIX OS was transformed into something that looked a lot more like traditional IOS.  In ASA 8.2 they even changed the NAT code to look like IOS.  With Airespace it took a little longer to transform the alien CLI into something IOS-like.  They even lost functionality in doing so, simply to give networking teams an interface that is more friendly to them.  Cisco wants all their devices to run a CLI that is IOS-like.  Junos fans are probably snickering right now.

In calling out Arista for infringing on the “generic command line interface” in patent #7,047,526, Cisco has effectively said that they will start going after companies that copy the IOS interface too well.  This leaves companies in a bit of conundrum.  How can you continue to produce an OS with an “industry standard” CLI and hope that you don’t become popular enough to get noticed by Cisco?  Granted, it seems that all network switching vendors are in the market somehow.  But at what point does being a big enough get the legal hammer brought to bear?  Do you have to be snarky in marketing messages? Attack the 800-pound gorilla enough that you anger them?  Or do you just have to have a wildly successful quarter?

Laid To REST

Instead, what will happen is a tough choice.  Either continue to produce the same CLI year and year and hope that you don’t get noticed or overhaul the whole system.  Those that choose not to play Russian Roulette with the legal system have a further choice to make.  Should we create a new, non-infringing CLI from the ground up? Or scrap the whole idea of a CLI moving forward?  Both of those second choices are going to involve a lot of pain and effort.  One of them has a future.

Rewriting the CLI is a dead-end road.  By the time you’ve finished your Herculean task you’ll find the market has moved on to bigger and better things.  The SDN revolution is about making complex networks easier to program and manage.  Is that going to be accomplished via yet another syntax?  Or will it happen because of REST APIs and programing interfaces?  Given an equal amount of time and effort on both sides, the smart networking company will focus their efforts on scrapping the CLI and building programmability into their devices.  Sure, the 1.0 release is going to sting a little.  It’s going to require a controller and some rough interface conventions.  But building the seeds of a programmable system now means it will be growing while other CLIs are withering on the vine.

It won’t be easy.  It won’t be fun.  And it’s a risk to alienate your existing customer base.  But if your options are to get sued or spend all your effort on a project that will eventually go the way of the dodo your options don’t look all that appealing anyway.  If you’re going to have to go through the upheaval of rewriting something from the ground up, why not choose to do it with an eye to the future?

Tom’s Take

Cisco and Arista won’t be finished for a while.  There will probably be a settlement or a licensing agreement or some kind of capitulation on both sides in a few years time.  But by that point, the fallout from the legal action will have finally finished off the CLI for good.  There’s no sense in gambling that you won’t be the next target of a process server.  The solution will involve innovative thinking, blood, sweat, and tears on the part of your entire development team.  But in the end you’ll have a modern system that works with the new wave of the network.  If nothing else, you can stop relying on the “industry standard” ploy when selling your interface and start telling your customers that you are setting the new standard.

 

Vendor Whitebox Switches – Better Together?

Posted on by
1

ChocoPeanut

Whitebox switching has moved past the realm of original device manufacturers and has been taken up by traditional networking vendors. Andre Kindness (@AndreKindness) of Forrester recently posted that he fields several calls from his customers every day asking about a particular vendor’s approach to whitebox switching. But what do these vendor offerings look like? And can we predict how a given vendor will address the whitebox market?

Chocolate In My Peanut Butter

Dell was one of the first traditional networking vendors to announce a whitebox switch offering that decoupled the operating system from the switching hardware. Dell offered packages from Cumulus Linux and Big Switch Networks alongside their PowerConnect lineup. This makes sense when you consider that the operating system on the switch has never been the strong suit of Dell. The PowerConnect OS is not very popular with network engineers, being very dissimilar from more popular CLIs such as Cisco IOS and its look-alikes.  Their attempts to capitalize on the popularity of Force Ten OS (FTOS) and adapt it or use on PowerConnect switches has been difficult at best, due to the divide been hardware architecture of the two platforms.

What Dell is very good at is offering hardware at a greatly reduced cost. By utilizing this strength, they can enter the whitebox market successfully by partnering with OS vendors to provide customer options. This also gives them time to adapt FTOS to more switches and attempt to drive acquisition posts down once the port of FTOS to PowerConnect is complete.

Peanut Butter In My Chocolate

What happens when a vendor sees software as their strength? You get an announcement like the one last week from Juniper Networks. Juniper has put a significant amount of time and effort into Junos. The FreeBSD base of the system gives it the adaptability that Cumulus enjoys. Since Juniper sees Junos as a huge advantage, their oath to whitebox switching was to offer hardware that reduces the acquisition cost. Porting Junos to run on the OCP-based OCX1100 allows Juniper to use silicon that is more in line with merchant offering price points. The value to the customer comes from existing experience with Junos allowing for reduced learning time on the new platform.

So how will the rest of the market adopt whitebox switching offerings? HP will likely go the same route as Dell, as their software picture is murky with products split evenly between HP Procurve OS and 3Com/H3C Comware. HP has existing silicon manufacturing facilities that allow for economy of scale to reduce acquisition costs to the customer. Conversely, Brocade will likely leverage existing Vyatta development and investment in projects like OpenDaylight to standardize their whitebox offerings on software while offering OCP-style hardware platforms.

The 800-pound Whitebox Gorilla

And what of Cisco? Cisco had invested significant time and effort into both hardware and software. IOS is being renovated with API access and being ported into containers to broaden the platforms on which it can operate. The Cisco investment in custom silicon development is significant as well, with only the Nexus 3000 and 9000 series using merchant offerings from Broadcom. Their eventual whitebox offering could take any form.

Cisco feels very strongly about keeping IOS and its variants exclusive to Cisco hardware. Given that they sued Arista Networks late last week for patent infringement in EOS, it should be apparent how strongly they feel about IOS. That will be the impetus that pushes them to offering some limited custom silicon that is capable of running third-party operating systems. This allows Cisco to partner closely with one of those developers to ensure peak performance and tight integrations with whatever hardware Cisco includes.  They would likely offer this platform with a bundle of SmartNET support services, recouping the costs of producing the switch with some very high margin services.

The possibility of porting IOS to an OCP-like reference platform is remote at best. A whitebox IOS offering would still carry a high price tag to reflect Cisco R&D and would be priced too high above what customers would be willing to pay for total acquisition cost.  It would also open the door for someone to “port” that version of IOS to run on platforms that it shouldn’t be running on.  At the very least, it will expose Cisco in the market as having too high a price tag on their intellectual property in IOS and give competitors like Juniper and Big Switch ammunition to fight back.

Tom’s Take

When evaluating vendor whitebox offerings, be sure your assessment of the strengths matches theirs. Wide adoption of a given strategy will solidify that approach in the future. Be sure to give feedback to your local account teams and tell them the critical features you need to be supported. That will ensure the vendor has you in mind when the time comes to produce a whitebox offering.  And remember that you always have the option of going your own way.  Nothing says that you have to buy a solution with bundled services from traditional networking vendors.  If you’re willing to fly without a safety net for a while, you can find some great deals on ODM switches and OSes to run on them.

HP Networking – Hitting The Right Notes

Posted on by
2

HP has quietly been making waves recently with their networking strategies.  They recently showed off their technology around software defined networking (SDN) applications at Interop New York.  Here’s a video:

It would seem that HP has been doing a lot of hard work on the back end with SDN.  So why haven’t we heard about it?

Trumpet and Bugle

HP Networking hasn’t been in the news as much as Cisco and VMware as of late.  When you consider that both of those companies are pushing agendas related to redefining the paradigm of networking around policy and virtualization their trumpeting of those agendas makes total sense.  But even members of the League of Non-Aligned Vendors like Brocade are talking a lot about their SDN strategy with the Vyatta Controller and OpenStack integrations.  Vendors have layers and layers of plans for the “new” networking.  But HP has actually been doing it!  Why haven’t we known until now?

HP has been content to play the role of the bugler to the trumpeters of the bigger organizations.  Rather than talking over and over again about what they are planning on doing, HP waits until they’ve actually done it to talk about it.  It’s a sound strategy.  I love making everything work first and then discussing what you’ve done rather than spending week after week, month after month, talking about a plan that may or may not come to fruition.

The issue with HP is that they need to bugle a little more often to stay afloat in the space.  Only making announcements won’t cut it.  The breakneck pace of innovation and adoption is disrupting the ability of laggard developers to stay afloat.  New technologies are being supplanted by upstarts.  Docker is old news.  Now we’re talking about SocketPlane and Rocket.  You’d be forgiven if you haven’t been keeping up as a blogger or engineer.  But if you’ve missed the boat as a vendor, you’re going to have a hard time treading water.

The Tijuana Brass

How can HP solve their problem?  Technically, they need to keep doing what they’ve been doing all along.  They are making good decisions and innovating around ideas like the HP SDN App Store.  What they need to do it tell more people about it.  Get the word out.  Start some discussions around what you’re doing.  Don’t be afraid to engage.  The more you talk to people about your solutions, the more your name will come up in conversation. You need to be loud and on-key.  Herb Alpert and the Tijuana Brass weren’t popular right away.  It took years of recording and playing before the mainstream “discovered” them and popularized their music.

HP Networking has spent considerable time building SDN infrastructure.  The fact that their are OpenFlow images for a wide variety of their existing switch infrastructure is proof they are concerned about making everything fit together.  Now it’s time to tell the story.  With the impending divestiture of HP’s enterprise businesses from the consumer line, it will be far too easy to get lost in the shuffle of reorganization.  They way to prevent that is to step out and make yourself known.  Write blogs, record podcasts, and interact with the community.  Don’t be afraid to toot your own horn a little.

Disclaimer

HP invited me to attend HP Discover Barcelona as their guest.  They provided travel and lodging expenses during my time in Europe.  They did not require any blog posts or consideration for this invitation, nor where they offered any on my part.  The opinions and analysis expressed herein represents my thoughts alone.