The Hook Brings You Back

If I asked you to summarize the great works of literature in a few paragraphs, how would you do it? Would you read over the whole thing and try to give a play-by-play of the book? Would it be more like Cliff’s Notes, summarizing the major themes but skipping over the details? Maybe you’d offer up the conclusion only and leave it as an exercise to the reader to find out? There are a lot of ways to do it and almost all of them seem insurmountable.

What if there was an easy way to jump right into starting to discuss a topic or summarize something? What if you could find a way to easily get people interested in your ideas? Believe or not, it’s not as hard as you might think. People usually freak out because they feel like there are too many places to start when they want to write something. They decide to try and figure out the perfect way to get going and, more often than not, they paralyze themselves with inaction.

So how do you get things moving? You have to find the hook.

By Hook or Crook

What’s the hook? Most people think it’s like a fish hook. Something you set to reel someone in. And that’s not far from the truth. The hook, when talking about writing or even music, is a section that is designed to catch your attention and keep it. The hook is what’s responsible for those catchy choruses you can’t get out of your head once you hear them.

But the hook is also the way you can get into a heady topic. The hook is the way you get things start. You find the attention-catching part of the story or the topic that you want to talk about and you grab it. Set the hook. That’s the first step. Figuring out what you want to talk about and setting that hook.

The key is to avoid getting overwhelmed. Don’t try to say too much. The hook doesn’t work if it’s too big. It doesn’t work if it’s too complicated. You have to find something small and relatable if you want people to bite. You need a single idea. A single topic of some kind. Make it easy and your audience will surely bite on it.

Reeling Them In

Okay, so you’ve successfully set the hook. Now what? Do you just tug and tug on it until you get what you’re after? Every fisherman knows that’s a bad idea. You have to gently pull and convince your quarry to come. You have to build something that leads people to where you want them to be.

Writing is no different. You have your hook but you have to support it with facts and evidence. You have to come back to your main idea and reinforce it over and over. That’s how the hook gets into the reader’s mind. You have to make sure they aren’t going to forget it. The hook is the takeaway from the piece you’re writing.

When your reader finishes you want them to have that idea ringing in their ears and in their head. You want them to think of your idea like a chorus from a song. Resonating and repeating. Not in the insidious ear worm kind of way. But in the way of your favorite movie scenes or favorite songs. Something that they enjoy and want to keep coming back to.

Fishing In Practice

Okay, so this is all well and good when you’re trying to sit down and write something. But what about when you’re listening to a presentation? How can this help you with your pre-writing?

The biggest thing to do is to start looking for hooks early. Most good presenters will tee up an idea or a theme and run with it. They’ll do most of the hard work for you. All you have to do it pick up on it. Find the theme running through everything and start taking notes about it. Using things like mind maps are great for this style of note taking because you’re going to try and pull all your details back to that main hook.

But what if there isn’t a hook? What if the main idea is scattered or the presentation isn’t built in such a way as to present something that has a clear, definitive theme? Well, that’s where the creative part comes into play. You’re going to have to do a little fishing of your own. You need to look at the media you’re given and try to find your hook. You may have to try a few things out first to get something worth talking about. But once you find the hook in the information you’re given you’re going to want to run with it. That’s how you know you’ve found something good.


Tom’s Take

A lot of my briefings and other coverage writing on Gestalt IT uses this kind of style now. I try to find the hook to pull people in to read about what I’m discussing. It’s not always mean or nefarious. Instead I want to engage people and show them how I look at things. Hopefully it gives them a new perspective and helps them understand deep technical topics. And maybe it’s enough to bring them back for more along the way.

Tom’s Virtual Corner at Cisco Live US 2020

One of the things that I look forward to most during Cisco Live is the opportunity to meet with people. It’s been quite a few years since I’ve been to a session during the conference. My work with Tech Field Day has kept me very busy for the past several Cisco Live events. But at the end of the day I enjoy strolling down to the Social Media hub and talking to anyone I see. Because people make Cisco Live what it is.

The Legend of Tom’s Corner has grown over the years. It’s more than just a few tables in a place where people hang out. It stands for a community. It means a lot to so many different people. It’s about meeting new friends and catching up with old ones and feeling like you belong. For so many, Tom’s Corner and the Social Media Hub is the center of Cisco Live.

And yet, we now live in extraordinary times. The plan we had for what Cisco Live would look like for us earlier this year is radically different right now. Prohibitions on travel and meetings in large groups means we will be experiencing Cisco Live from our homes afar instead of the Mandalay Bay in Las Vegas. The sessions we attend will be online. The keynotes streamed without seating and traffic directions. Although the office chairs at home will probably be more comfortable than conference seating.

But what about that in-person aspect to things? What about meeting up at the Social Media Hub and hanging out with all our friends? Well, the social media aspect to the event is going to be even more important now. Twitter and Slack and iMessage are going to be our primary forms of communication. We’re going to be twice as social even without being able to be around people thanks to the need to use programs to connect. But it’s not going to feel the same without being able to see someone.

A Virtual Corner

Because things are so crazy and because we’re not all going to get to be in the same place this year to hang out at Tom’s Corner, it’s time to bring Tom’s Corner to the virtual landscape of Cisco Live. Thanks to the power of Zoom and the patronage of Tech Field Day, we’re going to be holding Tom’s Virtual Corner at Cisco Live US 2020!

With the power of the revolution of technology and video chat we’re going to have the option to hang out and chat just like we always do! Granted, we’re not going to have to fight over places to sit this year so it may be better this way. Also, less walking! We’re going to have the meeting running from about 8:00am PT through 1:00pm PT so don’t worry if you can’t join right at the start. I’m sure there are going to be people coming and going all day.

In order to be a part of Tom’s Virtual Corner at Cisco Live US 2020, you’re going to need to send me an email at tom@networkingnerd.net or a DM on Twitter with the email address you want the calendar invitation sent to. Yes, that’s a very manual process. But given the number of people that like to invade Zoom calls this is a necessary precaution. Just send me an email with the title “Tom’s Virtual Corner Invitiation” and I’ll make sure you’re on the list. After that we can get everything going just like if we were hanging at the actual corner.

This is supposed to be a fun time to hang and enjoy the company of each other in a format that is hard to replicate, so a couple of ground rules:

  • Disruptive attendees may be kicked at the discretion of the hosts.
  • Follow Wheaton’s Law as the Prime Behavior Directive. If you have a question about whether or not you’re violating that law, you probably are.
  • Be respectful of your peers and friends. Make this a positive experience for everyone. I don’t want to have to be the fun police but if that needs to happen so be it.

It’s that simple. Be cool, act cool, and we’ll have fun.


Tom’s Take

I’m going to miss the Social Media Hub this year. I’m going to miss my friends and I am also going to regret not getting to make new ones. But maybe we can salvage a bit of that spark this way. We might miss the sign pic or the crazy antics that happen with giant Lego figures or tiaras or unicorn masks. But we’ll be there in spirit and that’s what counts. And, if nothing else, the tenth anniversary of Tom’s Corner next year is going to blow the roof off the place!

Anthology Product Marketing

I’m a storyteller. I realize this based on the fact that I tell them a lot. I’ve been told by a lot of people that I tell stories all the time. I’m okay with this. And a lot of the time I’m totally good at it. But one of the side effects of being someone that enjoys telling stories is that you recognize them in others and you start critiquing.

One of the more recent trends I’ve seen in product marketing revolves around stories. We’ve seen people telling all kinds of narratives about how disparate pieces of the puzzle fit together. It’s important because it frames the discussion for everyone. But I’ve also noticed some companies focus less on the framing story and more on the pieces. And it made me realize that’s a different kind of story.

Pieces and Parts

Merriam-Webster defines an anthology as a collection of selected literary pieces or passages or works of art or music. When I think of an anthology movie or video series, I think of a collection of disconnected stories around a framing device. Sometimes that device is as tenuous as a shared narrator, such as the Twilight Zone or Tales from the Crypt. That these series have been made into movies shows how well the format can be adapted to longer media.

Whereas a typical drama has a beginning, middle, and end that follows the same characters throughout the whole runtime, anthologies tend to have segments that focus on a specific piece that’s not necessarily connected to the rest. It doesn’t have to be connected because it’s a self-contained piece. The only connection to the rest of the story is the framing device.

If you’re brain is already working on how to extend this to technology, you’ve probably already equated the framing device to the usual “positioning statement” that’s given at the beginning of a presentation. Here’s the strategy or the vision for how we want to change the world. The individual pieces that the company makes are the parts of the anthology. They are the singular stories that tell the bigger narrative. Or at least they’re supposed to.

In the case of the Twilight Zone, there is no connection aside from Rod Serling telling us about the story. It’s like he’s reading them out of different books. On the opposite side of the spectrum is Pulp Fiction. This is probably the most beloved Quentin Tarantino movie. It’s a tightly-integrated anthology. All three stories are interwoven with each other. Even though they are three separate narratives they share the same characters and setting. Characters from the first story appear in the second and third. It feels like a real connected narrative.

The difference between Pulp Fiction and the Twilight Zone is pretty apparent. So too does the difference between companies that have tightly integrated the story for their individual pieces versus a company that has just put someone in front of the parts to tell you how it should all work together.

Discussion in the Details

When you’re deciding how to tell your product marketing story, ask yourself every once in a while “How does this tie into the big picture?” If it takes you more than ten seconds to answer that question yourself you’re on the road to an anthology series and not a cohesive story. Always refer back to the original statement. Frame your discussion along the lines of the basic premise of your story.

Think of it like writing paragraphs in middle school. Have a main idea and a couple of supporting details that refer back to the main idea. Always make sure you’re referring back to the main idea. If you don’t you need to evaluate what you’re trying to say. If you want a cohesive discussion you have to see the thread that ties everything together.

That’s not to say that every product marketing story needs to be tightly integrated and cohesive across everything. In fact, trying to tie some random piece of technology into the bigger story with a random framing device can feel stilted and out of place. It has to make sense in the narrative. Claiming you have a cohesive strategy for cloud storage is great when you add in telemetry and SD-WAN support. But if you try to pivot to talking about 5G and how it supports your cloud storage you’re not going to be able to tie that into anything without it feeling out of place.

Go back to the basics. Ask yourself what the story is. Don’t try to focus on the pieces. Focus instead on what you want to tell. Some of the best anthologies work because they have different storytellers contributing to the overall piece. If you have a story from a single storytelling you get some exciting integration. But if you have different ideas and visions working together you can come up with some really interesting discussions. Don’t sell your people’s ideas short. Just give them the direction they need to make it work.


Tom’s Take

Before anyone starts filling in the blanks about who the company in question might be, the answer is “all of them”. At some point or another, almost every company I’ve ever seen has failed at telling a good story about their technology. I don’t fault them for it. Marketing is hard. Making deep tech work for normal people is hard to do. I’m not trying to single any one company out. Instead, what I’m saying is that everyone needs to do a better job of telling the story. Focus on what you want to say. Figure out how to make your vision sound more like Infinity War and less like Twilight Zone. The more integrated your message, the less likely people are to focus on the parts they like the best to the detriment of the rest of the story.

The Devil Is In The Licensing

If you don’t already know that I’m a co-host of a great podcast we do at Gestalt IT, here’s a great way to jump in. This episode was a fun one to record and talk about licensing:

Sometimes I have to play the role of the genial host and I don’t get to express my true opinion on things. After all, a good podcast host is really just there to keep the peace and ensure the guests get to say their words, right?

Double Feature

I once said that every random feature in a certain network operating system somehow came from a million-dollar PO that needed to be closed. It reflects my personal opinion that sometimes the things we see in code don’t always reflect reality. But how do you decide what to build if you’re not listening to customers?

It’s a tough gamble to take. You can guess at what people are going to want to include and hope that you get it right. Other times you’re going to goof and put something your code that no one uses. It’s a delicate balance. One of the biggest traps that a company can fall into is waiting for their customers to tell them what they want to see in the next release. Steve Jobs is famous for having said the following:

Some people say, “Give the customers what they want.” But that’s not my approach. Our job is to figure out what they’re going to want before they do. I think Henry Ford once said, “If I’d asked customers what they wanted, they would have told me, ‘A faster horse!'” People don’t know what they want until you show it to them. That’s why I never rely on market research. Our task is to read things that are not yet on the page.

Granted, it’s a bit different when you’re building a cutting edge consumer device. And if you look at the track record of Apple it’s not spotless. But when you’re trying to figure out what features need to be built into an operating system you should probably know what your customers want.

But no choice about including code or features comes without a cost. Even if you have engineers on staff writing code day and night you’re going to incur a work cost. Development is measure in hours and hours equate to dollars1. If you have a team of hundreds working on a single feature you’re going to rack up some pretty significant costs. And including that feature in the base operating system only makes sense if you’re trying to capture market share or address a huge issue your customers have.

But how can you track adoption? Number of downloads of the OS or the program? Not a great measure if it’s something everyone needs to install. If you were trying to track the number of Apple Mail users based on the number of people running iOS on a device you’d be pretty far off the mark. Just because it’s installed doesn’t mean it’s used. So how can we track that feature and recoup some of the development costs at the same time? That’s right! Licensing!

The Double-Edged Sword

Licensing, in and of itself, it’s evil. You have to agree to a license every time you use software. Even if you’re using something with a license that says you can do whatever you want with it. The inherent evil part of the license is when it’s applied in an unfair way.

A friend once told me that a networking vendor had a great idea on how to recoup the costs of developing their software-defined strategy. Instead of charging more to turn the feature on for the whole switch they wanted to charge per flow that used the feature. The rest of the room was speechless. How in the world can you charge for a feature in a switch by the flow? Even with bundling of the licenses you’d incur a significant amount of costs just to operate whatever that was. Amazingly enough the person that suggested it had come from a consumer productivity software background, which per-use licensing was the norm.

The idea is sound. Charge people for what they use. But the application failed. Could you imagine someone charging you per phone call? It’s happened before. Remember when calling cards were a thing? You could pay a few cents a minute to talk. Today? The idea of mobile phones and unlimited voice plans makes the idea of per-use phones antiquated at best.

Another great example of licensing backfiring is when Cisco decided they wanted to start charging a license fee for each different phone type they sold. After all, it should cost more to connect a video phone than it should to connect a regular desk phone, right? After spending years fighting against Device License Units (DLUs) and watching them get tossed to side in favor of modified user licensing because of the rise of software over voice, I realized that this is a game that really never ends. I was the proud owner of an old unlimited data plan from back in the day when the iPhone first came out and my provider wanted to charge you more for the voice minutes instead of the data. Today the data usage is much more valuable to them. Trends change. Devices change. And that means you have to keep your licensing fair and even.

Would you license a firewall per hundred flows? Per VPN connection? Maybe per concurrent MAC address? These are all things that have been done before. I have installed firewalls that could be “upgraded” to more capable units by removing an artificial limit on the number of concurrent users. It was wrong to me but the company made money. It was an easy “fix” to get a few hundred dollars more plus some recurring support revenue. But did it accurately reflect the way that the users operated the device? Not really. It was more about getting extra funding for some other feature or for keeping your business unit in business.

The dark side of licensing comes from greed. Ensuring proper feature adoption or tracking development costs is fine and dandy. But when you charge more just because you can it becomes wrong. Worse yet, when you charge a fortune to keep all but a select few from using your feature set it’s even worse. You can’t expect to feel good about yourself charging a million dollars to license a feature that you really expect only a couple of customers to use. But that’s happened before too. And we’re not even going to get into the argument from the podcast about licensing being tied to the myth of “shareholder value”. I’d need another 2,500 words for that one.


Tom’s Take

Licensing is a necessary evil. We have to have rules and guidelines to use things properly. We also have to have a way to tie development to use. Most modern software is going to charge you for some feature, whether it’s a model of paying once for every major update or a freemium model that lets you pay a regular fee for regular updates. I can’t predict that market any more than I can predict the end of unlimited data plans and DLUs. But I can say that if licensing stops being about keeping software use sane and keeps running down the path of keeping shareholders deliriously rich, you’re going to find out that licensing was the real villain all along.


  1. Or the currency of your region ↩︎

Failure Is Fine, Learning Is Mandatory

“Failure is a harsh teacher because it gives the test first and the lesson afterward.” — Vernon Law

I’m seeing a thread going around on Twitter today that is encouraging people to share their stories of failure in their career. Maybe it was a time they created a security hole in a huge application. Perhaps it was creating a routing loop in a global corporation. Or maybe it was something as simple as getting confused about two mailboxes and deleting the wrong one and realizing your mail platform doesn’t have undelete functionality.

We fail all the time. We try our hardest and whatever happens isn’t what we want. Some of those that fail just give up and assume that juggling isn’t for them or that they can never do a handstand. Others keep persevering through the pain and challenge and eventually succeed because they learn what they need to know in order to complete their tasks. Failure is common.

What is different is how we process the learning. Some people repeat the same mistakes over and over again because they never learn from them. In a professional setting, toggling the wrong switch when you create someone’s new account has a very low learning potential because it doesn’t affect you down the road. If you accidentally check a box that requires them to change their password every week you’re not going to care because it’s not on your account. However, if the person you do that to has some kind of power to make you switch it back or if the option puts your job in jeopardy you’re going to learn very quickly to change your behavior.

Object Failure

Here’s a quick one that illustrates how the motivation to learn from failure sometimes needs to be more than just “oops, I screwed up”. I’ll make it a bullet point list to save time:

  • Installed new phone system for school district
  • Used MGCP as the control protocol
  • Need to solve a PRI caller ID issue at the middle school
  • Gateway is at the high school
  • Need to see all the call in the system
  • Type debug mgcp packet detail in a telnet session
  • A. Telnet. Session.
  • Router locks up tight and crashes
  • Hear receptionist from the other room say, “Did you just hang up on me?”
  • Panic
  • Panic some more
  • Jump in my car and break a couple of laws getting across town to restart router that I’m locked out of
  • Panic a lot in the five minutes it takes to reboot and reassociate with CallManager
  • Swear I will never do that again

Yes, I did the noob CCIE thing of debugging packets on a processing device in production because I underestimated the power of phone calls as well as my own stupidity. I got better!

But I promise that if I’d have done this and it would have shut down one phone call or caused an issue for one small remote site I wouldn’t have leaned a lesson. I might even still be doing that today to look at issues. The key here is that I shut down call processing for the entire school district for 20 minutes at the end of the school day. You have no idea how many elementary school parents call the front office at the end of the day. I know now.

Lessons have more impact with stress. It’s something we see in a lot of situations where we train people about how to behavior in high pressure situations. I once witnessed a car accident right in front of me on a busy highway and it took my brain almost ten seconds to process that I needed to call emergency services (911 in the US) even though I had spent the last four years programming that dial peer into phone systems and dialing it for Calling Line ID verification. I’d practiced calling 911 for years and when I had to do it for real I almost forgot what to do. We have to know how people are going to react under stress. Or at least anticipate how people are going to behave. Which is why I always configured 9.911 as a dial peer.

Lessons Learned

The other important thing about failure is that you have to take stock of what you learn in the post-mortem. Even if it’s just an exercise you do for yourself. As soon as you realize you made a mistake you need to figure out how to learn from it and prevent that problem again. And don’t just say to yourself, “I’m never doing that again!” You need to think about what caused the issue and how you can ingrain the learning process into your brain.

Maybe it’s something simple like creating a command alias to prevent you from making the wrong typo again and deleting a file system. Maybe it’s forcing yourself to read popup dialog boxes as you click through the system to make sure you’re deleting the right file or formatting the right disk. Someone I used to work with would write down the name of the thing he was deleting and hold it up to the screen before he committed the command to be sure they matched. I never asked what brought that about but I’m sure it was a ton of stress.


Tom’s Take

I screw up. More often than even I realize. I try to learn as much as I can when I’m sifting through the ashes. Maybe it’s figuring out how I went wrong. Perhaps it’s learning why the thing I wanted to do didn’t work the way I wanted it to. It could even be as simple as writing down the steps I took to know where I went wrong and sharing that info with a bunch of strangers on the Internet to keep me from making the same mistake again. As long as you learn something you haven’t failed completely. And if you manage to avoid making the exact same mistake again then you haven’t failed at all.

Eventually Secure?

I have a Disney+ account. I have kids and I like Star Wars, so it made sense. I got it all set up the day it came out and started binge watching the Mandalorian. However, in my haste to get things up and running I reused an old password instead of practicing good hygiene. As the titular character might scold me, “This is not the way.” I didn’t think anything about it until I got a notification that someone from New Jersey logged into my account.

I panicked and reset my password like a good security person should have done in the first place. I waited for the usual complaints that people had been logged out of the app and prepared to log everyone in again and figure out how to remove my New Jersey interloper. Imagine my surprise when no one came to ask me to turn Phineas and Ferb back on. Imagine my further surprise when I looked in the app and on the Disney+ website and couldn’t find a way to see which devices were logged in to this account. Nor could I find a way to disconnect a rogue device as I could with Netflix or Hulu.

I later found out that this functionality exists but you have to call the Disney+ support team to make it happen. I also have no doubts that the functionality will eventually come to the app as more and more people are sharing account information so they can binge watch Clone Wars. However, this eventual security planning has me a bit concerned. And that concern extends beyond Mice and Mandalorians.

Minimum Secure Product

If you’re figuring out how to secure your newest application or a new building or even just a new user, you first have to figure out what “secure” looks like. If you have trouble figuring that out, all you need to do is look at your closest competitor. They will usually have a good baseline of the security and accessibility features you should have.

Maybe it’s basic device and user controls like the Disney+ example above. Maybe it’s encryption of your traffic end-to-end, as Zoom learned a couple of weeks ago. Or maybe it’s something as simple as ensuring that you don’t have a hard-coded backdoor password for SSH, like Fortinet remembered earlier this year. The real point is that you can survey the landscape and figure out what you need to do to make your product or app meet a minimum standard.

On the extremely off-chance that you’re developing something new and unique and never-before-seen in the world, you have a different problem. For one, you need to chill on the marketing. Maybe you’re using something in a novel and different way. But unless you’ve developed psychic powers or anti-gravity boosters or maybe teleportation you haven’t come up with anything completely unique. Secondly, you still have some references to draw on. You can look for similar things and use similar security controls.

If your teleport requires a login by a qualified person to operate you should look at login security for other industries that are similar to determine what is appropriate. Maybe it’s like a medical facility where you have two-factor authentication (2FA) with smart cards or tokens as well as passwords or biometrics. Maybe it’s a lockout system with two operators required to engage the mechanism so someone’s arm doesn’t actually get teleported away without the rest of them. Even if your teleport produces massive amounts of logs you should keep them lest someone show up on the other pad with a different color hair than when they left. Those logs may be different from anything ever seen before, but even Airbus knows how to store the flight data from every A380 flight.

Security isn’t a hard problem. It’s a series of challenges that must be overcome. All of them are rooted in common sense and discovery. Sure, you may not know all the problems right now. But you know what they look like in general and you also know what the outcome should look like. Common sense comes into play when you start thinking like a bad actor. If I were able to get into this app, what would I want to do? Maybe I want to sign up for the all-inclusive package and not get a confirmation sent to an account. So put a control in place that makes you confirm that. Sure, it reduces the likelihood that someone is going to sign up for something without realizing what they’ve done. But the side effect is that you also have happier customers because they were stopped from doing something they may not have wanted to do. Your security controls served a double purpose.


Tom’s Take

Ultimately, security should be about preventing bad or unwanted outcomes. Theft, destruction, and impersonation are all undesired outcomes of something. If your platform doesn’t protect against those you are not secure. If your process requires intervention to make those outcomes happen you’re not there yet. Disney+ could have launched with device reports and the ability to force logoff after password change. But the developers were focused on other things. It’s time for developers to learn how to examine what the minimum requirements are to be secure and ensure they’re included in the process along the way. We shouldn’t have to hope that we might one day become eventually secure.

Creating Conspicuously Compelling Content

It’s funny how little things change in the middle of big, world changing experiences. I’ve noticed that my daily blog viewership has gone down, as have many other folks I’ve talked to. The number of people reading has been reduced for some reason. However the number of video views of content on other platforms like Youtube has gone up dramatically. It’s almost like the people that were reading because they wanted to get a quick digest now have the free time to watch a whole video on a topic.

I got on the bandwagon too, recently publishing my first episode of Tomversations this week. I’ve also talked to several friends that are either starting or restarting a podcast. The gold mine for content creation has opened for business. However, I still hear the same refrains about content that I’ve heard for years when I talk about writing:

  • “I don’t have anything to say!”
  • “It’s hard to write things down!”
  • “Isn’t it easier to just talk about stuff?”

These are all valid questions, no matter what medium you’re developing for. But let me give you a roadmap to take those objections, turn them on their heads, and be able to create any kind of content you want to produce. And yes, because you’re reading this instead of watching it, be prepared to write just a little. I promise it will pay off.

Writer’s Clearinghouse

You can’t create without ideas, right? You need some way to jot down all the things you think about. Photographers have a saying that the best camera is the one you have with you. I would say that the best note taking device you own is the one you have with you that you use. I know a lot of people that carry pens and little notebooks, like my favorite ones from Field Notes. They think that having a few pieces of paper in their pocket is enough to get their ideas to spring forth from their forehead like an ethereal Athena. Sadly, that’s not the case. If you don’t use your note taking device often you won’t build a habit of using it when you get an idea.

For example, I take notes in a variety of places. One of them is a program called Drafts. I’ve recently started using it to corral all my random ideas. Thoughts about posts. Story outlines. Scripts for videos. You name it. If it think it, it goes in a draft somewhere. It’s like my digital version of The Jones Grail Diary. It’s not organized, but it doesn’t have to be. Just enough reference for me to remember what I was talking about and the main idea. Sometimes I’ll pull out my phone during conversations to take notes. Those drafts are then synced back to my laptop for perusal and consolidation. Whatever tool your using, make sure you use it as soon as you get the idea. If that poor thought escapes into the nether realm of your brain it’s no good to anyone.

And don’t be afraid to jot down the craziest things. No idea is wasted if it’s on paper somewhere. You never know when you’ll create BGP on napkins. Just make sure you have all those papers or drafts in a place where you check them. If not writing something down is bad, writing it down and forgetting to check in on it is just a little bit better, but still bad.

Outline Everything

People think that when they start a conversation or join a podcast recording that magic is just going to happen. The ideas are going to flow and we’re going to have compelling content. The real world couldn’t get any further from the truth. Ideas spring from nowhere, but they grow very slowly. In order to really build around them, you need to nurture then along with some help. And that help usually takes the form of an outline.

Outlines help you plan out your ideas and support them. Remember how we were all taught to write paragraphs in elementary school? Main ideas followed by two or three supporting sentences. It’s basically and reads like formula written by a fourth grader. Guess what? That’s a perfect outline. When I started writing this post in my head, I started with the main ideas and then wrote down supporting ideas. Now that you’re out of high school grammar class you can build around your paragraphs with more than just a detail or two. You can add anecdotes or data or even pictures. And that makes your content nice and supported.

Outlines also help the thinking process. When I record podcasts I have an outline. The Gestalt IT Rundown happens because Rich researches the stories that we riff on. I can make jokes because I know the stories ahead of time. We work on where to put stories because some are better fodder for jokes than others. That’s the outline process. Podcasts are no different no matter how many guests you have. Maybe it’s a one-on-one episode. There’s an outline of the flow of the episode. It may be very detailed to hit all the points. If it’s a community show or discussion, there may be a loose outline designed to give some guardrails to the content. Even a one-sentence main idea for the topic can be and outline if you keep referring your discussion and arguments back to it.

Savage Writing

I know far too many people that treat their first draft like some kind of sacred relic. This is the best thing I’ve ever produced and it can never change from this form. I will pour my effort into it and that’s all I need.

That’s crap.

First drafts are one step removed from outlines and notes. They’re tying things together. Treat them like sketches and not paintings. Don’t be afraid to rearrange, delete, or outright destroy them. There have been many drafts that have been deleted or radically changed by the time I got to the end of the last paragraph. Likewise, there are times when I realize halfway through a conversation that we need to take things in a different direction. The value of being able to change your mind is that you do it when you need to.

Drafts should be massaged and built up to get to a final product. But don’t be afraid to put them on the shelf and let them sit until the time is right. I have dozens of drafts in my archives waiting for more attention, more research, or better timing to be effective. The ideas are sound. The outlines are good. They just need more than I can give right now. Or maybe the topic isn’t quite ready to be discussed at length. What’s important is that the work I’ve done is already waiting for me when I want to come back to it.

Coming back to your work after the fact is an important thing to try if you feel stuck. I’ve been known to walk away from a draft post or script because I need to get my head out of the wagon rut thinking I was in. Forcing myself to do something else or talk to someone to change my way of thinking has done wonders. Coming back to something with fresh eyes and brain cells often makes a huge difference. You can catch little mistakes or realize there’s a better way to state your argument. The time it takes to change your mind for a few minutes probably would have been wasted on doing nothing anyway.

Just Record.

Okay, you’ve jotted down ideas, built your outline, and written a script or a first draft. What do you do now? Well, like my other famous advice, you need to record your thoughts. Just. Record.

Don’t get caught up in things like perfect lighting or audio balance. Don’t freak out if you stammer or someone drives a garbage truck past your recording studio. Just get the thoughts down. Get a feel for how the flow works. Often, you’ll find that you think of changes on the fly. New ways to word things. New supporting ideas that work better for your discussion. I’ve been known to come up with some really great analogies halfway through an explanation that I would never have been able to think of otherwise. You have to get the content down somewhere.

You can always record again. You can always edit mistakes. You can record the intro last and the ending first. You can fix just about anything in post-production after you get the hang of it. The key is that you’re capturing content. Just like writing or outlining or note taking. It’s happening and the content is being created.


Tom’s Take

Content may not be perfect the first time, but neither was the electric light bulb. It’s only through the process of forming things that we can refine them to something that works. Every creative endeavor is rough around the edges. As time goes on, the wear is less apparent as you focus on the good instead of the bad. The errors are less conspicuous than the content you want to share.

BGP Hell Is Other People

If you configure a newsreader to alert you every time someone hijacks a BGP autonomous system (AS), it will probably go off at least once a week. The most recent one was on the first of April courtesy of Rostelecom. But they’re not the only one. They’re just the latest. The incidences of people redirecting BGP, either by accident or by design, are becoming more and more frequent. And as we rely more and more on things like cloud computing and online applications to do our daily work and live our lives, the impact of these hijacks is becoming more and more critical.

Professional-Grade Protocol

BGP isn’t the oldest thing on the Internet. RFC 1105 is the initial draft of Border Gateway Protocol. The version that we use today, BGP4, is documented in RFC 4271. It’s a protocol that has enjoyed a long history of revisions and a reviled history of making networking engineers’ lives difficult. But why is that? How can a routing protocol be so critical and yet obtuse?

My friend Marko Milivojevic famously stated in his CCIE training career that, “BGP isn’t a routing protocol. It’s a policy engine.” When you look at the decisions of BGP in this light it makes a lot more sense. BGP isn’t necessarily concerns with the minutia of figuring out exactly how to get somewhere. Sure, it has a table of prefixes that it uses to make decisions about where to forward packets. Almost every protocol does this. But BGP is different because it’s so customizable.

Have you ever tried to influence a route in RIP or OSPF? It’s not exactly easy. RIP is almost impossible to manipulate outside of things like route poisoning or just turning off interfaces. Sometimes the simplest things are the most hardened. OSPF gives us a lot more knobs to play with, like interface bandwidth and link delay. We can tweak and twerk those values to our heart’s content to make packets flow a certain direction. But we don’t have a lot of influence outside of a specific area for those values. If you’ve ever had to memorize the minutia of OSPF not-so-stubby-areas, ASBRs, and the different between Type 5 and Type 7 LSAs you know that these topics were all but created for certification exams.

But what about BGP? How can you influence routes in BGP? Oh, man! How much time do you have??? We can manipulate things all sorts of ways!

  • Weight the routes to prefer one over another
  • Set LOCAL_PREFERENCE to pick which route to use in a multiple exit system
  • Configure multi-exit discriminator (MED) values
  • AS Path Prepending to reduce the likelihood of a path being chosen
  • Manipulate the underlying routing protocol to make certain routes look more preferred
  • Just change the router ID to something crazy low to break all the other ties in the system

That’s a lot of knobs! Why on earth would you do that to someone? Because professionals need options.

Optional Awfulness

BGP is one of those curious things that seems to be built without guardrails because it’s never used on accident. You’ve probably seen something similar in the real world whenever a person removes a safety feature or modifies a device to increase performance and remove an annoyance designed to slow them down. It could be anything from wrapping a bandana around a safety switch lockout to keep something running to pulling the trigger guard off a nail gun so you don’t keep hitting it with your fingers. Some professionals believe that safety features aren’t keeping them safe as much as they are slowing them down. Something as simple as removing the safety from a pellet gun can have dire consequences in the name of efficiency.

So, how does this apply to our new favorite policy engine that happens to route packets? Well, it applies quite a bit. There is no system of guardrails that keeps you from making dumb choices. Accidentally paste your own AS into the AS Path? That’s going to be a routing decision that is considered. Make a typo for an AS that doesn’t exist in the routing table? That’s going into the formula, too. Announcing to the entire world you have the best path to an AS somewhere on the other side of the world? BGP is happy to send traffic your way.

BGP assumes that professionals are programming it. Which means it’s not going to try and stop you from shooting off your own foot. And because the number of knobs that are exposed by the engine are large and varied you can spend a lot of time trying to troubleshoot just how half of a cloud provider’s traffic came barreling through your network for the last hour. CCIEs spend a lot of time memorizing BGP path selection because every step matters when trying to figure out why BGP is acting up. Likewise, knowing where the knobs are best utilized means knowing how to influence path selection. AS Path prepending is probably the best example of this. If you want to put that AS number in there a hundred times to influence path selection you go for it. Why? Because it’s something you can do. Or, more explicitly, something you aren’t prohibited from doing.

Which leads to the problem of route hijacking. BGP is going to do what you tell it to do because it assumes you’re not trying to do anything nefarious or stupid when you program it. Like an automation script, BGP is going to do whatever it is instructed to do by the policy engine as quickly as possible. Taking out normal propagation delays, BGP will sync things up within a matter of minutes. Maybe a few hours. Which means it’s not hard to watch a mistake cascade through the Internet. Or, in the case of people that are doing less-than-legal things, to watch the fruits of your labors succeed.

BGP isn’t inherently bad any more than claiming a catwalk without a handrail has an evil intent. Yes, the situation you find yourself in is less-than-ideal. Sure, something bad can happen if you screw up or do something you’re not supposed to. But blaming the protocol or the object or the situation is not going to fix the issue. We really only have two options at this point:

  • Better educate our users and engineers about how to use BGP and ensure that only qualified people are able to work on it
  • Create controls in BGP that limit the ability to use certain knobs and options in order to provide more security and reliability options.

Tom’s Take

I’m a proponent of both of those options. We need to ensure that people have the right training. However, we also need to ensure that nefarious actors are locked out and that we are protected from making dumb mistakes or that our errors aren’t propagated at light speed through the dark corners of the Internet. We can’t fix everything wrong with BGP but it’s the best option we have right now. Hellish though it may be, we have to find a way to make a better combination of the protocol and the people that use it.

SD-WAN and Technical Debt

Back during Networking Field Day 22, I was having a fun conversation with Phil Gervasi (@Network_Phil) and Carl Fugate (@CarlFugate) about SD-WAN and innovation. I mentioned that it was fascinating to see how SD-WAN companies kept innovating but that bigger, more established companies that had bought into SD-WAN seemed to be having issues catching up. As our conversation continued I realized that technical debt plays a huge role in startup culture in all factors, not just with SD-WAN. However, SD-WAN is a great example of technical debt to talk about here.

Any Color You Want In Black

Big companies have investments in supply chains. They have products that are designed in a certain way because it’s the least expensive way to develop the project or it involves using technology developed by the company that gives them a competitive advantage. Think about something like the Cisco Nexus 9000-series switches that launched with Cisco ACI. Every one of them came with the Insieme ASIC that was built to accelerate the policy component of ACI. Whether or not you wanted to use ACI or Insieme in your deployment, you were getting the ASIC in the switch.

Policies like this lead to unintentional constraints in development. Think back five years to Cisco’s IWAN solution. It was very much the precursor to SD-WAN. It was a collection of technologies like Performance Routing (PfR), Application Visibility Control (AVC), Policy Based Routing (PBR), and Network Based Application Recognition (NBAR). If that alphabet soup of acronyms makes you break in hives, you’re not alone. Cisco IWAN was a platform very much market by potential and complexity.

Let’s step back and ask ourselves an important question: “Why?” Why was IWAN so complicated? Why was IWAN hard to deploy? Why did IWAN fail to capture a lot of market share and ride the wave that eventually became SD-WAN? Looking back, a lot of the choices that were made that eventually doomed IWAN can come down to existing technical debt. Cisco is a company that makes design decisions based on what they’ve been doing for a while.

I’m sure that the design criteria for IWAN came down to two points:

  1. It needs to run on IOS.
  2. It needs to be an ISR router.

That doesn’t sound like much. But imagine the constraints you run into with just those two limitations. You have a hardware platform that may not be suited for the kind of work you want to do. Maybe you want to take advantage of x86 chipset acceleration. Too bad. You have to run what’s in the ISR. Which means it could be underpowered. Or incapable of doing things like crypto acceleration for VPNs, which is important for building a mesh of encrypted tunnels. Or maybe you need some flexibility to build a better detection platform for applications. Except you have to use IOS. Which uses NBAR. And anything you write to extend NBAR has to run on their platforms going forward. Which means you need to account for every possible permutation of hardware that IOS runs on. Which is problematic at best.

See how technical debt can creep in from the most simplistic of sources? All we wanted to do was build a platform to connect WANs together easily. Now we’re mired in a years-old hardware choice and an aging software platform that can’t help us do what needs to be done. Is it any wonder why IWAN didn’t succeed in the original form? Or why so many people involved with the first generation of SD-WAN startups were involved with IWAN, even if just tangentially?

Debt-Free Development

Now, let’s look at a startup like CloudGenix, who was a presenter at Networking Field Day 22 and was recently acquired by Palo Alto Networks. They started off on a different path when they founded the startup. They knew what they wanted to accomplish. They had a vision for what would later be called SD-WAN. But instead of shoehorning it into an existing platform, they had the freedom to build what they wanted.

No need to keep the ISR platform? Great. That means you can build on x86 hardware to make your software more universally deployable on a variety of boxes. Speaking of boxes, using commercial off-the-shelf (COTS) equipment means you can buy some very small devices to run the software. You don’t need a system designed to use ATM modules or T1 connections. If all you little system is ever going to use is Ethernet there’s no reason to include expansion at all. Maybe USB for something like a 4G/LTE modem. But those USB ports are baked into the board already.

A little side note here that came from Olivier Huynh Van of Gluware. You know the USB capabilities on a Cisco ISR? Yeah, the ISR chipset didn’t support USB natively. And it’s almost impossible to find USB that isn’t baked into an x86 board today. So Cisco had to add it to the ISR in a way that wasn’t 100% spec-supported. It’s essentially emulated in the OS. Which is why not every USB drive works in an ISR. Take that for what’s it’s worth.

Back to CloudGenix. Okay, so you have a platform you can build on. And you can build software that can run on any x86 device with Ethernet ports and USB devices. That means your software doesn’t need to do complicated things. It also means there are a lot of methods already out there for programming network operating systems for x86 hardware, such as Intel’s Data Plane Development Kit (DPDK). However CloudGenix chose to build their OS, they didn’t need to build everything completely from scratch. Even if they chose to do it there are still a ton of resources out there to help them get started. Which means you don’t have to restart your development every time you need to add a feature.

Also, the focus on building the functions you want into an OS you can bend to your needs means you don’t need to rely on other teams to build pieces of it. You can build your own GUI. You can make it look however you want. You can also make it operate in a manner that is easiest for your customer base. You don’t need to include every knob or button or bell and whistle. You can expose or hide functions as you wish. Don’t want customers to have tons of control over VPN creation or certificate authentication? You don’t need to worry about the GUI team exposing it without your permission. Simple and easy.

One other benefit of developing on platforms without technical debt? It’s easy to port your software from physical to virtual. CloudGenix was already successful in porting their software to run from physical hardware to the cloud thanks to CloudBlades. Could you imagine trying to get the original Cisco IWAN running in a cloud package for AWS or Azure? If those hives aren’t going crazy right now I’m sure you must have nerves or steel.


Tom’s Take

Technical debt is no joke. Every decision you make has consequences. And they may not be apparent for this generation of products. People you may never meet may have to live with your decisions as they try to build their vision. Sometimes you can work with those constraints. But more often than not brilliant people are going to jump ship and do it on their own. Not everyone is going to succeed. But for those that have the vision and drive and turn out something that works the rewards are legion. And that’s more than enough to pay off any debts, technical or not.

The Bane of Backwards Compatibility

I’m a huge fan of video games. I love playing them, especially on my old consoles from my formative years. The original Nintendo consoles were my childhood friends as much as anything else. By the time I graduated from high school, everyone had started moving toward the Sony Playstation. I didn’t end up buying into that ecosystem as I started college. Instead, I just waited for my brother to pick up a new console and give me his old one.

This meant I was always behind the curve on getting to play the latest games. I was fine with that, since the games I wanted to play were on the old console. The new one didn’t have anything that interested me. And by the time the games that I wanted to play did come out it wouldn’t be long until my brother got a new one anyway. But one thing I kept hearing was that the Playstation was backwards compatible with the old generation of games. I could buy a current console and play most of the older games on it. I wondered how they managed to pull that off since Nintendo never did.

When I was older, I did some research into how they managed to build backwards compatibility into the old consoles. I always assumed it was some kind of translation engine or enhanced capabilities. Instead, I found out it was something much less complicated. For the PS2, the same controller chip from the PS1 was used, which ensured backwards compatibility. For the PS3, they essentially built the guts of a PS2 into the main board. It was about as elegant as you could get. However, later in the life of those consoles, system redesigns made them less compatible. Turns out that it isn’t easy to create backwards compatibility when you redesign things to remove the extra hardware you added.

Bringing It Back To The Old School

Cool story, but what does it have to do with enterprise technology? Well, the odds are good that you’re about to fight a backwards compatibility nightmare on two fronts. The first is with WPA3, the newest security protocol from the Wi-Fi Alliance. WPA3 fixes a lot of holes that were present in the ancient WPA2 and includes options to protect public traffic and secure systems from race conditions and key exchange exploits. You’d think it was designed to be more secure and would take a long time to break right? Well, you’d be wrong. That’s because WPA3 was exploited last year thanks to a vulnerability in the WPA3-Transition mode designed to enhance backwards compatibility.

WPA3-Transition Mode is designed to keep people from needing to upgrade their wireless cards and client software in one fell swoop. It can configure a WPA3 SSID with the ability for WPA2 clients to connect to it without all the new enhanced requirements. Practically, it means you don’t have to run two separate SSIDs for all your devices as you move from older to newer. But practical doesn’t cover the fact that security vulnerabilities exist in the transition mechanism. Enterprising attackers can exploit the weaknesses in the transition setup to crack your security.

It’s not unlike the old vulnerabilities in WPA when it used TKIP. TKIP was found to have a vulnerability that allowed for exploiting. People were advised to upgrade to WPA-AES as soon as possible to prevent this. But if you enabled older non-AES capable clients to connect to your SSIDs for compatibility reasons you invalidated all that extra security. Because AES had to operate in TKIP mode to connect the TKIP clients. And because the newer clients were happy to use TKIP over AES you were stuck using a vulnerable mode. The only real solution was to have a WPA-AES SSID to connect to for your newer secure clients and leave a WPA-TKIP SSID active for the clients that had to use it until they could be upgraded.

4Gs for the Price of 5

The second major area where we’re going see issues with backwards compatibility is with 5G networking. We’re hearing about the move to using 5G everywhere. We’ve no doubt heard by now that 5G is going to replace enterprise wireless or change the way we connect to things. Honestly, I’m not surprised someone has tried to make the claim that 5G can make waffles and coffee yet. But 5G is rife with the same backwards compatibility issues present in enterprise wireless too.

5G is an evolution of the 4G standards. Phones issued today are going to have 4G and 5G radios and the base stations are going to mix the radio types to ensure those phones can connect. Just like any new technology, they’re going to maximize the connectivity of the existing infrastructure and hope that it’s enough to keep things running as they build out the new setup. But by running devices with two radios or having a better connection from the older devices, you’re going to set yourself up to have your new protocol inherently insecure thanks to vulnerabilities in the old versions. It’s already projected that governments are going to take advantage of this for a variety of purposes.

We find ourselves in the same boat as we do with WPA3. Because we have to ensure maximum compatibility, we make sacrifices. We keep two different versions running at the same time, which increases complexity. We even mark a lot of necessary security upgrades as optional in order to keep people from refusing to implement them or fall behind because they don’t understand them1.

The biggest failing for me is that we’re pushing for backwards compatibility and performance over security. We’re not willing to make the hard choices to reduce functionality in order to save our privacy and security. We want things to be backwards compatible so we can buy one device today and have it work on everything. We’ll just make the next one more secure. Or the one after that. Until we realize that we’re still running old 802.11 data rates in our newest protocols because no one bothered to remove them. We have to make hard choices sometimes and sacrifice some compatibility in order to ensure that we’re safe and secure with the newer technology.


Tom’s Take

Backwards compatibility is like the worst kind of nostalgia. I want the old thing but I want it on a new thing that runs faster. I want the glowing warmth of my youth but with the convenience of modern technology. It’s like buying an old sports car. Sure, you get all the look and feel of an old powerful engine. You also lose the safety features of the new body along with the comforts you’ve become accustomed to. You have to make a hard choice. Do you keep the old car original and lose out on what you like to get what you want? Or do you create some kind of hybrid that has exactly what you want and need but isn’t what you started with? It’s a tough choice to make. In the world of technology, there’s no right answer. But we need to remember that every compromise we make for performance can lead to compromises in security.


  1. I’m looking at you, OWE ↩︎