Being the Best at Beginning

The other day I was listening to an excellent episode of The Art of Network Engineering talking about technical marketing engineers (TME). The discussion was excellent and there was one line from Pete Lumbis in the episode that stuck with me. He said that one of the things that makes you good as a TME is being an “expert beginner”. That phrase resonates at lot with me.

Fresh Eyes on the Problem

I talked a bit about this last year when I talked about being a beginner and how exciting that it was to start over with something. As I compared that post to the AONE episode I realized that what Pete was talking about was a shift in mindset that gives you the energy and focus to pick things up quickly.

You may have heard the phrase “familiarity breeds contempt”. It’s a common phrase used to describe how we feel less impressed with things the more we learn about then. Our brains are wired to enjoy new things. We love new experiences, going to new places, or even meeting new people. The excitement and rush that we get from something unfamiliar causes our brain to devour things. It’s only once we become familiar with them that we feel the contempt set in.

It’s not always possible to avoid the contempt part of things. Think about something as dreary as your morning commute to work, whether it’s walking down the stairs or driving to the office. I used to joke that my car was practically on auto-pilot most mornings because I knew every bump in the road and every turn by heart. When I would go somewhere new I would have to focus more on road signs or directions.

Back to Basics

The beginner aspect of things is easier to deal with. That’s because we can trick ourselves into seeing something with fresh eyes. On a number of occasions I’ve mentioned my friend and mentor John Pross and his assertion that every upgrade or deployment that happened was like the first time doing it. He never took anything for granted. He always took things step-by-step. While this had the affect of him making sure everything was followed to the letter it also gave him the beginner aspect of looking for ways to improve or discovering new solutions to problems along the way.

Once the contempt or apathy sets in you’re going to get very good at just clicking through the steps to get to the end as fast as possible. If you don’t believe me think about how many times you’ve given directions that involve something like “Just click Next, Next, Next, Next and then you’re done”. Trust me, it sounds funnier when you say it out loud. But it speaks to the fact that we know the dialog boxes so well that we know they aren’t important. But what if they are?

If you want to understand what it feels like to be a beginner again and you’re having a hard time getting yourself in that mindset you should find a beginner and coach them through a task like a setup. Don’t just tell them what to do. Let them figure it out. Answer questions as they come up. Make them explain why they’re doing something. I bet you’ll learn a lot more as you have to help them understand why that configuration line is in there or why you always choose twice the amount of RAM in an instance. Once you see the process through the eyes of a beginner you have to learn it more completely in order to help them understand it.

In some roles, like a TME or a VAR engineer, the ability to be an expert beginner is critical to your job. You have to see a technology for the first time and pick up the basics quickly. I used to tell people that the excitement of being an engineer at a VAR was the variety of problems I’d be called on to solve. One day might be wireless clients. The next could be iSCSI storage arrays. Whatever the case may be I could count on finding myself in a new situation pretty regularly. It kept things exciting and made me realize I had to stay on my toes.

For those that work as product managers or on more specialized teams you need to make sure you’re taking time to approach things as a beginner. The “same old, same old” may not actually be the same any more. That kind of contempt and familiarity leads to the phrase “the way we’ve always done it” and doesn’t force you to challenge the process to understand how to improve it. Sometimes you need to step back and remember that you have to see everything for the first time.


Tom’s Take

Beginners shouldn’t feel like they’re a nuisance. In fact they should be celebrated for the energy and focus they bring to a task or project. For roles like a TME it’s important to bring the same kind of energy to new things. You can learn a lot when you allow your brain to soak up knowledge like a fresh sponge. More importantly, the ability to be a beginner helps you refine your knowledge base more and will ensure that you can explain a concept or process to someone with absolute certainty.

Friday Networking Field Day Thoughts

I’m wrapping up Networking Field Day 30 this week and as is always the case there was a lot of great discussion from both the presenters and the delegates outside of the presentations. It’s one of the reasons why I love doing this job even after almost ten years. I get to meet fun people and have an impact on so many things in the tech industry.

  • Network-as-a-Service is coming. We recorded a roundtable discussion about it and I think the impact that it’s going to have on mid-sized businesses is massive. It’s going to be like cloud. Not just in operational capability. It’s also going to be a huge driver for what you can do with your network in support of applications. The snowflakes may melt under the weight of the cookies we make from the cookie cutter deployments.
  • It feels like a lot of companies are trying to find what’s next. Part of that is coming from the ways that organizations are changing their outlook for what an office should be after the pandemic shutdowns. But still others are realizing they can’t use the same revenue stream for the next five years and hope to survive. This isn’t simply a game of trying to find an adjacent market to move into to drive growth to keep shareholders and investors happy. This is more about finding something that needs to be done because the alternative is no longer having a company.
  • Sadly, it looks as though third party Twitter clients are gone for good. This is beyond irritating to me because of the way that I choose to consume Twitter. I’ve seen a lot of chatter in various comment threads about third parties making money from the good graces of Twitter but the fact is that those programs drive a LOT of the way that power users interact with the system. If the exodus wasn’t already accelerating I would imagine you’re going to see a lot more coming very, very soon.

Tom’s Take

Stay tuned for all the great info from Networking Field Day on the Tech Field Day YouTube channel and don’t forget to thank your networking team today. You may not need to call them today to tell them something is down but trust me they will appreciate you calling just to say you appreciate them.

Controlling Your View of the World

Straw Bales on Hill Landscape, Tuscany, Italy

As I’m writing this it looks like Twitter has made some changes to the way that third-party clients interact with service. My favorite client, Tweetbot, is locked out right now. The situation is still developing but it’s not looking pretty for anyone using anything other than the web interface. While I will definitely miss the way I use Tweetbot I think it’s the kick I needed to move away from Twitter more than before.

A Window on the World

The apps that we use to consume and create content are the way that we view things. Maybe you prefer a webpage over an app or the way that one client displays things over another but your entire view is based on those preferences. If the way you consume your media changes your outlook on it changes too.

I didn’t always use Tweetbot to view Twitter. I tried using the standard app for a long time. It wasn’t until the infamous “Dickbar” incident back in 2011 that I broke away for something that wasn’t so slavishly dependent on ads. The trending topic bar might not have been specifically for ads at the time but the writing on the wall was there. The way I chose to view my content wasn’t compatible with the way that the service wanted to monetize it.

Fast forward to today. Instagram doesn’t show you time sequence posts. Neither does Facebook. Twitter now defaults to an algorithmic timeline. Apps like TikTok are built on their algorithms. It’s all a way to show you things the system thinks you want to see with a sprinkling of ads mixed in whenever they want to show them to you. Could you imagine a TV channel where the ads were just thrown in to the show whenever they wanted you to see one as opposed to more of a standard time? You probably can imagine because that’s what it feels like to watch platforms like Youtube.

Platforms cost money to operate. That fact isn’t lost on me. What is annoying is that trying to change the way I access the platform in order to serve more ads is going to cause more damage in the long run. Given the choice between using the web version of Twitter and just not using it I’m more inclined to the latter. I want a timeline that shows me tweets in chronological order without the need to change to that format every time I open the page. I want to see what I want to see without being forced to view things that don’t interest me. Forcing me to use the web app to see ads is almost as bad as forcing me to follow people that you think are interesting that I have no desire to interact with.

The Only Winning Move

I’m more than a passive participant in services like Twitter. I create content and use them to share it. I participate actively. And I don’t like the way I’m being forced to play.

I started a Mastodon account many years ago the last time Twitter looked like it was making changes. In the past few months I’ve migrated it to a new profile. The Mastodon interface has robust apps and a great way to interact with people. It may not have the numbers that Twitter does right now but it won’t take long for a lot of the best creators to go there instead.

More importantly, choosing an open platform over a monetized nightmare gives me hope that the real value of what we do on the Internet isn’t selling ads. It’s creating valuable things that people enjoy and sharing them with a receptive audience. That may not be the millions of people on Twitter right this minute but those millions of people are about to find out what it feels like when the best reasons to be on a platform are gone.


Tom’s Take

I want to choose how I see the world, not how someone wants me to see it. I want to decide how I share what I’ve made with people and when they see it, not have it placed behind other things a software function decides are more likely to be clicked. If the future of social media is endless ads and trickery designed to make me spend more time fighting through the timeline instead of consuming it then I guess my view of the world is outdated and needs to change. But it will change on my terms.

Problem Replication or Why Do We Need to Break It Again?

There was a tweet the other day that posited that we don’t “need” to replicate problems to solve them. Ultimately the reason for the tweet was that a helpdesk refused to troubleshoot the problem until they could replicate the issue and the tweeter thought that wasn’t right. It made me start thinking about why troubleshooters are so bent on trying to make something happen again before we actually start trying to fix an issue.

The Definition of Insanity

Everyone by now has heard that the definition of insanity is doing the same thing over and over again and expecting a different result. While funny and a bit oversimplified the reality of troubleshooting is that you are trying to make it do something different with the same inputs. Because if you can make it do the same thing over and over again you’re closer to the root cause of the issue.

Root cause is the key to problem solving. If you don’t fix what’s actually wrong you are only dealing with symptoms and not issues. However, you can’t know what’s actually wrong until you can make it happen more than once. That’s because you have to narrow the actual issue down from all of the symptoms. If you do something and get the same output every time then you’re on the right track. However, if you change the inputs and get the same output you haven’t isolated the issue yet. It’s far too common in troubleshooting to change a bunch of things all at once and not realize what actually fixed the problem.

Without proper isolation you’re just throwing darts. As the above comic illustrates sometimes the victory is causing a different error message. That means you’re manipulating the right variables or inputs. It also means you know what knobs need to be turned in order to get closer to the root cause and the solution. So repeatability in this case is key because it means you’re not trying to fix things that aren’t really broken. You’re also narrowing the scope of the fixes by eliminating things that don’t need to be monitored.

Resource Utilization

How long does it take to write code? A couple of hours? A day? Does it take longer if you’re trying to do tests and make sure your changes don’t break anything else? What about shipping that code as part of the DevOps workflow? Or an out-of-band patch? Do you need to wait until the next maintenance window to implement the changes? These are all extremely valuable questions to ask to figure out the impact of changes on your environment.

Now multiply all of those factors by the number of things you tried that didn’t work. The number of times you thought you had the issue solved only to get the same error output. You can see how wasting hours of a programming team working on things can add up to the company’s bottom line quickly. Nothing is truly free or a sunk cost. You’re still paying people to work on something, whether it’s fixing buggy code or writing a new feature for your application. If they’re spending more of their time tracking down bugs that happen infrequently with no clear root cause are they being used to their highest potential?

What happens when that team can’t fix the issue because it’s too hard to cause it again? Do they bring in another team? Pay for support calls? Do you throw up your hands and just live with it? I’ve been involved in all of these situations in the past. I’ve tried to replicate issues to no avail only to be told that they’ll just live with the bug because they can’t afford to have me work on it any longer. I’ve also spent hours trying to replicate issues only to find out later that the message has only ever appeared once and no one knows what was going on when it happened. I might have been working for a VAR at the time but my time was something the company tracked closely. Knowing up front the issue had only ever happened once might have changed the way I worked on the issue.

I can totally understand why people would be upset that a help desk closed a ticket because they couldn’t replicate an issue. However, I also think that prudence should be involved in any structured troubleshooting practice. Having been on the other side of the Severity One all-hands-on-deck emergencies only to find a simple issue or a non-repeatable problem in the past I can say that having that many resources committed to a non-issue is also maddening as a tech and a manager of technical people.


Tom’s Take

Do you need to replicate a problem to troubleshoot it? No. But you also don’t need to follow a recipe to bake a cake. However, it does help immensely if you do because you’re going to get a consistent output that helps you figure out issues if they arise. The purpose of replicating issues when troubleshooting isn’t nefarious or an attempt to close tickets faster. Instead it’s designed to help speed problem isolation and ensure that resources are tasked with the right fixes to the issues instead of just spraying things and hoping one of them gets the thing taken care of.

Making It Work in 2023

We’re back to the first of the year once again. January 1, 2023 is a Sunday which feels somewhat subdued. That stands in contrast to the rest of the year that felt like a rollercoaster always one heartbeat away from careening out of control. As is the tradition, I’ll look at the things I wanted to spend more time working on in 2022:

  • More Analytical Content: I have to honestly give myself a no on this one, at least from a technical perspective. I did spend some time making analytical content for my Tomversations series. However, the real difference in analytical content came from my posts about leadership and more “soft skill” focused ideas. I’ve gotten more comments about those posts than anything in 2022 and I couldn’t be more proud.
  • Saying No to More Things: This is the part where I would insert an animated GIF of someone laughing manically. While I did make strides in telling people that I have way too much going on to take care of one extra thing the reality is that I took on more things that I probably should have. That’s something that I definitely do need to change but the real hard part isn’t saying No. It’s making it stick.
  • Getting In Front of Things: This one actually was one that I had the hardest time with. I was able to work on some of this in my Tech Field Day job but it was my blog that suffered the most. I wanted to start spending more time thinking about post topics earlier in the week so I wasn’t always posting on Fridays. The irony is that toward the end of the year I did manage to get some posts out on other days. I just did it because I was having severe writer’s block and I was late on several posts. In fact, I technically missed a post in 2022 for the first time in twelve years. I think it’s more a reflection on how important it is to keep your eye on the prize as you create content. I don’t have metrics like those on YouTube channels driving me to put out 2-3 videos a week. Instead I really have to make sure I’m aware of what needs to happen to keep the content coming out.

2022 was a year of getting things back to a version of normal but it also was a year of me trying to implement things that didn’t go the way I wanted. That seems to be the challenge for all good intentions. So let’s look at what I want to do in 2023 to keep myself from struggling the way that I did before.

  • Keeping Track of Things: I knew I was in for trouble when I stopped writing things down two weeks into the year. My plans for bullet journaling seemed to evaporate because I tried to make changes that didn’t stick. So I’m resetting it back to Square Zero. I’m writing everything down somewhere and I’m not going to forget it this time. I know where my notes are and I know what I need to do to keep myself on track. The difference between writing it down when I do it and just trying to remember to do it is very, very stark. So rather than thinking I have a good memory and forgetting that I don’t I’m not leaving anything up to chance.
  • Creating Evergreen Content: During my workouts this year I’ve spent more time listening to podcasts like Hidden Brain and Huberman Lab which focus on behavior and the brain. Why? Because I’m fascinated to learn why we think and do the things we do. When you couple that with my outside work in the Wood Badge leadership program I think I’m starting to see the value of creating content more around skills and leadership instead of just talking about the next iteration of wireless or SD-WAN. That doesn’t mean my technical writing is going to go away. It does mean that I’m going to try to sprinkle in more posts about mentoring, leadership, and creating a culture that will help you pay dividends unlike ATM, HD-DVD, and several other forgotten technologies.
  • Ensuring Intentionality: This one feels a bit nebulous but that’s because it’s hard to pin down what being intentional really means to everyone. I’ve used the world a bit more in the latter half of 2022 when discussing certain aspects of my job and it seems to have stuck with others. Intentionality to me means making sure that I’m focused on ensuring outcomes happen. A lack of intentionality is like gathering the ingredients for a cake, assembling them on a table, and then hoping that a cake somehow magically appears. You have to combine the ingredients in the proper amounts and make things happen to create a cake. Even with all the right conditions you still need to do the work to make things happen. And if you think that’s an easy thing to do take a look around in your office and tell me just how many people that you work with that aren’t intentional in what they do.

Tom’s Take

One of the things that I can look back on over the past few years is that my first of the year posts have reflected the challenges that I’ve faced. The shifting landscape of content has forced me to look at what I create. The challenges of a world that went into hibernation for a while have changed the way I look at how I get my work done. Even the growth that I’ve experienced over the years has shifted my thinking. I won’t be the next big YouTube star. My gift is in writing. And focusing on that as my starting point for the year to come is going to help me make things work.

Testing Your Weakest Links as a Chain

You may have heard in the news this week that there was a big issue with Southwest Airlines this holiday season. The issues are myriad and this is going to make for some great case studies for students in the future. However, one thing I wanted to touch on briefly in this whole debacle was the issue of a cascade failure.

The short version is that a weather disruption in the flight schedule became a much bigger problem when the process for rescheduling the flight crews was overwhelmed. Turns out that even after the big computer system upgrades and all the IT work that has gone into putting together a modern airfare booking system that one process was still very manual. The air crew rescheduling department was relatively small in nature and couldn’t keep up with the demands placed on it by the disruptions. It got to the point where Southwest had to reduce their number of flights in order to get the system back to normal.

Worst Case Scenario

I’m not an expert at airline scheduling but I have spent a lot of time planning for disaster recovery. One of the things that we focus on more than anything else is the recovery aspect. The whole purpose of the plan is to get things up and running, right? That requires a look at the big picture of what data needs to be saved and where it needs to be stored and how people are going to do it. In the above example the focus of the airline was getting passengers booked on flights as soon as possible.

However, the details matter just as much as the big picture. If you don’t know the process at every step of the way you’re going to find that the weakest link in the chain is the one that breaks. All the upgrades in the world for remote storage or immutable snapshots won’t matter if someone doesn’t have a key to the data center to turn everything on. Just ask the engineers at Facebook that didn’t realize the door controls for the data center relied on the internal systems that were unreachable during their 2021 outage.

How can you catch these little details? How can you be so sure that everything isn’t going to fall apart because someone forgot the keys to the closet with the disaster recovery binder? The answer, of course, is testing. You’re going to have to test every aspect of the plan from top to bottom. Most everyone will agree that you have to test everything properly to make sure no one problem overwhelms the system. However, that’s where this whole thing falls apart.

Forest for the Trees

If you’re just looking at the individual aspects of your disaster recovery plan in a vacuum you’re going to have a miserable time of it when things go wrong. Unit testing is a popular way to look at the components of the plan to ensure they work without incurring too much cost or too many resources for the testing. However, unit testing alone doesn’t look at the way the details integrate.

That’s where integration testing comes into play. It’s not enough to check the individual pieces. Maybe the computer system is good at rescheduling passengers and balancing the gate assignments. However, if they can’t get on the plane because the system doesn’t think there is a crew due to the way the process interacts with a different area then you don’t have a functional system no matter how great one part of it is.

Disaster recovery tests can be done at the unit level to make sure new modules or processes are solid but you have to make sure you have integrated full tests at least once every six months or so. You have to find the holes in the system caused by the interactions of the details. Sure, the generators might fire up on cue but what if someone is parked in front of the fuel delivery area? What if the keys to the backup cages are on someone’s keychain instead of in a central area? These are questions you want to answer before everyone is running around with their hair on fire.

More importantly, when this happens you need to document all of it. If there is a particular integration that fails you need to write it down and discuss it with your teams. Understand why it happened and put process and procedure in place to cover it. Then make sure that everyone is aware the plan was updated. If people think that something has to be done a certain way because that’s the way it’s always done they’re going to keep doing it that way until they are told differently. Communication is key in any kind of adverse situation.

Lastly, be honest when you’re evaluating these process failures. Don’t try to explain it away or minimize the impact. If someone didn’t do their job then make sure everyone knows what needed to happen and how it failed. If a system doesn’t work properly then analyze the system and fix it. Don’t throw blame where it’s not warranted but don’t explain it away to salve an ego. You need to make the process work and make it work every time so that you don’t run into issues again.


Tom’s Take

Disasters happen. If you’re really lucky you will have something in place to keep the disaster from spiraling out of control. The plural of lucky is good and in order to be good you need to analyze how the process works in concert with every component to make sure there are no weak links. If the chain breaks like it did for Southwest you’ll be very lucky to lose money and customers. If you’re not lucky you’ll lose a lot more.

The Power of Complaining Properly

Recently I’ve started listening to a new podcast all about the brain and behaviors called Hidden Brain. It’s got a lot great content and you should totally check it out. One of the latest episodes deals with complaining and how it can make us less productive and more likely to repeat patterns or shut people out.

Complaining is as old as language. I’m sure as soon as the first person to create communications around spoken words was able to teach another person one of the first things they did was complain about the weather or something they hated. Our mind is built to express itself about things we don’t like, such as bad drivers or silly behaviors at work.

The episode explores the ways that our brain can trap us in cycles of complaining simply for the sake of complaining. It also discusses how we should try to spend more time trying to be productive in how we address complaints. I’ve experienced this a lot in IT as well as in my career after being directly involved in IT and there’s a lot of merit in changing the way we complain about things.

Airing Grievances

Complaining without a suggested solution is just whining.

I’ve always found complaining just for the sake of complaining to be counterproductive. Sure, it might feel awesome to just let it all out and pick apart someone’s decision making process or their personality but that’s not sustainable long term. As in the episode above when you spend your time complaining just for the sake of complaints you eventually fall into a pattern and you can’t break out of it. We all have that one friend or coworker that comes to us and complains about stuff no matter what, right?

In part this happens because we create an agreeable environment for it. That’s not always a bad thing. People sometimes just want to complain. If you’ve ever had to deal with someone getting upset because you didn’t just agree with their complaints you know how that can go. There are those in society that would rather just let it all out without disagreement or challenge.

The opposite side of that situation is when someone is challenging our assumptions or forcing us to see things in a different light. I’m sure that everyone reading this can think of someone they know that will show them another side of the argument or help them understand the path to solving issues instead of just whining about them. This person is someone you may not go to all the time because you realize they’re going to make you confront what’s going on instead of just agreeing with it.

We cultivate both of these kinds of people in our circles. We have those we will commiserate with and those we will seek out for help. So how do we manage to spend more time on fixing issues instead of just falling into the patterns of whining and regressive behavior?

Outcomes Over Opining

The first key to figuring out how to break out of the cycle and focus on making this better is a trick I use with others that only want to complain about things in my presence. They want to tell me everything that’s wrong, or more accurately what I’ve done wrong. So I ask a simple question:

How would you like this situation resolved?

It sounds almost too simple. However, if you think about the above examples you realize there are those that simply want to complain. They may not have a solution in mind. Think of those on social media that just want to air their grievances about a company or a person on a perpetual basis. Are they looking to change the situation? Or would they just prefer to complain? Once you ask the person, or ask yourself, how they want the situation resolved then you’ve moved past complaining to a solution.

Once you’re able to break out of the complaining loop you need to keep the conversation focused on the outcome. It’s easy to slip back into complaining and whining mode when you lose sight of the goal. If the solution is to recognize that things need to improve work on the plan of improvement. Have a goal in mind. Is the solution to have better service in a restaurant? Or to not have something cost so much if it is of inferior quality? By making the outcome the focus you channel the negativity into something that can be positive. One other side effect of the focus on the outcome is that continued complaining will fall on deaf ears and usually shorten the conversation. Even if the person has a solid outcome in mind they’ll lose interest if the sole purpose of the conversation is venting instead of productive work.

Lastly, understand that this is really focused on complaining on a non-personal level. Personal discussions are often not going to have an outcome in mind. Maybe the goal is to just vent. That’s why I usually ask now if I’m serving as emotional support or problem solving. However, in a business environment the goal should be the outcome. Especially if it’s a conflict or a complaint from a team member. The goal should be reducing friction and not just being a sounding board for those that would rather expend energy on the problem and not the solution.


Tom’s Take

I complain, just like any other normal person does. Sometimes my complaints are just ways to get my emotional weight off my shoulders. However I have always subscribed to the idea that I need to have an outcome in mind to fix what is causing my issues. Sometimes that outcome is far outside of my control, such as fixing someone’s personality. Other times it is very much in my control but will require work on my part to make it happen. That’s where I always ask myself how much I want this issue resolved. I make sure I’m ready to invest the energy to make it better before I even start.Odds are good that if I’m complaining I’m talking myself into making it better. To me, that’s the power of a proper complaint.

The Power of Continuing Education on Certifications

I’m about six months away from recertifying my CCIE and even though I could just go Emeritus now I’m working on completing some continuing education at the end of the year to push it out another three years. I am once again very thankful that Cisco has this as an option instead of taking a test over and over again as the only option to renew my certifications.

As I embark on another journey to keep myself current in the networking community, I realize that the flexibility that education credits offer is more important that just passing a test or learning a new skill. Employers should also be thrilled that knowledge workers have the ability to work on other skills and be recognized for them. Because there are two different paths that this can lead to.

To Be The Best

One of the things that most professionals recognize with continuing education is that you can leverage your skills to race through things. If you’re already an expert at something like BGP or spanning tree why not take courses to improve the depth of your knowledge? This is part of the reason why there are a number of double CCIEs that have Routing and Switching and Service Provider. The skill sets have a big overlap which makes the additional study to pass the other relatively painless.

Taking pride in practicing the same skill set over and over again is something we traditionally associate with athletes and other skill positions. It is a very valid way of showing everyone that you truly are an expert at your craft. Knowing every nuance of the protocol or understanding it to a degree not possessed by anyone else is a real accomplishment. The value you gain in troubleshooting situations is unmatched. It’s easy to become the authoritative source on something because you’ve literally studied every piece of material on it and you know it inside out.

The downside of this kind of approach is that you naturally gravitate toward being an expert on exactly one or two things. Like a cake baker you are great at making one specific kind of thing. You may have more than enough work to keep you occupied for years but if the market shifts you may find yourself in trouble. The deep learning method works with technology that doesn’t get superseded quickly. IP routing is here to stay but we also said the same thing about traditional telephony and FORTRAN. Those may still exist in some form today and the experts are still needed to make them work but they aren’t nearly as big as they used to be.

Covering the Rest

The opposite of a deep expert is one that has a wide breadth of knowledge. This is the area where I feel a continuous learning program really shines. That’s because access to knowledge outside of your specific discipline can be hard to come by without help. Having a list of approved courses for a CE portal steers you in a good direction to take advantage of these offerings.

I remember telling people that I knew I was starting to gain on my knowledge and certification journey when I stopped finding the books I needed at the local book store. That’s absolutely true for those that are trying to reach the pinnacle of their specific skill set. However, those basic books are great to jump into an area you may not be familiar with.

You may think that you can spend your time studying and practicing and getting expert skill levels in a few key areas but you also need to realize that things can shift. Networking professionals today also need to understand programming and cloud and many other aspects of enterprise IT. It’s not even a case that knowing how to use those things is just easier. Instead it’s a case of requiring knowledge in those areas to understand how they interact so you can build more complete systems. You might be able to work on technology with a specific skill set but you won’t be able to work on anything new if you don’t know how all the parts work together.

You may not like the idea of studying lots of different areas of knowledge and that’s totally fine. But if you don’t at least understand that some knowledge of other areas is needed you’re going to find yourself opting out of many opportunities to work on things that are going to be important later.


Tom’s Take

You can choose to be the deep expert or the designer with breadth. The important thing is that the choice is yours thanks to the foresight of companies that embrace a model of learning over regurgitation. If you want to pick up new skills and get credit for them you can. If you’d prefer to be the best at a given discipline then the world is your oyster. No matter what you have the ability to make a choice that isn’t studying for a test every couple of years that doesn’t expand your knowledge. To me, the real value of a CE program is how it makes us all better.

ChatGPT and Creating For Yourself

I’m sure you’ve been inundated by posts about ChatGPT over the past couple of weeks. If you managed to avoid it the short version is that there is a new model from OpenAI that can write articles, create poetry, and basically answer your homework. Lots of people are testing it out for things as mundane as writing Amazon reviews or creating configurations for routers.

It’s not a universal hit though. Stack Overflow banned ChatGPT code answers because they’re almost always wrong. My own limited tests show that it can create a lot of words from a prompt that seem to sound correct but feel hollow. Many others have accused the algorithm of scraping content from others on the Internet and sampling it into answers to make it sound accurate but not the best answer to the question.

Are we ready for AI to do our writing for us? Is the era of the novelist or technical writer finished? Should we just hang up our keyboards and call it a day?

Byte-Sized Content

When I was deciding what I wanted to do with my life after college I took the GMAT to see if I could get into grad school for an MBA. I scored well on the exam but not quite to the magical level to get a scholarship. However, one area that I did do surprising well in for myself was the essay writing section. I bought a prep book that had advice for the major sections but spent a lot of time with the writing portion because it was relatively new at the time and many people were having issues with how to write an essay. The real secret is that the essay was graded by a computer, so you just had to follow a formula to succeed:

  1. Write an opening paragraph covering what you’re going to say with three points of discussion.
  2. Write a paragraph about point 1 and provide details to support it.
  3. Report for points 2 & 3
  4. Write a summary paragraph restating what you said in the opening.

That’s it. That’s the formula to win the GMAT writing portion. The computer isn’t looking for insightful poetry or groundbreaking sci-fi world building. It’s been trained to look for structure. Main idea statements, supporting evidence, and conclusions all tick boxes that provide points to pass the section.

If all that sounds terribly boring and formulaic you’re absolutely right. Passing a test of competence isn’t the same and pushing the boundaries of the craft. A poet like e e cummings would have failed because his work has no structure and contains capitalization errors compared to the standards of grammar. Yet no one would deny that he is a master of his craft. Likewise, always following the standards is only important when you want to create things that already exist.

Free Thinking

Tech writing is structured but often involves new ideas that aren’t commonplace. How can you train an algorithm to write about Zero Trust Network Architecture or VR surgery if no examples of that exist yet? Can you successfully tell ChatGPT to write about space exploration through augmented reality if no one has built it yet? Even if you asked would you know what sounded correct from the reply.

Part of the issue comes from content consumption. We read things and assume they are correct. Words were written so they must have been researched and confirmed before being committed to the screen. Therefore we tend to read content in a passive form. We’re not reacting to what we’re seeing but instead internalizing it for future use. That’s fine if we’re reading for fun or not thinking critically about a subject. But for technical skills it is imperative that we’re constantly challenging what’s written to ensure that it’s accurate and useful.

If we only consumed content passively we’d never explore new ideas or create new ways to achieve outcomes. Likewise, if the only content we have is created by algorithm based on existing training and thought patterns we will never evolve past the point we are today. We can’t hope that a machine will have the insight to look beyond the limitations imposed upon it by the bounds of the program. I talked about this over six years ago where I said that machine learning would always give you great answers but true AI would be able to find them where they don’t exist.

That’s my real issue with ChatGPT. It’s great at producing content that is well within the standard deviation of what is expected. It can find answers. It can’t create them. If you ask it how to enter lunar orbit it can tell you. But if you ask it how to create a spacecraft to get to a moon in a different star system it’s going to be stumped. Because that hasn’t been created yet. It can only tell you what it’s seen. We won’t evolve as a species unless we remember that our machines are only as good as the programming we impart to them.


Tom’s Take

ChatGPT and programs like Stable Diffusion are fun. They show how far our technology has come. But they also illustrate the importance that we as creative beings can still have. Programs can only create within their bounds. Real intelligence can break out of the mold and go places that machines can’t dream of. We’ve spent billions of dollars and millions of hours trying to train software to think like a human and we’ve barely scratched the surface. What we need to realize is that while we can write software that can approximate how a human can think we can never replace the ability to create something from nothing.

DPUs Could Change The Network Forever

You wouldn’t think that AWS re:Invent would be a big week for networking, would you? Most of the announcements are focused on everything related to the data center but teasing out the networking specific pieces isn’t as easy. That’s why I found mention of a new-ish protocol in an unrelated article to be fascinating.

In this Register piece about CPUs there’s a mention of the Nitro DPU. More importantly there’s also a reference to something that Amazon has apparently been working on for the last couple of years. It turns out that the world’s largest online bookstore and data center company is looking to get rid of TCP.

Rebuilding Transport

The new protocol was developed in 2020. Referred to as Scalable Reliable Datagram (SRD), it was build to solve specific challenges Amazon was seeing related to performance in their cloud. Amazon decided that TCP had bigger issues for them that they needed to address.

The first was that dropped packets required retransmission. In an environment like the Internet that makes sense. You want to get the data you lost. However, when TCP was developed fifty years ago the amount of data that was lost in transit was tiny compared to the flows of today. Likewise, TCP doesn’t really know how to take advantage of multi path flows. That’s because any packet that arrives out-of-order requires the whole thing to be reassembled in order to be read by the operating system. That makes total sense when you think about the relative power of a computer back in the 70s and 80s. If the CPU is already super busy trying to do other things you don’t want it to have to try to deal with a messy stream of packets too. Halting the flow until it’s reassembled makes sense.

Today, Amazon would rather have the packets arrive on multiple different paths and be reassembled higher up the stack. Instead of forcing the system to stop transmitting the entire flow until the lost or out-of-order pieces are fixed they’d rather get the whole thing delivered and start putting the puzzle together while the I/O interface is processing the next flow. That’s because the size of the flows is creating communications issues between systems. They would rather have slightly higher latency to increase performance.

DPUs Or Bust

If this is so obvious, why has no one rewritten TCP before? Well, we hinted at it when we discussed the fact that TCP is stopping the flow to sort out the mess. The CPU will be fully tasked with reassembling flows with something like SRD because networking communications are constant. If you’re hoping that whatever your successor is will just magically sort things out you’re going to find a system that is quite busy with things that shouldn’t be taking so much time. The perceived gains in performance are going to evaporate.

The key to how SRD will actually work lies not in the protocol but in the way it’s implemented in hardware. SRD only works if you’re using an AWS Nitro DPU. When Amazon says they want the packets to be reassembled “up the stack” they’re really saying the want the DPU to do the work to put the pieces back together before presenting the reassembled packet back to the system. The system itself doesn’t know the packets came in out-of-order. The system doesn’t even know how the packets arrived. It just knows it sent data somewhere else and that it arrived with no errors.

The magic here is the DPU. TCP works for the purpose it was designed to do. If you’re transmitting packets over a wide area in serial fashion and there’s packet loss on the link TCP is the way to go. Amazon SRD only works with Nitro-configured systems in AWS. It’s a given that many servers are now in AWS and more than a few are going to have this additional piece of hardware installed and configured. The value is that having this enabled is going to increase performance. But there’s a cost.

I think back to configuring jumbo Ethernet frames for a storage array that I was configuring back in 2011. Enabling 9000 byte frames between the switch and the array really increased performance. However, if you plugged the array in anywhere else or plugged anything into that port on the switch it broke. Why? Because 9000 byte Ethernet isn’t the standard. It’s a special configuration that only works when explicitly enabled.

Likewise, SRD works well within Amazon. You need to specifically enable it on your servers and those performance gains aren’t going to happen if you need to talk to something that isn’t SRD-enabled or isn’t configured with a Nitro DPU. Because the hard work is being done by the DPU you don’t have to worry about a custom hardware configuration in your application ruining your ability to migrate. However, if you’re relying on a specific performance level with the app to make things happen, like database lookups or writing files to a SAN, you’re not going to be able to move that app to another cloud without incurring a performance penalty.


Tom’s Take

I get that companies like Amazon are heavily invested in Ethernet but want higher performance. I also get that DPUs are going to enable us to do things with systems that we’ve never even really considered before, like getting rid of TCP and optimizing how packets are sent through the network. It’s a grand idea but it not only breaks compatibility but creates a gap in performance expectations as well. We already see this in the wireless space. People want gigabit file transfers with Wi-Fi 6 and are disappointed because their WAN connection isn’t that fast. If Amazon is promising super fast communications between systems users are going to be disappointed when they don’t get that in other clouds or between non-DPU systems. It creates lock-in and sets expectations that can’t be met. The future of networking is going to involve DPUs. But in order to really change the way we do things we need to work with the standards we have and make sure everyone can benefit.