Does Automation Require Reengineering?

During Networking Field Day 33 this week we had a great presentation from Graphiant around their solution. While the presentation was great you should definitely check out the videos linked above, Ali Shaikh said something in one of the sessions that resonated with me quite a bit:

Automation of an existing system doesn’t change the system.

Seems simple, right? It belies a major issue we’re seeing with automation. Making the existing stuff run faster doesn’t actually fix our issues. It just makes them less visible.

Rapid Rattletraps

Most systems don’t work according to plan. They’re an accumulation of years of work that doesn’t always fit well together. For instance, the classic XKCD comic:

When it comes to automation, the idea is that we want to make things run faster and reduce the likelihood of error. What we don’t talk about is how each individual system has its own quirks and may not even be a good candidate for automation at any point. Automation is all about making things work without intervention. It’s also dependent on making sure the process you’re trying to automate is well-documented and repeatable in the first place.

How many times have you seen or heard of someone spending hours trying to script a process that takes about five minutes to do once or even twice a year? The return on time investment in automating something like that doesn’t really make sense, does it? Sure, it’s cool to automate everything but it’s not really useful, especially if the task has changes every time it’s run that requires you to change in the inputs. It’s like building a default query for data that needs to be rewritten every time the query is run.

You’re probably laughing right now but you also have at least one or two things that would fit this bill. Rather than asking if you should be automating this task you should instead be asking why we’re doing it in the first place. Why are we looking to accomplish this goal if it only needs to be done on occasion? Is it something critical like a configuration backup? Or maybe just a sanity check to see that unused switch ports have been disabled or tagged with some kind of security configuration. Are you trying to do the task for safety or security? Or are you doing it for busy work purposes?

Streamlining the System

In all of those cases we have to ask why the existing system exists. That’s because investing time and resources into automating a system can result in a big overrun in budget when you run into unintended side effects or issues that weren’t documented in the first place. Nothing defeats an automation project faster than hitting roadblocks out of nowhere.

If you shouldn’t invest time in automating something that is already there, what should you do instead? How about reengineering the whole process instead? If you occasionally run configuration backups to make sure you have good copies of the devices why not institute change controls or rolling automatic backups? Instead of solving an existing problem with a script why shouldn’t you change the way you do things that might have other hidden benefits? If you’re scripting changes to ports to verify security status why not have a system in place that creates configuration on those ports when they’re configured and require change controls to enable them?

It feels like extra work. It always seems easier to jump in from the bottom up with both feet and work on a problem until you solve it. Top down means you’re changing the way the system does things instead so the problems either disappear or change to something more manageable. The important question to ask is “where are my resources best spent?” If you see your time as a resource to invest in projects are you better served making something existing work slightly faster? Or would it be better for you to take the time to do something in a different, potentially better way?

If you believe your process is optimized as much as possible and just needs to run on its own that makes for an easy conversation. But if you’re thinking you need to change the way you do things this is a great time to make those changes and use your time investment to do things properly this time around. You may have to knock down a few walls to get there but it’s way better than building a house of cards that is just going to collapse faster.


Tom’s Take

I’m a fan of automation. Batch files and scripting and orchestration systems have a big place in the network to reduce error and multiply the capabilities of teams. Automation isn’t a magic solution. It requires investment of time and effort and a return for the stakeholders to see value. That means you may need to approach the problem from a different perspective to understand what really should be done instead of just doing the same old things a little faster. Future you will thank you for reengineering today.

Networking Is Fast Enough

Without looking up the specs, can you tell me the PHY differences between Gigabit Ethernet and 10GbE? How about 40GbE and 800GbE? Other than the numbers being different do you know how things change? Do you honestly care? Likewise for Wi-Fi 6, 6E, and 7. Can you tell me how the spectrum changes affect you or why the QAM changes are so important? Or do you want those technologies simply because the numbers are bigger?

The more time I spend in the networking space the more I realize that we’ve come to a comfortable point with our technology. You could call it a wall but that provides negative connotations to things. Most of our end-user Ethernet connectivity is gigabit. Sure, there are the occasional 10GbE cards for desktop workstations that do lots of heavy lifting for video editing or more specialized workflows like medical imaging. The rest of the world has old fashioned 1000Mb connections based on 802.3z ratified in 1998.

Wireless is similar. You’re probably running on a Wi-Fi 5 (802.11ac) or Wi-Fi 6 (802.11ax) access point right now. If you’re running on 11ac you might even be connected using Wi-Fi 4 (802.11n) if you’re running in 2.4GHz. Those technologies, while not quite as old as GigE, are still prevalent. Wi-Fi 6E isn’t really shipping in quantity right now due to FCC restrictions on outdoor use and Wi-Fi 7 is a twinkle in hardware manufacturers’ eye right now. Why aren’t we clamoring for more, faster, better, stronger all the time?

Speedometers

How fast can your car go? You might say you’ve had it up to 100 mph or above. You might take a look at your speedometer and say that it can go as high as 150 mph. But do you know for sure? Have you really driven it that fast? Or are you guessing? Would you be shocked to learn that even in Germany, where the Autobahn has an effectively unlimited speed limit, that cars are often limited to 155 mph?. Even though the speedometer may go higher the cars are limited through an agreement for safety reasons. Many US vehicles are also speed limited between 110 and 140 mph.

Why are we restricting the speeds for these vehicles? Safety is almost always the primary concern, driven by the desire for insurance companies to limit claims and reduce accidents. However, another good reason is also why the Autobahn has a higher effective speed limit: road conditions. My car may go 100 mph but there are very few roads in my part of the US that I would feel comfortable going that fast on. The Autobahn is a much better road surface for driving fast compared to some of the two-lane highways around here. Even if the limit was higher I would probably drive slower for safety reasons. The roads aren’t built for screaming speeds.

That same analogy applies to networking. Sure, you may have a 10GbE connection to your Mac Mini and you may be moving gigs of files back and forth between machines in your local network. What happens if you need to upload it to Youtube or back it up to cloud storage? Are you going to see those 10GbE speeds? Or are you going to be limited to your ISP’s data rates? The fastest engine can only go as fast the pathways will permit. In essence, that hot little car is speed limited because of the pathway the data takes to the destination.

There’s been a lot of discussion in the space about ever-increasing connectivity from 400GbE to 800GbE and soon even into the terabit range. But most of it is specialized for AI workloads or other massive elephant flows that are delivered via a fabric. I doubt an ISP is going to put in an 800GbE cross connect to increase bandwidth for consumers any time soon. They won’t do it because they don’t need to. No consumer is going to be running quite that fast.

Likewise, increasing speeds on wireless APs to more than gigabit speeds is silly unless you want to run multiple cables or install expensive 10GbE cards that will require new expensive switches. Forgetting Multigig stuff for now you’re not going to be able to plug in a 10GbE AP to an older switch and get the same performance levels. And most companies aren’t making 10GbE campus switches. They’re still making 1GbE devices. Clients aren’t topping out their transfer rates over wireless. And even if they did they aren’t going to be going faster than the cable that plugs the AP into the rest of the network.

Innovation Idling

It’s silly, right? Why can’t we make things go faster?!? We need to use these super fast connections to make everything better. Yet somehow our world works just fine today. We’ve learned to work with the system we have. Streaming movies wouldn’t work on a dial-up connection but adding 10GbE connections to the home won’t make Netflix work any faster than it does today. That’s because the system is optimized to deliver content just fast enough to keep your attention. If the caching servers or the network degrades to the point where you have to buffer your experience is poor. But so long as the client is getting streaming data ahead of you consuming it you never know the difference, right?

Our networks are optimized to deliver data to clients running on 1GbE. Without a massive change in the way that workloads are done in the coming years we’re never going to be faster than that. Our software programs might be more optimized to deliver content within that framework but I wouldn’t expect to see 10GbE become a huge demand in client devices. Frankly, we don’t need that much speed. We don’t need to run flat out all the time. Just like a car engine we’re more comfortable running at a certain safe speed that preserves our safety and the life of the equipment.


Tom’s Take

Be honest with yourself. Do you want 10GbE or Wi-Fi 7 because you actually need the performance? Or do you just want to say you have the latest and greatest? Would you pay extra for a v12 engine in a sports car that you never drive over 80 mph? Just to say you have it? Ironically enough, this is the same issue that cloud migrations face today. We buy more than we need and never use it because we don’t know what our workloads require. Instead, we buy the fastest biggest thing we can afford and complain that something is holding it back. Rather than rushing out to upgrade your Wi-Fi or Ethernet, ask yourself what you need, not what you want. I think you’ll realize the network is fast enough for the foreseeable future.

Changing Diapers, Not Lives

When was the last time you heard a product pitch that included words like paradigm shift or disruptive or even game changing? Odds are good that covers the majority of them. Marketing teams love to sell people on the idea of radically shifting the way that they do something or revolutionizing an industry. How often do you feel that companies make something that accomplishes the goal of their marketing hype? Once a year? Once a decade? Of the things that really have changed the world, did they do it with a big splash? Or was it more of a gradual change?

Repetition and Routine

When children are small they are practically helpless. They need to be fed and held and have their diapers changed. Until they are old enough to move and have the motor functions to feed themselves they require constant care. In fact, potty training is usually one of the last things on that list above. Kids can feed themselves and walk places and still be wearing diapers. It’s just one of those things that we do as parents.

Yet, changing diapers represents a task that we usually have no issue with. Sure it’s not the most glamorous work. But it’s necessary. Children can’t do it themselves. Maybe they can take off a wet or soiled diaper on their own (my kids did on occasion), but they can’t quite put one on. We encourage them to conform to the societal norm of using a bathroom instead of using a disposable diaper.

I use changing diapers as a metaphor for something we do regularly that is thankless but necessary. Kids never thank you for changing their diapers when they get older but it needs to be done. You may not think it’s a life-changing experience at the time but you know it’s one small part of what needs to happen to make them better as people later on. As a company that is trying to change people’s lives with the products you’re selling you often aim toward the sky. You want a utopia of flying cars and automated homes and AI-driven everything. But do your customers want that?

Your customers don’t want self-driving cars. They want to not have to spend their time driving. They don’t want AI-powered dinner ordering. They want to not have to make dinner decisions. Your customers don’t want a magical dashboard that makes automatic configuration changes for them. They want to operate their systems without constant attention to every little detail to keep them from falling apart. They don’t want revolutionary. They want relief.

Aim Small, Miss Small

If your first thought when building a product is “we’re going to change the world!” then you need to stop back because you missed the target. One of smartest things I overheard regarding startups was “Don’t solve a problem. Solve a problem someone has every day.” People are so focused on making an impact a revolutionizing the world they often miss the opportunity to do something that really does change things by simply solving common problems that happen all the time.

When you go back to your vision, think about changing diapers, not lives. Think about solving the problems people have every day. Take network automation, for example. You’re not going to create a paradigm shifting organizational restructuring in a day or a week or even a year. What you can do is automate things like password changes or switch deployments. You can solve that everyday problem so there is more time to work on other things. You can remove errors and create responsiveness where it didn’t exist before. Sure, your Ansible script that provisions a switch isn’t going to get your name etched in stone in Silicon Valley. But it can lead to changes in the organization that create efficiency and make your team happier and more focused on solving other hard problems.

Likewise, if you tell someone your product is going to change their life they will probably laugh at you or shake their head in disbelief. After all, everything promises to change their lives. However, if you tell them your product will solve a specific issue they have then they are very likely to take you up on it. Your target market will identify what you do and respond positively. Rather than trying to boil an ocean with hype you’re providing clear messaging on what you can do and how it can help. People want that clarity over hype.


Tom’s Take

If you try to promise me a life-changing experience with an app or a piece of hardware I’m going to make sure you understand what that means and what it takes. On the other hand, if you come to me with a proposal to change something I dislike doing every day or simplifying it in some way I’m more likely to listen to your pitch. Changing lives is hard. Changing diapers is not fun but it is necessary and repetitive. Focus on the small things and make those easier to do before you take on the rest of the world. Your customers will be happier and you will too.

Mastodon Needs More Brand Support

As much as I want to move over to Mastodon full time, there’s one thing I feel that is massively holding it back. Yes, you can laud the big things about federations and freedom as much as you want. However, one thing I’ve seen hanging out in the fringes of the Fediverse that will ultimately hold Mastodon back is the hostility toward brands.

Welcoming The Crowd

If you’re already up in arms because of that opening, ask yourself why. What is it about a brand that has you upset? Don’t they have the same right to share on the platform as the rest of us? I will admit that not every person on Mastodon has this outward hostility toward companies. However I can also sense this feeling that brands don’t belong.

It reminds me a lot of the thinly veiled distaste for companies that some Linux proponents have. The “get your dirty binary drivers out of my pristine kernel” crowd. The ones that want the brands to bend to their will and only do things the way they want. If you can’t provide us the drivers and software for free with full code support for us to hack as much as we want then we don’t want you around.

Apply that kind of mentality to brands venturing into the Fediverse. Do you want them to share their message? Share links to content or help people join webinars to learn more about the solutions? Or do you only want the interns and social media professionals to be their authentic selves and pretend they aren’t working for a bigger company?

The fact is that in order to get people to come to Mastodon to consume content you’re going to need more than highly motivated people. You’re going to need people that are focused on sharing a message. You’re really going to want those that are focused on outreach instead of just sharing random things. Does that sound a lot like the early days of Twitter to you? Not much broadcast but lots of meaningless status updates.

That’s the biggest part of what’s holding Mastodon back. There’s no content. Yes, there’s a lot of sharing. There’s lots of blog posts or people clipping articles to put them out there for people to read. But it’s scattered and somewhat unsupported. There’s no driving force to get people to click through to sites with deeper information or other things that brands do to support campaigns.


Tom’s Take

You’re going to disagree with me and I don’t blame you one bit. You may not like my idea about getting more brand support on Mastodon but you can’t deny that the platform needs users with experience to grow things. And if you keep up the hostility you’re going to find people choosing to stay on platforms that support them instead of wading into the pool where they feel unwelcome.

Consuming Content the Way You Want

One of the true hidden gems of being a part of a big community is the ability to discuss ideas and see different perspectives. It’s one of the reasons why I enjoy working at Tech Field Day and why I’m lamenting the death spiral of Twitter. My move to Mastodon is picking up steam and I’m slowly replicating the way that I consume content and interact there but it’s very much the same way I felt about Twitter thirteen years ago. There’s promise but it needs work.

As I thought about my journey with social media and discussed it with people in the community I realized that a large part of what has me so frustrated is the way in which my experience has been co-opted into a kind of performative mess. Social media is becoming less about idea exchange and more about broadcast.

Give and Take

When I first started out on Twitter I could post things that were interesting to me. I could craft the way I posted those short updates. Did I want to be factual and dry? Or should I be more humorous and snarky? I crafted my own voice as I shared with others. My community grew organically. People that were interested in what I had to say joined up. Others chose to stick with their own circles. The key is that I was allowed to develop what I wanted to present to those around me.

As time went on I realized that I was an aberration in the grand scheme of Twitter. I made content. I offered opinions and analysis. I was a power user. Twitter wasn’t filled with power users. It was filled with passive consumers of content. Twitter wasn’t overly concerned with enabling features that allowed users like me to have an easier time. Instead, it was focused on delivering content to the passive audience. Content that Twitter determined was either interesting enough to keep users coming back to the service or generated enough revenue to keep the lights on.

That shift happens in pretty much every social platform that I’ve been a part of. Facebook moved from reading through other people’s status updates about their dogs or their lunch and into a parade of short form videos about craft projects or memes about Star Wars. Every interaction with those posts just enhanced the algorithm to show me more of them. Facebook only shoves more of what you see into your face. It doesn’t take what you create and build from there.

The algorithms that run these services now don’t care about you. They don’t facilitate the discussions and information exchange that make us all better. Instead, they feed us mindless interaction. They give us 60-word posts about a topic with vapid insights or any one of a number of endless popcorn videos about “life hacks” or people having accidents or, worse yet, very clever advertising that looks like a random person posting about how amazing a t-shirt is.

Does It Ad Up?

If you’re thinking to yourself this is starting to sound a lot like television advertising, you’re not far off the mark. The explosion of content that has been pushed in front of us is all about the advertising. It’s either brands that are looking to have users buy their product or service or it’s services looking to gain tons of users for other reasons.The advertising dollar rules all now.

This isn’t a new thing. Anyone that tries to tell you that invasive advertising is a modern construct has never opened up a copy of Computer Shopper magazine from the 90s or enjoyed hearing the host of a 60s game show shilling for Lucky Strike cigarettes. Advertising has always been a huge part of the content that we consume.

Modern YouTube videos have pre-roll ads and breaks in the middle for more ads. Podcasts have one or two ad reads, either by the hosts or through a slick, produced read. For a society that hates advertising we sure don’t mind taking money from them when they want to place an ad in the content we’re creating. Yet unless we’re willing to bankroll our own platforms completely we’re stuck with the way that those platforms make money.

This all comes together in an insidious way. The algorithms show us things out of order because they want to grab our attention. The system wants to weave in content we might enjoy along with ads that pay for the platform alongside of the content we actually want to see. Unlike broadcast television, which has specific rules about advertising, these systems can flood us with content that is designed to make us stick around or pay for something that someone wanted us to buy.

At no point in that whole process did we see highlighting of blog posts (unless they were boosted with ads) or bringing conversations to the top of the feed because we’ve interacted with those people. Power users and non-sponsored content creators are a drag on the system. Because they’re not interesting enough to draw in the regular users, unless they’re famous, and they don’t pay the platform to prioritize that content.

As the social network matures and relies less and less on users to create the interactions that sustains the user base it flips the model to be more focused on providing for the brands that pay to keep the lights on and the popcorn-style content that keeps the users hanging around. That’s the ultimate reason why the twilight of social media platforms feels so wasteful. What was once a place to grow and expand your horizons becomes the same mindless drivel that we see on TV. A late-stage social network is practically indistinguishable from what The History Channel has turned into.


Tom’s Take

I want Mastodon to succeed. I want the idea exchange to return. There are many on the platform right now that are hostile to brands because they worry about the inevitable slide into the advertising model. That doesn’t happen because of the brands themselves. The move happens when users grow and the platform needs to keep them around. When the costs of running the infrastructure grow past the ability of the users to support it. Here’s hoping the idea exchange and learning continue to be the primary focus for the time being. At least until the next new things comes along.

Why Do YOU Have To Do It?

One of the things that I’ve seen as a common thread among people in the industry as of late is the subject of burnout. Sure, burnout is a common topic no matter what year we’re in but a lot more of what I’m starting to hear about is self-inflicted burnout. Taking on too many projects, doing more than one job, and even having too many things going on outside of your specific role are all contributors to burnout. How can we keep that from happening?

Atlas and His Burden

For me, one of the biggest reasons why I find myself swimming in frustration is because I am very quick to volunteer to do things. In part it’s because I want to make sure the job is done correctly. In another part it’s because I want to be seen as someone that is always willing to get things done. Add in a dash of people pleasing and you can see how this spirals out of control. I’m sure you’ve even heard that as a career advice at some point. I’ve even railed against it many times on this blog.

How can you overcome the impulse to want to volunteer to do everything? If you’re not in a more senior role it’s going to be hard to tell someone you can’t or won’t do something. As I learned last year from commenters you don’t always have that luxury. If you are in a senior role you also may find yourself quickly volunteering to ensure that the job is done properly. That’s when you need to ask an important question:

Why do I have to do this?

Check your ego at the door and make sure that your Aura of Superiority is suppressed. This isn’t about you being better than the job or task. This is about determining why you are the best person to do this job. Seems easy at first, right? Just explain why this is something you have to do. But when you dig into it things get a little less clear.

Are you the most skilled person at this task in the company? That’s a good reason for you to do it. But could you offer to show someone else how to do the thing instead? Especially if you’re the only one that can do it? Cross training ensures that others know what to do when time is critical. It’s also nice to be able to take a vacation without needing to check your email every ten minutes. Enabling others to do things means you’re not the only phone call every time it needs to be done.

Is this something you’re worried won’t be done correctly if you don’t do it? Why? Is it something very difficult to accomplish? If so, why not have a team work on it? Is this something that you already have an idea of how you want to do it? That’s a recipe for trouble. Because you’ll implement your ideas for the thing and then either get bored or distracted and forget all about it. That leads to others thinking you’ve dropped the ball. It could also lead to people passing you over when you have good ideas because they’re afraid you’re going to take the ball and drop it later. If you think that it won’t be done correctly without your input you should find a way to add your input but not make yourself responsible for the completion of the project.

Are you just taking on the task for the accolades of a job well done? Do you enjoy the feeling of being called out for a successful completion of something? That’s fairly standard. Do you enjoy being chastised when you fail? Does it bother you when you’re called out in front of the team for not delivering something? Again, standard behavior of a normal person. The problem is when the need for the former outweighs the aversion to the latter.

In this excellent Art of Network Engineering episode with Mike Bushong he recounts a story of a manager that pushed back against him when he complained that no one knew how busy he really was. Her response of “everyone just sees you not getting things done” really made him stop and realize that taking the entire world on your shoulders wouldn’t make anything better if you kept failing to deliver.

I could go on and on and belabor the point more but I think you understand why it’s important to ask why the task can’t be reassigned or shared. Rather than just refusing you’re trying to figure out if anyone else should be doing it instead of you. As someone with too many things to do it’s critical you’re able to get those done. Adding more to your plate won’t make anyone’s job any easier.


Tom’s Take

I feel that I will always struggle to keep from taking on too many things at once. It’s not quite a compulsion but it’s also difficult not to want to do something for someone or take on a task that really should be done by someone with more skill or more time. The key for me is to stop and ask myself the question in the title. If I’m not the best person to be doing the job or if there is someone else that I can show so I’m not the only one that knows what to do then I need to do that instead. Sharing knowledge and ensuring others can do the tasks means everyone is involved and you’re not overwhelmed. And that makes for a happier workplace all around.

Friday Networking Field Day Thoughts

I’m wrapping up Networking Field Day 30 this week and as is always the case there was a lot of great discussion from both the presenters and the delegates outside of the presentations. It’s one of the reasons why I love doing this job even after almost ten years. I get to meet fun people and have an impact on so many things in the tech industry.

  • Network-as-a-Service is coming. We recorded a roundtable discussion about it and I think the impact that it’s going to have on mid-sized businesses is massive. It’s going to be like cloud. Not just in operational capability. It’s also going to be a huge driver for what you can do with your network in support of applications. The snowflakes may melt under the weight of the cookies we make from the cookie cutter deployments.
  • It feels like a lot of companies are trying to find what’s next. Part of that is coming from the ways that organizations are changing their outlook for what an office should be after the pandemic shutdowns. But still others are realizing they can’t use the same revenue stream for the next five years and hope to survive. This isn’t simply a game of trying to find an adjacent market to move into to drive growth to keep shareholders and investors happy. This is more about finding something that needs to be done because the alternative is no longer having a company.
  • Sadly, it looks as though third party Twitter clients are gone for good. This is beyond irritating to me because of the way that I choose to consume Twitter. I’ve seen a lot of chatter in various comment threads about third parties making money from the good graces of Twitter but the fact is that those programs drive a LOT of the way that power users interact with the system. If the exodus wasn’t already accelerating I would imagine you’re going to see a lot more coming very, very soon.

Tom’s Take

Stay tuned for all the great info from Networking Field Day on the Tech Field Day YouTube channel and don’t forget to thank your networking team today. You may not need to call them today to tell them something is down but trust me they will appreciate you calling just to say you appreciate them.

Controlling Your View of the World

Straw Bales on Hill Landscape, Tuscany, Italy

As I’m writing this it looks like Twitter has made some changes to the way that third-party clients interact with service. My favorite client, Tweetbot, is locked out right now. The situation is still developing but it’s not looking pretty for anyone using anything other than the web interface. While I will definitely miss the way I use Tweetbot I think it’s the kick I needed to move away from Twitter more than before.

A Window on the World

The apps that we use to consume and create content are the way that we view things. Maybe you prefer a webpage over an app or the way that one client displays things over another but your entire view is based on those preferences. If the way you consume your media changes your outlook on it changes too.

I didn’t always use Tweetbot to view Twitter. I tried using the standard app for a long time. It wasn’t until the infamous “Dickbar” incident back in 2011 that I broke away for something that wasn’t so slavishly dependent on ads. The trending topic bar might not have been specifically for ads at the time but the writing on the wall was there. The way I chose to view my content wasn’t compatible with the way that the service wanted to monetize it.

Fast forward to today. Instagram doesn’t show you time sequence posts. Neither does Facebook. Twitter now defaults to an algorithmic timeline. Apps like TikTok are built on their algorithms. It’s all a way to show you things the system thinks you want to see with a sprinkling of ads mixed in whenever they want to show them to you. Could you imagine a TV channel where the ads were just thrown in to the show whenever they wanted you to see one as opposed to more of a standard time? You probably can imagine because that’s what it feels like to watch platforms like Youtube.

Platforms cost money to operate. That fact isn’t lost on me. What is annoying is that trying to change the way I access the platform in order to serve more ads is going to cause more damage in the long run. Given the choice between using the web version of Twitter and just not using it I’m more inclined to the latter. I want a timeline that shows me tweets in chronological order without the need to change to that format every time I open the page. I want to see what I want to see without being forced to view things that don’t interest me. Forcing me to use the web app to see ads is almost as bad as forcing me to follow people that you think are interesting that I have no desire to interact with.

The Only Winning Move

I’m more than a passive participant in services like Twitter. I create content and use them to share it. I participate actively. And I don’t like the way I’m being forced to play.

I started a Mastodon account many years ago the last time Twitter looked like it was making changes. In the past few months I’ve migrated it to a new profile. The Mastodon interface has robust apps and a great way to interact with people. It may not have the numbers that Twitter does right now but it won’t take long for a lot of the best creators to go there instead.

More importantly, choosing an open platform over a monetized nightmare gives me hope that the real value of what we do on the Internet isn’t selling ads. It’s creating valuable things that people enjoy and sharing them with a receptive audience. That may not be the millions of people on Twitter right this minute but those millions of people are about to find out what it feels like when the best reasons to be on a platform are gone.


Tom’s Take

I want to choose how I see the world, not how someone wants me to see it. I want to decide how I share what I’ve made with people and when they see it, not have it placed behind other things a software function decides are more likely to be clicked. If the future of social media is endless ads and trickery designed to make me spend more time fighting through the timeline instead of consuming it then I guess my view of the world is outdated and needs to change. But it will change on my terms.

Making It Work in 2023

We’re back to the first of the year once again. January 1, 2023 is a Sunday which feels somewhat subdued. That stands in contrast to the rest of the year that felt like a rollercoaster always one heartbeat away from careening out of control. As is the tradition, I’ll look at the things I wanted to spend more time working on in 2022:

  • More Analytical Content: I have to honestly give myself a no on this one, at least from a technical perspective. I did spend some time making analytical content for my Tomversations series. However, the real difference in analytical content came from my posts about leadership and more “soft skill” focused ideas. I’ve gotten more comments about those posts than anything in 2022 and I couldn’t be more proud.
  • Saying No to More Things: This is the part where I would insert an animated GIF of someone laughing manically. While I did make strides in telling people that I have way too much going on to take care of one extra thing the reality is that I took on more things that I probably should have. That’s something that I definitely do need to change but the real hard part isn’t saying No. It’s making it stick.
  • Getting In Front of Things: This one actually was one that I had the hardest time with. I was able to work on some of this in my Tech Field Day job but it was my blog that suffered the most. I wanted to start spending more time thinking about post topics earlier in the week so I wasn’t always posting on Fridays. The irony is that toward the end of the year I did manage to get some posts out on other days. I just did it because I was having severe writer’s block and I was late on several posts. In fact, I technically missed a post in 2022 for the first time in twelve years. I think it’s more a reflection on how important it is to keep your eye on the prize as you create content. I don’t have metrics like those on YouTube channels driving me to put out 2-3 videos a week. Instead I really have to make sure I’m aware of what needs to happen to keep the content coming out.

2022 was a year of getting things back to a version of normal but it also was a year of me trying to implement things that didn’t go the way I wanted. That seems to be the challenge for all good intentions. So let’s look at what I want to do in 2023 to keep myself from struggling the way that I did before.

  • Keeping Track of Things: I knew I was in for trouble when I stopped writing things down two weeks into the year. My plans for bullet journaling seemed to evaporate because I tried to make changes that didn’t stick. So I’m resetting it back to Square Zero. I’m writing everything down somewhere and I’m not going to forget it this time. I know where my notes are and I know what I need to do to keep myself on track. The difference between writing it down when I do it and just trying to remember to do it is very, very stark. So rather than thinking I have a good memory and forgetting that I don’t I’m not leaving anything up to chance.
  • Creating Evergreen Content: During my workouts this year I’ve spent more time listening to podcasts like Hidden Brain and Huberman Lab which focus on behavior and the brain. Why? Because I’m fascinated to learn why we think and do the things we do. When you couple that with my outside work in the Wood Badge leadership program I think I’m starting to see the value of creating content more around skills and leadership instead of just talking about the next iteration of wireless or SD-WAN. That doesn’t mean my technical writing is going to go away. It does mean that I’m going to try to sprinkle in more posts about mentoring, leadership, and creating a culture that will help you pay dividends unlike ATM, HD-DVD, and several other forgotten technologies.
  • Ensuring Intentionality: This one feels a bit nebulous but that’s because it’s hard to pin down what being intentional really means to everyone. I’ve used the world a bit more in the latter half of 2022 when discussing certain aspects of my job and it seems to have stuck with others. Intentionality to me means making sure that I’m focused on ensuring outcomes happen. A lack of intentionality is like gathering the ingredients for a cake, assembling them on a table, and then hoping that a cake somehow magically appears. You have to combine the ingredients in the proper amounts and make things happen to create a cake. Even with all the right conditions you still need to do the work to make things happen. And if you think that’s an easy thing to do take a look around in your office and tell me just how many people that you work with that aren’t intentional in what they do.

Tom’s Take

One of the things that I can look back on over the past few years is that my first of the year posts have reflected the challenges that I’ve faced. The shifting landscape of content has forced me to look at what I create. The challenges of a world that went into hibernation for a while have changed the way I look at how I get my work done. Even the growth that I’ve experienced over the years has shifted my thinking. I won’t be the next big YouTube star. My gift is in writing. And focusing on that as my starting point for the year to come is going to help me make things work.

Testing Your Weakest Links as a Chain

You may have heard in the news this week that there was a big issue with Southwest Airlines this holiday season. The issues are myriad and this is going to make for some great case studies for students in the future. However, one thing I wanted to touch on briefly in this whole debacle was the issue of a cascade failure.

The short version is that a weather disruption in the flight schedule became a much bigger problem when the process for rescheduling the flight crews was overwhelmed. Turns out that even after the big computer system upgrades and all the IT work that has gone into putting together a modern airfare booking system that one process was still very manual. The air crew rescheduling department was relatively small in nature and couldn’t keep up with the demands placed on it by the disruptions. It got to the point where Southwest had to reduce their number of flights in order to get the system back to normal.

Worst Case Scenario

I’m not an expert at airline scheduling but I have spent a lot of time planning for disaster recovery. One of the things that we focus on more than anything else is the recovery aspect. The whole purpose of the plan is to get things up and running, right? That requires a look at the big picture of what data needs to be saved and where it needs to be stored and how people are going to do it. In the above example the focus of the airline was getting passengers booked on flights as soon as possible.

However, the details matter just as much as the big picture. If you don’t know the process at every step of the way you’re going to find that the weakest link in the chain is the one that breaks. All the upgrades in the world for remote storage or immutable snapshots won’t matter if someone doesn’t have a key to the data center to turn everything on. Just ask the engineers at Facebook that didn’t realize the door controls for the data center relied on the internal systems that were unreachable during their 2021 outage.

How can you catch these little details? How can you be so sure that everything isn’t going to fall apart because someone forgot the keys to the closet with the disaster recovery binder? The answer, of course, is testing. You’re going to have to test every aspect of the plan from top to bottom. Most everyone will agree that you have to test everything properly to make sure no one problem overwhelms the system. However, that’s where this whole thing falls apart.

Forest for the Trees

If you’re just looking at the individual aspects of your disaster recovery plan in a vacuum you’re going to have a miserable time of it when things go wrong. Unit testing is a popular way to look at the components of the plan to ensure they work without incurring too much cost or too many resources for the testing. However, unit testing alone doesn’t look at the way the details integrate.

That’s where integration testing comes into play. It’s not enough to check the individual pieces. Maybe the computer system is good at rescheduling passengers and balancing the gate assignments. However, if they can’t get on the plane because the system doesn’t think there is a crew due to the way the process interacts with a different area then you don’t have a functional system no matter how great one part of it is.

Disaster recovery tests can be done at the unit level to make sure new modules or processes are solid but you have to make sure you have integrated full tests at least once every six months or so. You have to find the holes in the system caused by the interactions of the details. Sure, the generators might fire up on cue but what if someone is parked in front of the fuel delivery area? What if the keys to the backup cages are on someone’s keychain instead of in a central area? These are questions you want to answer before everyone is running around with their hair on fire.

More importantly, when this happens you need to document all of it. If there is a particular integration that fails you need to write it down and discuss it with your teams. Understand why it happened and put process and procedure in place to cover it. Then make sure that everyone is aware the plan was updated. If people think that something has to be done a certain way because that’s the way it’s always done they’re going to keep doing it that way until they are told differently. Communication is key in any kind of adverse situation.

Lastly, be honest when you’re evaluating these process failures. Don’t try to explain it away or minimize the impact. If someone didn’t do their job then make sure everyone knows what needed to happen and how it failed. If a system doesn’t work properly then analyze the system and fix it. Don’t throw blame where it’s not warranted but don’t explain it away to salve an ego. You need to make the process work and make it work every time so that you don’t run into issues again.


Tom’s Take

Disasters happen. If you’re really lucky you will have something in place to keep the disaster from spiraling out of control. The plural of lucky is good and in order to be good you need to analyze how the process works in concert with every component to make sure there are no weak links. If the chain breaks like it did for Southwest you’ll be very lucky to lose money and customers. If you’re not lucky you’ll lose a lot more.