About networkingnerd

Tom Hollingsworth, CCIE #29213, is a former network engineer and current organizer for Tech Field Day. Tom has been in the IT industry since 2002, and has been a nerd since he first drew breath.

Pegasus Pisses Me Off

UnicornPegasus

In this week’s episode of the Gestalt IT Rundown, I jumped on my soapbox a bit regarding the latest Pegasus exploit. If you’re not familiar with Pegasus you should catch up with the latest news.

Pegasus is a toolkit designed by NSO Group from Israel. It’s designed for counterterrorism investigations. It’s essentially a piece of malware that can be dropped on a mobile phone through a series of unpatched exploits that allows you to create records of text messages, photos, and phone calls and send them to a location for analysis. On the surface it sounds like a tool that could be used to covertly gather intelligence on someone of interest and ensure that they’re known to law enforcement agencies so they can be stopped in the event of some kind of criminal activity.

Letting the Horses Out

If that’s where Pegasus stopped, I’d probably not care one way or the other. A tool used by law enforcement to figure out how to stop things that are tough to defend against. But because you’re reading this post you know that’s not where it stopped. Pegasus wasn’t merely a tool developed by intelligence agencies for targeted use. If I had to guess, I’d say the groundwork for it was laid when the creators did work in some intelligence capacity. Where things went off the rails was when they no longer did.

I’m sure that all of the development work on the tool that was done for the government they worked for stayed there. however, things like Pegasus evolve all the time. Exploits get patches. Avenues of installation get closed. And some smart targets figure out how to avoid getting caught or even how to detect that they’ve been compromised. That means that work has to continue for this to be effective in the future. And if the government isn’t paying for it who is?

If you guessed interested parties you’d be right! Pegasus is for sale for anyone that wants to buy it. I’m sure there are cursory checks done to ensure that people that aren’t supposed to be using it can’t buy it. But I also know that in those cases a few extra zeros at the end of a wire transfer can work wonders to alleviate those concerns.Whether or not it was supposed to be sold to everyone or just a select group of people it got out.

Here’s where my hackles get raised a bit. The best way to prevent a tool like this from escaping is to never have created it in the first place. Just like a biological or nuclear weapon, the only way to be sure it can never be used is to never have it. Weapons are a temptation. Bombs were built to be dropped. Pegasus was built to be installed somewhere. Sure, the original intentions were pure. This tool was designed to save lives. What happens when the intentions aren’t so pure? What happens when your enemies are terrorist but politicians with different views? You might scoff at the suggestion of using a counterterrorism tool to spy on your ideological opponents, but look around the world today and ask yourself if your opponents are so inclined.

Once Pegasus was more widely available I’m sure it became a very tempting way to eavesdrop on people you wanted to know more about. Journalist getting leaks from someone in your government? Just drop Pegasus on that phone and find out who it is. Annoying activist making the media hate you? Text him the Pegasus installer and dump his phone looking for incriminating evidence to shut him up. Suspect your girlfriend of being unfaithful? Pegasus can tell you for sure! See how quickly we went from “necessary evil to protect the people” to “petty personal reasons”?

The danger of the slippery slope is that once you’re on it you can’t stop. Pegasus may have saved some lives but it has undoubtedly cost many others too. It has been detected as far back as 2014. That means every source that has been compromised or every journalist killed doing their work could have been found out thanks to this tool. That’s an awful lot of unknowns to carry on your shoulders. I’m sure that NSO Group will protest and say that they never knowingly sold it to someone that used it for less-than-honorable purposes. Can they say for sure that their clients never shared it? Or that it was never stolen and used by the very people that it was designed to be deployed against?

Closing the Barn Door

The escalation of digital espionage is only going to increase. In the US we already have political leaders calling on manufacturers and developers to create special backdoors for law enforcement to use to detect criminals and arrest them as needed. This is along the same lines as Pegasus, just formalized and legislated. It’s a terrible idea. If the backdoor is created it will be misused. Count on that. Even if the people that developed it never intended to use it improperly someone without the same moral fortitude will eventually. Oppenheimer and Einstein may have regretted the development of nuclear weapons but you can believe that by 1983 the powers that held onto them weren’t so opposed to using them if the need should arise.

I’m also not so naive as to believe for an instant that the governments of the world are just going to agree to play nice and not developer these tools any longer. They represent a competitive advantage over their opponents and that’s not something they’re going to give up easily. The only thing holding them back is oversight and accountability to the people they protect.

What about commercial entities though? If governments are restrained by the people then businesses are only restrained by their stakeholders and shareholders. And those people only seem to care about making money. So if the best tool to do the thing appears and it can make them a fortune, would they forego they profits to take a stand against categorically evil behavior? Can you say for certain that would always be the case?


Tom’s Take

Governments may not ever stop making these weapons but perhaps it’s time for the private sector to stop. The best ways to keep the barn doors closed so the horses can’t get out is not to build doors in the first place. If you build a tool like Pegasus it will get out. If you sell it, even to the most elite clientele, someone you don’t want to have it will end up with it. It sounds like a pretty optimistic viewpoint for sure. So maybe the other solution is to have them install their tool on their own devices and send the keys to a random person. That way they will know they are being watched and that whomever is watching them can decide when and where to expose the things they don’t want known. And if that doesn’t scare them into no longer developing tools like this then nothing will.

Should We Embrace Points of Failure?

failure-pic-1

There was a tweet making the rounds this last week that gave me pause. Max Clark said that we should embrace single points of failure in the network. His post was an impassioned plea to networking rock stars out there to drop redundancy out of their networks and instead embrace these Single Points of Failure (SPoF). The main points Mr. Clark made boil down to a couple of major statements:

  1. Single-device networks are less complex and easier to manage and troubleshoot. Don’t have multiple devices when an all-in-one works better.
  2. Consumer-grade hardware is cheaper and easier to understand, therefore it’s better. Plus, if you need a backup you can just buy a second one and keep it on the shelf.

I’m sure more networking pros out there are practically bristling at these suggestions. Others may read through the original tweet and think this was a tongue-in-cheek post. Let’s look at the argument logically and understand why this has some merit but is ultimately flawed.

Missing Minutes Matter

I’m going to tackle the second point first. The idea that you can use cheaper gear and have cold standby equipment just sitting on the shelf is one that I’ve heard of many times in the past. Why pay more money to have a hot spare or a redundant device when you can just stock a spare part and swap it out when necessary? If your network design decisions are driven completely by cost then this is the most appealing thing you could probably do.

I once worked with a technology director that insisted that we forgo our usual configuration of RAID-5 with a hot spare drive in the servers we deployed. His logic was that the hot spare drive was spinning without actually doing anything. If something did go wrong it was just as easy to slip the spare drive in by taking it off the shelf and firing it up then instead of running the risk that the drive might fail in the server. His logic seemed reasonable enough but there was one variable that he wasn’t thinking about.

Time is always the deciding factor in redundancy planning. In the world of backup and disaster recovery they use the acronym RTO, which stands for Recovery Time Objective. Essentially, how long do you want your systems to be offline before the data is restored? Can you go days without getting your data back? Or do you need it back up and running within hours or even minutes? For some organizations the RTO could even be measured in mere seconds. Every RTO measurement adds additional complexity and cost.

If you can go days without your data then a less expensive tape solution is best because it is the cheapest per byte stored and lasts forever. If your RTO is minutes or less you need to add hardware that replicates changes or mirrors the data between sites to ensure there is always an available copy somewhere out there. Time is the deciding factor here, just as it is in the redundancy example above.

Can your network tolerate hours of downtime while you swap in a part from off the shelf? Remember that you’re going to need to copy the configuration over to it and ensure it’s back up and running. If it is a consumer-grade device there probably isn’t an easy way to console in and paste the config. Maybe you can upload a file from the web GUI but the odds are pretty good that you’re looking at downtime at least in the half-hour range if not more. If your office can deal with that then Max’s suggestions should work just fine.

For organizations that need to be back up and running in less than hours, you need to have fault tolerance in your network. Redundant paths for traffic or multiple devices to eliminate single points of failure are the only way to ensure that traffic keeps flowing in the event of a hardware failure. Sure, it’s more complicated to troubleshoot. But the time you spend making it work correctly is not time you’re going to spend copying configurations to a cold device while users and stakeholders are yelling at you to get things back online.

All-In-Wonder

Let’s look at the first point here. Single box solutions are better because they are simple to manage and give you everything you could need. Why buy a separate switch, firewall, and access point when you can get them all in one package? This is the small office / branch office model. SD-WAN has even started moving down this path for smaller deployments by pushing all the devices you need into one footprint.

It’s not unlike the TVs you can buy in the big box stores that have DVD players, VHS players, and even some streaming services built in. They’re easy to use because there are no extra wires to plug in and no additional remote controls to lose. Everything works from one central location and it’s simple to manage. The package is a great solution when you need to watch old VHS tapes or DVDs from your collection infrequently.

Of course, most people understand the drawbacks of this model. Those devices can break. They are much harder to repair when they’re all combined. Worse yet, if the DVD player breaks and you need to get it repaired you lose the TV completely during the process instead of just the DVD player. You also can’t upgrade the components individually. Want to trade out that DVD for a Blu-Ray player? You can’t unless you install one on its own. Want to keep those streaming apps up-to-date? Better hope the TV has enough memory to keep current. Event state-of-the-art streaming boxes will eventually be incapable of running the latest version of popular software.

All-in-one devices are best left to the edges of the network. They function well in offices with a dozen or so workers. If something goes bad on the device it’s easier to just swap the whole thing instead of trying to repair the individual parts. That same kind of mentality doesn’t work quite so well in a larger data center. The fact that most of these unified devices don’t take rack mounting ears or fit into a standard data center rack should be a big hint that they aren’t designed for use in a place that keeps the networking pieces off of someone’s desk.


Tom’s Take

I smiled a bit when I read the tweet that started this whole post. I’m sure that the networks that Max has worked on work much better with consumer all-in-one devices. Simple configurations and cold spares are a perfectly acceptable solution for law offices or tag agencies or other places that don’t measure their downtime in thousands of dollars per second. I’m not saying he’s wrong. I’m saying that his solution doesn’t work everywhere. You can’t run the core of an ISP with some SMB switches. You should run your three-person law office with a Cat6500. You need to decide what factors are the most important for you. Don’t embrace failure without thought. Figure out how tolerant you or your customers are of failure and design around it as best you can. Once you can do that you’ll have a much better idea of how to build your network with the fewest points of failure.

VARs See You As Technical Debt

I’ve worked for a Value Added Reseller (VAR) in the past and it was a good run of my career before I started working at Tech Field Day. The market was already changing eight years ago when I got out of the game. With the advent of the pandemic that’s especially true today. Quite a few of my friends say they’re feeling the pressure from their VAR employer to stretch beyond what they’re accustomed to doing or outright being treated in such a way as to be forced out or leaving on their own. They tell me they can’t quite understand why that’s happening. After some thought on the matter I think I know. Because you represent debt they need to retire.

Skill Up

We don’t start our careers knowing everything we need to know to make it. The industry spends a lot of time talking about careers and skill paths and getting your legs under you. Networking people need to learn Cisco or Juniper or whatever configuration language makes the most sense for them. Wireless people need to learn how to do site surveys and configure access points. Server people need to learn operating systems and hypervisors. We start accumulating skills to land jobs to earn money and hopefully learn more important skills to benefit our careers.

Who benefits from that learning though? You certainly do because you gain new ways to further your career. But your VAR gains value as well because they’re selling your skills. The “value added” part is you. When you configure a device or deploy a network or design a system you’re adding value through your skills. That’s what the VAR is charging for. Your skills are their business model. No VAR stays in business just reselling hardware.

Accumulating skills is the name of the game. Those skills lead to new roles and more responsibility. Those new roles lead to more money. Perhaps that means moving on to new companies looking to hire someone that has your particular expertise in an area. That’s a part of the game too, especially for VARs. And that’s where the whole debt mess starts.

Double Down on Debt

Your skills are valuable. They’re also debt. They represent a cost in time, money, and resources. The investment that your VAR makes in you is a calculated return on that debt. If your company primarily deploys Cisco networks then the training you get to install and configure Cisco switches is a return on your VAR being able to hire you out to do that skill. Being able to install and configure Juniper switches isn’t a valuable skill set for them unless they move into a new line of business.

People are no different. We acquire skills that suit us for a time that we may or may not use forever. It’s like riding a bike. We use it a lot when we’re young. We stop using it when we start to drive. We may start again when we need to use a bike for college or for living in a large city or if we pick up cycling or mountain biking as a sport. However, the bike riding skill is always there. It is a sunk cost for us because we acquired it and keep it with us.

For a VAR, your skill is not a sunk cost. It’s a graph of keeping the amount of billable hours you contribute above the line of debt that you create to the company. If you spend 85% of your time installing Cisco switches you are well above the debt line to the company. But if your company stops installing so many switches your value starts to fall as well. It could be that the technology is old and no one is buying it. It could be that companies have shifted the way they do business and need different resources and technology. It could be that a new partnership has created competition inside your organization.

No one wants to the be a last buggy whip manufacturer. VARs thrive on attacking markets that are hot with huge potential for profits. When a skill set becomes a commodity VARs are competing on pricing they can’t always win. That drives them to investigate new markets to offer to the customer base. In order to deliver those new technologies and solutions they need skilled people to install and configure them. The easiest solution is to acquire talent to make that happen. As above, VARs are always willing to pay top dollar to professionals with the specific skill sets they need. Bringing someone in to do that new line of business means they’re producing from Day One and keeping their value above the debt line of their salary.

The other way that VARs compete in these new markets is by training existing professionals on the new technology. Everyone that has ever worked in a VAR knows of the people that get tasked with learning how to deploy new storage systems, new network equipment, and even entirely new solutions that customers are asking for. I know I was that person at my old VAR. If it needed to be learned I was the one to do it first. I jumped in to deploying iSCSI storage, wireless access points, and even VoIP phone systems. Each time I had to spend time learning those new skills and adding them to my existing set. It was a cheaper method in the short term than bringing entirely new talent on board.

Get Out of Town

The friction in the training approach comes when it’s time to value your employees and their skill sets. If I’m getting paid to deploy Cisco switches and now my company wants me to learn how to install Palo Alto firewalls then I’m going to eventually get a raise or a new role to cover this expanded skill set. And rarely, if ever, do employee salaries get adjusted downward to compensate for old skills that are no longer relevant being supplanted by new marketable skills. Suddenly all those technologies I spent so much time learning are technical debt my VAR is paying for.

VARs need to be able to jump into new lines of business in order to survive. And that sometimes means shedding technical debt. If you’re a highly paid employee that earns twice as much as someone that has the specific skill set your VAR needs for a new project then your value to the at this current moment is likely much closer to the negative line of skills versus debt. You may have more experience or more familiarity with the process but that doesn’t translate as well into real value. If it did contractors wouldn’t be as well compensated as they are.

Now your VAR has a choice: keep paying you a lot and investing in their technical debt or bring on someone new that moves more closely with their new lines of business and start the escalator ride all over again. Unless you’re an exceptional employee or you are moved into a management role that usually means you’re let go or encourage to find another role somewhere. Maybe you get lucky and another VAR needs exactly what you offer and they’re willing to pay to get it. No matter what, the VAR is ridding themselves of technical debt. It should be no different than retiring an old laptop or installing new software to do help desk ticketing. But because it’s a person with a life and a family it feels wrong.

Rise Above

Is there an answer to this problem? If there is I don’t think we’ve found it yet. Obviously the solution would be to keep people on staff and pay them what their skill set is worth to the company. But that could entail retraining or readjustment in compensation that people aren’t always willing to do. VARs aren’t going to pay hefty salaries for skills that aren’t making them money. Other VARs may want to pay you for your skills but that’s not always a guarantee, especially if your skill set is extremely specific.

The other possibility is more akin to the contractor system, where you’re only hired for your skills for the period of time that they are needed. In theory that works very well. In practice the challenges of capital asset acquisition and personal benefits make contracting full-time almost as much of a hassle as changing jobs every few years chasing a bigger paycheck or a company that values your skills. There isn’t a clear-cut answer. Part of that reasoning is because the system works just fine the way it is. Why fix it if it’s not broken? It would take a massive shift in IT toward a new paradigm to force the kind of soul searching necessary to change the way VARs handle their staff. Cloud is close. So too is DevOps and programmatic IT. But for the kind of change we’re talking about it’s going to take something even bigger than those two things combined.


Tom’s Take

After reading this I’m sure half of you are scared to death and swear you will never work for a VAR. That’s a bit short-sighted. Remember that they’re a great source of training and experience. Customer networks stay fairly static and only require specific kinds of maintenance from time to time outside of deployments. If you want to hone your skills on a variety of technologies and get very good at troubleshooting then VAR life is absolutely where you need to be. Just remember that you are a resource with a value and a burden. Despite the mantra of being a “family” or other feel-good nonsense you will eventually reach the point of being the uncle that constantly incurs debt for very little return. Every family shuns those kinds of members. Make sure you know your value and how you can contribute. If that’s not possible where you are now then make sure it is wherever you land with whatever skills you need.

Friday Thoughts on Going Back To the Office

EmptyOffice

We’re halfway through 2021 and it’s been going better than last year. Technology seems to be rebounding and we’re seeing companies trying to find ways to get employees to come back into the office. Of course, that is being met head on by the desire to not go back at all and continue to do the job from home that has been done over the past year. Something is going to have to give and I don’t know what that might be.

  • Working from home is comfortable for sure. And the lack of schedule means that people are unknowingly putting in hours beyond what they normally would at the office. At least in the office you can walk away from your desk at the end of the day.
  • Unlimited PTO and flexible work schedules sound great in theory. Except not tracking your PTO hours also means you don’t accrue them. You don’t get paid for time you don’t take off. And a flexible work schedule sounds great in theory but reality says that you’re not likely to get much support if you suddenly decide you want to work noon to 10pm Hawaiian time. Flexible really means “work longer than normal”.
  • The office is filled with tech that you don’t have to maintain. That means when you’re there and the Internet goes down you don’t have to spend your time trying to fix it and keep up with your workload. IT departments have a role to play just like you do. Only their role ends at the office or with confirming that your company-issued equipment is working properly. If it’s your provider or your own personal gear that’s a different story.

It may sound like I’m advocating for you to go back into the office and the nine-to-five grind all over again. That’s not quite the point though. What I’m advocating for is figuring out what’s the best way to get your job done. There are numerous stories in the news about companies asking their workers to return, hearing the refusal, and then making it a mandate to get back to their office to do some part of their job that can’t be done remotely.

Fully Tasked with Partial Credit

The refrain of “I’ve been working remotely for the last year” is a pretty common answer to the call for coming back to the office. But have you been doing 100% of your job remotely? Has every aspect of what you do been able to be completed away from your desk? And if it has, are you doing 100% of the work you were doing in January 2020? I think a lot of the remote work that we’ve seen as of late is a consequence of our jobs needing to be done away from the office but also a reduction in things that have to be done in person. We are able to do our jobs from our house because we’ve reduced or eliminated the things that have to be done face-to-face.

I can say for sure that my role, even having been remote in the past, isn’t the same as it was in early 2020. I used to be on airplanes at least twice a month. I’m finally getting back on one for the first time in over a year next week. The idea of almost foreign to me at this point. And it’s because we knew that there were things that were going to need to change at work due to our inability to do them in person. So while I can say that I can do my job entirely from my house right now it’s only because the part of my job that requires me to get on an airplane all the time hasn’t fully come back into force yet.

This rings even more true for companies that have specific in-person needs. Apple is making news because they’re still pushing to have their employees come back to Cupertino where necessary. That may sound draconian to some until you remember that there is a lot work work on hardware prototypes and development that happens in that building. Those aren’t really things you can do at home. And given how tightly Apple holds that information there’s no way they’re going to allow it to be outside their walls unless absolutely necessary. I don’t know what the right answer is for Apple or for hardware companies in general but the extremes of both sides aren’t likely to get their way entirely.

Compromised Compromises

Several of my friends have remarked that they hate the phrase “new normal” when referring to how society has changed over the past year. The idea that the things we’re doing are going to be permanent parts of our lives from here on out. Yet, for all the grousing about wearing masks or supply shortages or lockdowns when the situation benefits us we’re happy to make it permanent.

The working from home mandates we’ve seen should be examined just like those other measures that aren’t “normal”. It was an emergency measure designed to keep the doors open as long as possible until we could pull through everything going on. Now that it’s time to look at those decisions again people are chafing because this is the one thing they actually like out of the whole pandemic response.

Compromise doesn’t work like that. You don’t get to pick and choose the things you get to have your way on. Instead, you need to figure out what makes the most sense and implement the things that are best for those all around. If that means going back into the office two days a week to do things that can only be accomplished there then maybe that’s what needs to happen. Granted there are still ways to find common ground and negotiate. Maybe you can work from home every other Friday. Or you can adjust your schedule in other ways. But holding out hope that the situation will continue to benefit you as it is right now without any form of further compromise isn’t a likely scenario.


Tom’s Take

I know it sounds a lot like “doom and gloom” for those that want to continue to work from home all the time. As someone that has been doing it for a while I don’t know if I could ever go back into an office full-time. But I also know that when the time comes soon for me to get back to my “office” on an airplane that it’s going to need to happen. Because we can’t get back to the old normal without getting back to the way things were done before. There very well could be a paradigm shift on the horizon for working in offices and how our jobs can be changed to not require in-person work. But I don’t think we’re going to see that happen directly after what we’ve all experienced. That road has more twists and turns yet to come whether it’s headed back to the office or all the way home.

Putting the FUN Back in Productivity

It’s not a secret that it’s hard to get stuff done. Procrastination is practically a super power for me. I’ve tried so many methods and systems to keep myself on track over the years that I should probably start a review site. Sadly, the battle of my executive function being on constant vacation and the inability to get organized saps a lot of my ability to execute. It’s gotten to the point where I’ve finally realized that I need to start tricking my brain into getting things done.

Any reputable researcher will tell you that dealing with neurodivergent behaviors like ADHD is all about understanding the reasons why you do the things you do. I know what needs to be done. I just don’t want to do it. Worse yet, anything that I can do to avoid working on something is going to capture my attention because I’d rather be doing something unproductive as opposed to something I don’t like. This can manifest itself in strange ways like preferring to do the dishes instead of writing a blog post or mowing the yard instead of practicing a presentation.

Not DisFUNctional

It’s taken me a while but I’ve finally come up with a system that makes it easier to get me into a rhythm to get things done. And because you wouldn’t remember it unless I made it spell out some memorable word, we’re going to call it the FUN System. Because more than three points would likely have gotten lost anyway.

F – Fake It! – It’s going to sound silly but the first step in convincing yourself to do something is often to lie to yourself about how much better it will be when you get it done. Your brain has convinced itself that this is bad and you shouldn’t be doing it. So in order to get it done you’re going to have to convince it otherwise.

We do this all the time to others. Telling kids that veggies taste good. Telling our friends that they should do something for us so they feel better. Selling pretty much anything to anyone. It’s all about convincing someone skeptical to do something they don’t want to do. Your brain is no different. You need to convince yourself to get the thing done. Maybe you promise yourself a reward or some extra downtime or something that just gets you moving. You don’t even have to keep the promise. The key is to use it to overcome the objections your brain has already but up. Fake it however you need to in order to make something happen.

U -Understand It – This one is especially powerful for me. I love learning. Like a lot. Enough that I can often convince myself to get a bigger task accomplished more quickly by learning about it. Understanding the details or the process or figuring out how to make it all work. I binge watch documentaries on Youtube and enjoy reading up on random things to learn more about how they work or why they are the way they are.

This extends to things beyond emails and simple tasks for me. Cooking was something that was easier to accomplish and do more often when I learned how it all works together. Why 350 degrees is the magic baking temperature, for example. Or how different spices can create different styles of flavors. It’s all about learning the ins-and-outs of what you’re trying to do.

The key here is not to fall down the hole of learning more about what you’re trying to do than actually doing it. It’s very easy to get paralyzed by over learning and just sitting there going over the details again and again instead of putting them into practice. Using the above example you may have to tell yourself you can come back to the investigation after you’ve tried it once or twice. Ensure that you use the desire to learn as the driver for getting something accomplished before you procrastinate your day away.

N – Next On The List – The third way I tell myself to get things done is to move them down on the list behind an easy task. It’s a cruel trick that relies on momentum. I tell myself that I got the little easy thing done so I might as well tackle the bigger thing. And it works more often than you might think.

The brain only needs a little dopamine from a sense of accomplishment to keep going. It’s the idea that you’re being productive. So if you need to write something long then put it after a short response email. If you’re dreading a phone call then do it after you’ve tidied your desk or taken out the trash. Doing something small will help you get prepared for the big task and ensure that you can carry forward that little extra push to get through it. As a bonus, the sense of accomplishment from that extra big task will carry forward to a couple others! It’s a like a productivity feedback loop.


Tom’s Take

The usual disclaimers apply here. This is my method and it may not work for you. You have to learn how your brain works and find ways to keep it moving and working. There are other things that help create the sense of accomplishment, like routine or the enjoyment of results. But in the long run the key is finding a way to get your brain out of the funk of not wanting to do stuff. My FUN System helps me and maybe it will help you too. Try it out if you’re struggling and use it as a basis to make your own fun.

Don’t OutSMART Your Goals

290C1DEE-0D66-432D-839A-C3C4B79B5F6D

I read a piece on LifeHacker yesterday that made me shake my head a bit. I’m sure the title SMART Goals Are Overrated was designed to get people to click on it, so from that perspective it succeeded. Wading into the discourse there was an outline of how SMART goals were originally designed for managers to give tasks to employees and how SMART doesn’t fit every goal you might want to set, especially personal aspirational ones. Since I have a lot of experience with using SMART goals both for myself and for others I wanted to give some perspective on why SMART may not be the best way to go for everything but you’re a fool if you don’t at least use it as a measuring tool.

SMRT, Eh?

As a recap, SMART is an acronym for the five key things you need to apply to your goal:

  • S – Specific (what are you going to do)
  • M – Measurable (how will you know when you’ve succeeded)
  • A – Attainable or Assignable (can you or the person you’ve selected do this thing)
  • R – Relevant or Relatable (is this goal appropriate for me or for the person doing it)
  • T – Timely or Time-Based (when are you going to accomplish this goal)

Aside from the obvious reason that the originators really wanted their system to spell “smart”, what does all this mean? Well, when we teach SMART goals at Wood Badge, we ask people to envision something they want to do in the next year. Maybe it’s taking a vacation. Or perhaps it’s another project they want to get done around the house. Once they’ve picked it out, we ask them to think about how they’re going to accomplish it. This second exercise is where the application of the ideas behind SMART goals comes into play.

In an enterprise IT sense we don’t do projects with no planning. At least, I really hope we don’t. We need to understand what we’re doing and why and how we’re going to get it done and when. We already have those constraints put in place when we begin the process. SMART just formalizes them into something memorable. Take an IDC switch upgrade for example:

  • Specific – We are going to upgrade the switches in the gym IDF
  • Measurable – We’re done when the switches are installed, cabled, and configured properly
  • Attainable – This is easy to accomplish and the team has done many before
  • Relevant – The networking team is doing the work, not the storage team or the accounting department. Relevant to our needs because we have more bandwidth in the gym during basketball games and we need to increase the amount of concurrent users and devices
  • Timely – We’re doing the upgrade next Friday when everyone is out of school, which is two weeks before basketball season starts to make sure we have everything ready to go with minimal disruptions. The switches are scheduled to arrive tomorrow.

See? SMART helps us plan the whole thing. Specific keeps us from setting goals like “make things faster” and forces us to be very specific. That goes hand-in-hand with Measurable, which also prevents scope creep. We’re done when we’ve met the measurable case. Setting more measurable things will help your projects work much better.

Attainable just means we’re not setting goals we can’t reach. Switching out one IDC at a time is better than trying to reconfigure the whole network in a weekend. Having been roped into unattainable projects before I really wish more of them had this condition figured out up front. Relevant helps answer why we need it or who is relates to. The accounting department may want the fastest access to the data center or the cloud but if they want us to pay thousands of dollars a month for a circuit only they can use it’s going to be hard to meet the Relevant section of the goal. Timely gives you a date to shoot for for completion. That keeps your project from sitting on the “in process” part of your kanban board until the end of time itself.

Don’t Dumb It Down

LifeHacker’s writer, Beth Skwarecki, says that SMART is deceptive because it creates a bait-and-switch mentality of setting pass-fail goals with a deadline. There’s no inherent motivation to get things done and no reason to set goals that require you to stretch your limits because you don’t want to fail. Looking at goal setting in a vacuum would validate her reasoning. However, looking at SMART as the only source of input into the goal setting process is also setting yourself up for failure.

It’s true that SMART encourages you to set deadlines and spell out what you’re doing. That’s because many people struggle with the process of actually defining goals. Like vacation planning they have the big picture of sitting on the beach clearly in mind. They stumble when it comes to booking hotels and rental cars and when to buy the airline tickets and how they’re going to get to the beach and what they need to bring when they get there and so many other things not even on their radar. SMART gives them a framework for figuring out how to make it all work.

SMART isn’t a motivator. It doesn’t make you want to do something. Instead, it gives you a way to measure progress or force yourself to understand when things need to happen. In the article, Beth says that it’s bad if you set a time goal for yourself and then you procrastinate until the week before because there is no inherent drive to work on things in a timely manner. I’d argue that has nothing to do with the SMART framework. Sure, you set yourself a goal to be finished. But we do that all the time.

We want to be able to run a 5k race by the time of the race in the fall. We want to go to Disneyland on our vacation in July. We want to buy a house before we turn 30. All of these goals have a Timely component. Maybe you don’t have a Gantt chart breaking down every minute of the planning process yet. That doesn’t mean putting a time on it doesn’t help you do things better. When I was working on my SMART goal project back in 2017 I had a whiteboard on my desk with deadlines and checkpoints to make sure I was getting things done. The motivation to finish on time came from me setting smaller, attainable goals and not big red circles on the calendar looming on the horizon.

The last thing I’ll say about the article is that SMART goals aren’t supposed to push you to challenge yourself. Beth says that SMART encourages you to do things that are attainable so you don’t fail. I’d argue that the purpose of SMART is to help you set attainable goals and then help you reflect on what you could be doing better or more often. Yes, everyone wants to succeed as often as possible. Constant failure is discouraging. You also need to make sure you aren’t just setting targets to knock down for the sake of knocking them over.

Goals that are set without a check-in aren’t really helping you. Projects with SMART goals should be living documents that get updated frequently. Are you sailing through your running goals? Time to reset your yardstick and stretch yourself a bit. Run a faster time or go for a longer distance. Are you having struggles with your project because things aren’t coming together? Sit down and be honest with yourself and figure out how to make the most out of what you have. Maybe it’s not replacing every AP in the office but just the ones in the employee areas in the main building. If you’re not adjusting your goals along the way based on the feedback you get from the process then you’re going to fall into the trap of making things too easy to fail or too hard to succeed.


Tom’s Take

I’m a big fan of using the right tools for the job. Don’t use screwdrivers as chisels. Don’t use a flamethrower to light cigars. And don’t forget that you can find other ways to make things work for you. SMART isn’t the superior system for every situation out there. There are times when it’s maddening and doesn’t properly fit. However, running your projects and goals through the SMART filter will usually help you identify where you need to tighten up language or timelines. It certainly can’t hurt. And if it’s not working at all then try to find a better way to make it work for you. Use a tool or framework instead of just thinking you’ll do it your own way. That’s the kind of thinking that leads smart people into making dumb decisions.

A Decade of CCIE Certification

I was notified this week that I’m eligible for the 10-year CCIE plaque. Which means that it’s been a decade since I walked out of Cisco’s Building C in San Jose with a new number and a different outlook on my networking career. The cliche is that “so many things have changed” since that day and it’s absolutely accurate because the only constant in life is change.

Labbing On the Road

I think the first thing that makes me think about the passage of time since my certification is the fact that the lab where I took the exam no longer exists. Building C was sold to the company that owns and operates the San Francisco 49ers stadium just down Tasman drive from the old letter buildings. Those real estate locations were much more valuable to the NFL than to Cisco. I can’t even really go and visit my old stomping grounds any more because the buildings were gutted, renovated, and offered to other operations that aren’t from Cisco.

Now, you don’t even go to San Jose or RTP for the lab. Three years ago the labs in the US moved to Richardson, TX. The central aspect of the location is pretty appealing when you think about it. A part of me wishes I would have had the opportunity to take the lab there since I wouldn’t have to jump on a plane and burn three days of my work schedule. The costs of my lab attempts would have been a lot less if I only had to drive down for one night in a hotel and got to come back and sleep in my bed that same night. I realize that it’s equally inconvenient for people to need to fly to the middle of the country when they used to be closer to the lab when it was on either coast. However, real estate in RTP and San Jose is beyond crazy when it comes to price. Moving the lab to somewhere more reasonable means Cisco is getting value out of their buildings elsewhere.

The mobile lab is another aspect of the changes in the CCIE certification program that are a welcome change. By putting the lab on the road and giving people in countries far away from a lab location the opportunity to get certified the program can continue to be relevant. This is due in large part to the changes in the lab that allow a large part of it to be virtualized or operated remotely from a rack located somewhere else. I remember starting my lab studies and thinking to myself that the rack that I was working on was just across the room. Not that there was much that I could do about it. The idea that there could be something going on that was just out of my reach was an itch I had to get over. Today, you would never even start to believe that you had a hardware issue in your lab because of the streamlining of the process. That can only happen when you optimize your offerings to the point where you can just virtualize the whole thing.

The Next Ten Years

Right now, I still have a year to go on my certification before I have to make the decision to keep it current or go to Emeritus retirement. My role on the CCIE Advisory council doesn’t matter either way. I’m likely going to just go Emeritus when the opportunity presents itself because I don’t use those lab skills every day. I’m not configuring BGP filter lists and port channels like I used to. The technical skills that I honed in Building C serve me more now to understand technology at an architecture level. I can see how people are using tools to solve problems and offer commentary when they are making poor decisions or when a better protocol exists.

The CCIE itself is still a very valuable certification to hold and study for. IT certification on the whole has been trending away from being the gold standard for hiring. Cloud and DevOps focus more on skills instead of papers hanging on a wall. However, operations teams still need ways to differentiate their people. If nothing else the CCIE is a great forcing function for you to figure out how deeply into networking you really want to get. It’s not enough to be curious about BGP or Frame Relay and traffic shaping QoS. You have to understand it at a level that would bore most others to tears. If you’re not prepared to know the minutia of a protocol the way that some people memorize batting averages or random movie trivia than you might not be up for this particular challenge.

The CCIE also isn’t going away any time soon. I remarked to someone the other day that the CCIE is a technology bellweather. I can remember the clamor to introduce the “new” SDN changes into the program so many years ago. I also chuckle when I think about the CCIE OpenFlow that more than a couple of people proposed. The certification program exists to refine and highlight the technology solutions that people are using today. It’s not a sneak peak at things that might be important later on in life. Think about how long it took for them to remove ISDN, ATM, and even frame relay from the test. And even frame relay was debated heavily because more than a few claimed they still used it in production.

The CCIE is a testament to the way that people study for and build networks at a high level. It’s not a cool badge to keep on your list like a hunting trophy. It’s a testament to the commitment that it takes to attain something like that. The JNCIE and the VCDX are much the same. They represent an investment of time and energy into something that proves your capabilities. More than any other certification, the CCIE challenges people. It creates study habits and builds communities. It makes people ask themselves hard questions about desire and commitment and helps the best rise to the occasion. It’s more than just a certification.


Tom’s Take

I wouldn’t change a thing about my CCIE journey. I learned as much from the failures as I did from the success. The opportunities afforded to me because of that number have been immeasurable. But through it all I realized that the process of getting my lab has helped shape me into who I am today. A decade past late night study sessions and soul-crushing failures I know that it was all worth it because it helped me take technology more seriously and form the habits and process that have served me well from then on. I’m happy to get the new plaque that marks me as a veteran of the lab plus ten years. My status as a CCIE might pass into Emeritus but the lessons I learned along the way will always be there.

Charting the Course For Aruba

By now you’ve seen the news that longtime CEO of Aruba Keerti Melkote is retiring. He’s decided that his 20-year journey has come to a conclusion and he is stepping down into an advisory role until the end of the HPE fiscal year on October 31, 2021. Leaving along with him are CTO Partha Narasimhan and Chief Architect Pradeep Iyer. It’s a big shift in the way that things will be done going forward for Aruba. There are already plenty of hot takes out there about how this is going to be good or bad for Aruba and for HPE depending on which source you want to read. Because I just couldn’t resist I’m going to take a stab at it too.

Happy Trails To You

Keerti is a great person. He’s smart and capable and has always surrounded himself with good people as well. The HPE acquisition honestly couldn’t have gone any better for him and his team. The term “reverse acquisition” gets used a lot and I think this is one of the few positive examples of it. Aruba became the networking division of HPE. They rebuilt the husk that was HP’s campus networking division and expanded it substantially. They introduced new data center switches and kept up with their leading place in the access point market.

However, even the best people eventually need new challenges. There was always a bit of a looming role on the horizon for Keerti according to many industry analysts. As speculated by Stephen Foskett on this past week’s episode of the Gestalt IT Rundown, Keerti was the odds-on favorite to take over HPE one day. He had the pedigree of running a successful business and he understood how data moving to the cloud was going to be a huge driver for hardware in the future. He even had taken over a combined business unit of networking devices and edge computing renamed Intelligent Edge last year. All signs pointed to him being the one to step up when Antonio Neri eventually moved on.

That Keerti chose to step away now could indicate that he realized the HPE CEO job was not going to break his way. Perhaps the pandemic has sapped some of his desire to continue to run the business. Given that Partha and Pradeep are also choosing to depart as well it could be more of an indicator of internal discussions and not a choice by Keerti to move on of his own accord. I’m not speculating that there is pressure on him. It could just be that this was the best time to make the exit after steering the ship through the rough seas of the pandemic.

Rearranging the Deck Chairs

That brings me to the next interesting place that Aruba finds itself. With Keerti and company off to greener pastures, who steps in to replace them? When I first heard the news of the departure of three very visible parts of Aruba all at once my first thought jumped immediately to David Hughes, the former CEO of Silver Peak.

HPE bought Silver Peak last year and integrated their SD-WAN solutions into Aruba. I was a bit curious about this when it first happened because Aruba had been touting their SD-Branch solution that leveraged ClearPass extensively. To shift gears and adopt Silver Peak as the primary solution for the WAN edge was a shift in thinking. By itself that might have been a minor footnote.

Then a funnier thing happened that gave me pause. I started seeing more and more Silver Peak names popping up at Aruba. That’s something you would expect to see when a company gets acquired. But the people that were hopping into roles elsewhere outside of the WAN side of the house was somewhat shocking. It felt for a while like Silver Peak was taking over a lot of key positions inside of Aruba on the marketing side of the house. Which meant that the team was poised for something bigger in the long run.

When David Hughes was named as the successor to Partha and Pradeep as the CTO and Chief Architect at Aruba it made sense to me. Hughes is good at the technology. He understand the WAN and networking. He doesn’t need to worry about much about the wireless side of the house because Aruba has tons of wireless experts, including Chuck Lukaszewski. Hughes will do a great job integrating the networking and WAN side of the house to embrace the edge mentality that Aruba and HPE have been talking about for the past several months.

So, if David Hughes isn’t running Aruba, who is? That would be Phil Mottram, a veteran of the HPE Communications Technology Group. He has management material written all over him. He’s been an executive at a number of companies and he is going to steer Aruba in the direction that HPE wants it to go. That’s where the real questions are going to start being asked around here. I’m sure there’s probably going to be some kind of a speech by Antonio Neri about how Aruba is a proud part of the HPE family and the culture that has existed at Aruba is going to continue even after the departure of the founder. That’s pretty much the standard discussion you have with everyone after they leave. I’m sure something very similar happened after the Meraki founders left Cisco post-acquisition.

The Sky’s The Limit

What is HPE planning for Aruba? If I were a betting man, I’d say the current trend is going to see Aruba become more integrated into HPE. Not quite on the level of Nimble Storage but nowhere near the practical independence they’ve had for the last few years. We’re seeing that HPE is looking at Aruba as a valuable brand as much as anything else. The moves above in relation to the departure of Keerti make that apparent.

Why would you put a seasoned CEO in the role of Chief Architect? Why would you name a senior Vice President to the role of President of that business unit? And why would the CEO agree to be where he is willingly when that carrot is just out of reach? I would say it’s because David Hughes either realizes or has been told that the role of Chief Architect is going to be much more important in the coming months. That would make a lot of sense if the identity of Aruba begins to be subsumed into HPE proper.

Think about Meraki and Cisco. Meraki has always been a fiercely independent company. You would have been hard pressed for the first year or two to even realize that Cisco was the owner. However, in the past couple of years the walls that separate Cisco and Meraki have started to come down. Meraki is functioning more like a brand inside of Cisco than an independent part of the organization. It’s not a negative thing. In fact, it’s what should happen to successful companies when they get purchased. However, given the independence streak of the past it seems more intriguing than what’s on the surface.

Aruba is going to find itself being pulled in more toward HPE’s orbit. The inclusion of Aruba in the HPE Intelligent Edge business unit says that HPE has big plans for the whole thing. They don’t want to have their customers seeing HPE and Aruba as two separate things. Instead, HPE would love to leverage the customers that Aruba does have today to bring in more HPE opportunities. The synergy between the two is the whole reason for the acquisition in the first place. Why not take advantage of it? Perhaps the departure of the old guard is the impetus for making that change?


Tom’s Take

Aruba isn’t going to go away. It’s not going to be like a storage solution being eaten alive and then disappearing into a nameplate on a rack unit. Aruba has too much value as a brand and a comfortable position in the networking space to be completely eliminated. However, it is going to become more valuable to have the expertise of the Aruba teams creating more synergy inside of HPE and leading efforts to integrate the edge networking and compute solutions together to come out ahead as people shift some of their workloads around to take advantage of all the work that’s been done there. Time will tell if Aruba stays separate enough to be remembered as the titan they’ve been.

Document The First Time, Every Time

2053fountain_pen

Imagine you’re deep into a massive issue. You’ve been troubleshooting for hours trying to figure out why something isn’t working. You’ve pulled in resources to help and you’re on the line with the TAC to try and get a resolution. You know this has to be related to something recent because you just got notified about it yesterday. You’re working through logs and configuration setting trying to gain insights into what went wrong. That’s when the TAC engineer hits you with with an armor-piecing question:

When did this start happening?

Now you’re sunk. When did you first start seeing it? Was it happening before and no one noticed? Did a tree fall in the forest and no one was around to hear the sound? What is the meaning of life now?

It’s not too hard to imagine the above scenario because we’ve found ourselves in it more times than we can count. We’ve started working on a problem and traced it back to a root cause only to find out that the actual inciting incident goes back even further than that. Maybe the symptoms just took a while to show up. Perhaps someone unknowingly “fixed” the issue with a reboot or a process reload over and over again until it couldn’t work any longer. How do we find ourselves in this mess? And how do we keep it from happening?

Quirky Worky

Glitches happen. Weird bugs crop up temporarily. It happens every day. I had to reboot my laptop the other day after being up for about two months because of a series of weird errors that I couldn’t resolve. Little things that weren’t super important that eventually snowballed into every service shutting down and forcing me to restart. But what was the cause? I can’t say for sure and I can’t tell you when it started. Because I just ignored the little glitches until the major ones forced me to do something.

Unless you work in security you probably ignore little strange things that happen. Maybe an application takes twice as long to load one morning. Perhaps you click on a link and it pops up a weird page before going through to the actual website. You could even see a storage notification in the logs for a router that randomly rebooted itself for no reason in the middle of the night. Occasional weirdness gets dismissed by us because we’ve come to expect that things are just going to act strangely. I once saw a quote from a developer that said, “If you think building a steady-state machine is easy, just look at how many issues are solved with a reboot.”

We tend to ignore weirdness unless it presents itself as a more frequent issue. If a router reboots itself once we don’t think much about it. The fourth time it reboots itself in a day we know we’ve got a problem we need to fix. Could we have solved it when the first reboot happened? Maybe. But we also didn’t know we were looking at a pattern of behavior either. Human brains are wired to look for patterns of things and pick them out. It’s likely an old survival trait of years gone by that we apply to current technology.

Notice that I said “unless you work in security”. That’s because the security folks have learned over many years and countless incidents that nothing is truly random or strange. They look for odd remote access requests or strange configuration changes on devices. They wonder why random IP addresses from outside the company are trying to access protected systems. Security professionals treat every random thing as a potential problem. However, that kind of behavior also demonstrates the downside of analyzing every little piece of information for potential threats. You quickly become paranoid about everything and spend a lot of time and energy trying to make sense out of potential nonsense. Is it any wonder that many security pros find themselves jumping at every little shadow in case it’s hiding a beast?

Middle Ground of Mentions

On the one hand, we have systems people that dismiss weirdness until it’s a pattern. On the other we have security pros that are trying to make patterns out of the noise. I’m sure you’re probably wondering if there has to be some kind of middle ground to ensure we’re keeping track of issues without driving ourselves insane.

In fact, there is a good policy that you need to get into the habit of doing. You need to write it all down somewhere. Yes, I’m talking about the dreaded documentation monster. The kind of thing that no one outside of developers likes to do. The mean, nasty, boring process of taking the stuff in your brain and putting it down somewhere so someone can figure out what you’re thinking without the need to read your mind.

You have to write it down because you need to have a record to work from if something goes wrong later. One of the greatest features I’ve ever worked with that seems to be ignored by just about everyone is the Windows Shutdown Reason dialog box in Windows Server 2003 and above. Rebooting a box? You need to write in why and give a good justification. That way if someone wants to know why the server was shut off at 11:15am on a Tuesday they can examine the logs. Unfortunately in my experience the usual reason for these shutdowns was either “a;lkjsdfl;kajsdf” or “because I am doing it”. Which aren’t great justifications for later.

You don’t have to be overly specific with your documentation but you need to give enough detail so that later you can figure out if this is part of a larger issue. Did an application stop responding and need to be restarted? Jot that down. Did you need to kill a process to get another thing running again? Write down that exact sentence. If you needed to restart a router and you ended up needing to restore a configuration you need to jot that part down too. Because you may not even realize you have an issue until you have documentation to point it out.

I can remember doing a support call years ago with a customer and in conversation he asked me if I knew much about Cisco routers. I chuckled and said I knew a bit. He said that he had one that he kept having to copy the configuration files to every time it restarted because it came up blank. He even kept a console cable plugged into it for just that reason. Any CCNA out there knows that’s probably a config register issue so I asked when it started happening. The person told me at least a year ago. I asked if anyone had to get into the router because of a forgotten password or some other lockout. He said that he did have someone come out a year and a half prior to reset a messed up password. Ever since then he had to keep putting the configuration back in. Sure enough, the previous tech hadn’t reset the config register. One quick fix and the customer was very happy to not have to worry about power outages any longer.

Documenting when things happen means you can build a timeline per device or per service to understand when things are acting up. You don’t even need to figure it out yourself. The magic of modern systems lies in machine learning. You may think to yourself that machine learning is just fancy linear regression at this point and you would be right more often than not. But one thing linear regression is great at doing is surfacing patterns of behavior for specific data points. If your router reboots on the third Wednesday of every month precisely at 3:33am the ML algorithms will pick up on that and tell you about it. But that’s only if your system catches the reboot through logs or some other record keeping. That’s why you have to document all your weirdness. Because the ML systems can analyze what they don’t know about.


Tom’s Take

I love writing. And I hate documentation. Documentation is boring, stuffy, and super direct. It’s like writing book reports over and over again. I’d rather write a fun blog post or imagine an exciting short story. However, documentation of issues is critical to modern organizations because these issues can spiral out of hand before you know it. If you don’t write it down it didn’t happen. And you need to know when it happened if you hope to prevent it in the future.

When Hardware Drives Software Upgrades

8006701A-85B8-4391-BD36-487B35755676

What’s your favorite version of Microsoft Windows? Is it Windows 10? Maybe it’s Windows XP? Windows 95? Odds are good that you have one version you appreciated more than most. Windows XP, Windows 7, and Windows 10 tend to rank high on the list. Windows ME and Windows 8 seem to rank pretty low. Yet, for all their impressive love and all the users clinging to them we don’t really use anything other than Windows 10 any more.

You might be tempted to say that the OS isn’t supported any longer so there’s no reason to run it. Yet we still drive vehicles that are no longer under warranty. We still buy classic cars that are older than we are and put parts in them to keep them running. Why is software different? What drives us to keep needing to upgrade our programs?

You might be shocked to learn that the most popular reason to upgrade software is, in fact, driven by hardware. It’s not the memory requirements or the fancy new user interface that drives people to move to the new platform. More often than not it’s because a new piece of hardware has requirements that only work on the latest version of the system.

It happened to me once or twice. I can distinctly remember needing to go out and buy a new printer to replace some cheap HP Inkjet I purchased for a project because when I upgraded from Windows XP to Windows 7 the drivers didn’t support the move. Why spend money writing new drivers for a cheap printer when you could just make people go out and buy another new cheap printer? I swear that’s what happened. And, of course, the most expensive the device you purchase the more likely it stays supported, right?

The Lords of COBOL

By now I’m sure you’re all familiar with the little tidbit of information that most of the world’s insurance companies run their databases on ancient mainframes. Why? Well, most of their software still requires COBOL to run. Large organizations don’t like to move to new platforms very often. It wasn’t that long ago that Southwest Airlines moved to a new booking system because the old one only had two days you could schedule flights – Monday through Friday and Sunday. If you scheduled a flight for Monday through Friday you had to have the same flight at the same time every day no matter what. It’s even widely believed that part of the reason that United Airlines merged with Continental was because they wanted to switch to a better booking system.

Why do companies keep these systems around? It should be easy to just migrate off of them, right? Well, reality is that between the sunk cost of operating a mainframe for years and patching the software that you’ve built to operate your business the desire to move to something else isn’t always a driver. After all, if it ain’t broke why fix it? Companies can keep maintaining old systems as long as someone sticks around to keep the lights on. I can remember working with a number of IT professionals over the years that had their jobs mostly because they were the final remaining mainframe wizard that knew how to put the system into maintenance mode or remembered the magical incantations to reboot the old machine after a power failure.

Alas, nothing lasts forever. The current pattern seems to be pretty standard. The old wizards finally decide to retire. They’ve had enough and they’re ready to move to somewhere warm and enjoy not working. The management keeps the lights going because it’s not that hard, right? It would take way too much to rewrite the software or move people to a new platform. Until the day when the system stops working. The day when everything doesn’t come back up. Then it’s panic mode. Was that just the database? What if it’s the actual hardware. Do they still make parts for this? Does anyone even know what this button does? Eventually either the hard decision is made to cut over somehow or an exorbitant amount of money is paid to the former operations people to come back and get things running again long enough to figure out how to keep this from happening again. And if you think you’re going to be able to train a developer to just pick up where the grizzled old wizard left off, good luck. Go find a COBOL training course somewhere. I’ll wait.

Modern Makers Make Mistakes Too

If you think that the modern era of cloud development is any different than writing FORTRAN or COBOL on a mainframe you’ve got a nice set of rose-colored glasses. We’re locking ourselves into the same patterns of thought that brought on the monoliths we’re currently trying to tear down. Every time you enable a feature that only works on one cloud platform or you choose to develop in a hot new language that isn’t fully supported everywhere you’re putting up a barrier that will eventually lead to you making hard choices.

You know what’s different this time, though? You don’t have the luxury of a position where you get to be the wizard that knows how to keep the lights on. As the article above mentions, the race is on to get the COBOL migrated to a modern platform that allows integration with languages like C# and Java. Do you believe that having platforms like that means you’ll get to a point where you can be the last remaining person around that remembers what crazy setup you used to minimize the number of containers an app was using? Or do you think it’s more likely they’ll just fire you, figure out how to integrate your legacy code into a new platform, and go on painting themselves right back into corners?

Hardware is the last true driver to keep people moving along into a place where they are forced to do things the right way. If your hardware doesn’t support something you don’t do it. If you need to ensure that your code is portable you don’t bake in features that require specific hardware or you create a situation where you’re tied to that platform forever. That’s why cloud is a bit scary in my mind. Because you’re agnostic from the hardware. You can do whatever you want without limit.

Want to write software that requires the use of hundreds of processing threads? You can do it because why not? You aren’t limited to just one chip any longer. Want to eat up tons of memory and storage? Go for it. You get to use as much as your credit card can hold. Now the bounds of a programmer’s imagination is no longer limited to physical hardware limitations. If you don’t believe me then ask yourself why there are apps on the App Store today that are bigger then the entire amount of storage that the original iPhone was capable of. Sure, hardware brought us to this point. But ditching the hardware for the magic of the cloud means there isn’t anything holding back those that want to build the biggest, burliest, baddest application they can!


Tom’s Take

Somewhat ironically, I’m not really that worried about the cloud letting people build ugly things to their hearts’ content. Why? Just like terrible movie directors, once you’ve removed their limitations you expose their vulnerabilities and they build something that is unsustainable. Build the biggest app you can. You’ll find out that it collapses under its own weight. Even the promise of the mythical giant virtual machines with 1 TB of RAM haven’t made them materialize. Why? Because it turns out that removing restrictions just enforces them through trial and error. If you have to build small because you can’t get crazy due to hardware you’re held back by external forces. But when you are held back because you tried it that way the last time and you failed by creating an app that takes ten minutes to load you learned your lesson. You get leaner and better and more portable next time. And that’s the kind of driver that makes software and hardware better for us all.