Charting the Course For Aruba

By now you’ve seen the news that longtime CEO of Aruba Keerti Melkote is retiring. He’s decided that his 20-year journey has come to a conclusion and he is stepping down into an advisory role until the end of the HPE fiscal year on October 31, 2021. Leaving along with him are CTO Partha Narasimhan and Chief Architect Pradeep Iyer. It’s a big shift in the way that things will be done going forward for Aruba. There are already plenty of hot takes out there about how this is going to be good or bad for Aruba and for HPE depending on which source you want to read. Because I just couldn’t resist I’m going to take a stab at it too.

Happy Trails To You

Keerti is a great person. He’s smart and capable and has always surrounded himself with good people as well. The HPE acquisition honestly couldn’t have gone any better for him and his team. The term “reverse acquisition” gets used a lot and I think this is one of the few positive examples of it. Aruba became the networking division of HPE. They rebuilt the husk that was HP’s campus networking division and expanded it substantially. They introduced new data center switches and kept up with their leading place in the access point market.

However, even the best people eventually need new challenges. There was always a bit of a looming role on the horizon for Keerti according to many industry analysts. As speculated by Stephen Foskett on this past week’s episode of the Gestalt IT Rundown, Keerti was the odds-on favorite to take over HPE one day. He had the pedigree of running a successful business and he understood how data moving to the cloud was going to be a huge driver for hardware in the future. He even had taken over a combined business unit of networking devices and edge computing renamed Intelligent Edge last year. All signs pointed to him being the one to step up when Antonio Neri eventually moved on.

That Keerti chose to step away now could indicate that he realized the HPE CEO job was not going to break his way. Perhaps the pandemic has sapped some of his desire to continue to run the business. Given that Partha and Pradeep are also choosing to depart as well it could be more of an indicator of internal discussions and not a choice by Keerti to move on of his own accord. I’m not speculating that there is pressure on him. It could just be that this was the best time to make the exit after steering the ship through the rough seas of the pandemic.

Rearranging the Deck Chairs

That brings me to the next interesting place that Aruba finds itself. With Keerti and company off to greener pastures, who steps in to replace them? When I first heard the news of the departure of three very visible parts of Aruba all at once my first thought jumped immediately to David Hughes, the former CEO of Silver Peak.

HPE bought Silver Peak last year and integrated their SD-WAN solutions into Aruba. I was a bit curious about this when it first happened because Aruba had been touting their SD-Branch solution that leveraged ClearPass extensively. To shift gears and adopt Silver Peak as the primary solution for the WAN edge was a shift in thinking. By itself that might have been a minor footnote.

Then a funnier thing happened that gave me pause. I started seeing more and more Silver Peak names popping up at Aruba. That’s something you would expect to see when a company gets acquired. But the people that were hopping into roles elsewhere outside of the WAN side of the house was somewhat shocking. It felt for a while like Silver Peak was taking over a lot of key positions inside of Aruba on the marketing side of the house. Which meant that the team was poised for something bigger in the long run.

When David Hughes was named as the successor to Partha and Pradeep as the CTO and Chief Architect at Aruba it made sense to me. Hughes is good at the technology. He understand the WAN and networking. He doesn’t need to worry about much about the wireless side of the house because Aruba has tons of wireless experts, including Chuck Lukaszewski. Hughes will do a great job integrating the networking and WAN side of the house to embrace the edge mentality that Aruba and HPE have been talking about for the past several months.

So, if David Hughes isn’t running Aruba, who is? That would be Phil Mottram, a veteran of the HPE Communications Technology Group. He has management material written all over him. He’s been an executive at a number of companies and he is going to steer Aruba in the direction that HPE wants it to go. That’s where the real questions are going to start being asked around here. I’m sure there’s probably going to be some kind of a speech by Antonio Neri about how Aruba is a proud part of the HPE family and the culture that has existed at Aruba is going to continue even after the departure of the founder. That’s pretty much the standard discussion you have with everyone after they leave. I’m sure something very similar happened after the Meraki founders left Cisco post-acquisition.

The Sky’s The Limit

What is HPE planning for Aruba? If I were a betting man, I’d say the current trend is going to see Aruba become more integrated into HPE. Not quite on the level of Nimble Storage but nowhere near the practical independence they’ve had for the last few years. We’re seeing that HPE is looking at Aruba as a valuable brand as much as anything else. The moves above in relation to the departure of Keerti make that apparent.

Why would you put a seasoned CEO in the role of Chief Architect? Why would you name a senior Vice President to the role of President of that business unit? And why would the CEO agree to be where he is willingly when that carrot is just out of reach? I would say it’s because David Hughes either realizes or has been told that the role of Chief Architect is going to be much more important in the coming months. That would make a lot of sense if the identity of Aruba begins to be subsumed into HPE proper.

Think about Meraki and Cisco. Meraki has always been a fiercely independent company. You would have been hard pressed for the first year or two to even realize that Cisco was the owner. However, in the past couple of years the walls that separate Cisco and Meraki have started to come down. Meraki is functioning more like a brand inside of Cisco than an independent part of the organization. It’s not a negative thing. In fact, it’s what should happen to successful companies when they get purchased. However, given the independence streak of the past it seems more intriguing than what’s on the surface.

Aruba is going to find itself being pulled in more toward HPE’s orbit. The inclusion of Aruba in the HPE Intelligent Edge business unit says that HPE has big plans for the whole thing. They don’t want to have their customers seeing HPE and Aruba as two separate things. Instead, HPE would love to leverage the customers that Aruba does have today to bring in more HPE opportunities. The synergy between the two is the whole reason for the acquisition in the first place. Why not take advantage of it? Perhaps the departure of the old guard is the impetus for making that change?


Tom’s Take

Aruba isn’t going to go away. It’s not going to be like a storage solution being eaten alive and then disappearing into a nameplate on a rack unit. Aruba has too much value as a brand and a comfortable position in the networking space to be completely eliminated. However, it is going to become more valuable to have the expertise of the Aruba teams creating more synergy inside of HPE and leading efforts to integrate the edge networking and compute solutions together to come out ahead as people shift some of their workloads around to take advantage of all the work that’s been done there. Time will tell if Aruba stays separate enough to be remembered as the titan they’ve been.

When Hardware Drives Software Upgrades

8006701A-85B8-4391-BD36-487B35755676

What’s your favorite version of Microsoft Windows? Is it Windows 10? Maybe it’s Windows XP? Windows 95? Odds are good that you have one version you appreciated more than most. Windows XP, Windows 7, and Windows 10 tend to rank high on the list. Windows ME and Windows 8 seem to rank pretty low. Yet, for all their impressive love and all the users clinging to them we don’t really use anything other than Windows 10 any more.

You might be tempted to say that the OS isn’t supported any longer so there’s no reason to run it. Yet we still drive vehicles that are no longer under warranty. We still buy classic cars that are older than we are and put parts in them to keep them running. Why is software different? What drives us to keep needing to upgrade our programs?

You might be shocked to learn that the most popular reason to upgrade software is, in fact, driven by hardware. It’s not the memory requirements or the fancy new user interface that drives people to move to the new platform. More often than not it’s because a new piece of hardware has requirements that only work on the latest version of the system.

It happened to me once or twice. I can distinctly remember needing to go out and buy a new printer to replace some cheap HP Inkjet I purchased for a project because when I upgraded from Windows XP to Windows 7 the drivers didn’t support the move. Why spend money writing new drivers for a cheap printer when you could just make people go out and buy another new cheap printer? I swear that’s what happened. And, of course, the most expensive the device you purchase the more likely it stays supported, right?

The Lords of COBOL

By now I’m sure you’re all familiar with the little tidbit of information that most of the world’s insurance companies run their databases on ancient mainframes. Why? Well, most of their software still requires COBOL to run. Large organizations don’t like to move to new platforms very often. It wasn’t that long ago that Southwest Airlines moved to a new booking system because the old one only had two days you could schedule flights – Monday through Friday and Sunday. If you scheduled a flight for Monday through Friday you had to have the same flight at the same time every day no matter what. It’s even widely believed that part of the reason that United Airlines merged with Continental was because they wanted to switch to a better booking system.

Why do companies keep these systems around? It should be easy to just migrate off of them, right? Well, reality is that between the sunk cost of operating a mainframe for years and patching the software that you’ve built to operate your business the desire to move to something else isn’t always a driver. After all, if it ain’t broke why fix it? Companies can keep maintaining old systems as long as someone sticks around to keep the lights on. I can remember working with a number of IT professionals over the years that had their jobs mostly because they were the final remaining mainframe wizard that knew how to put the system into maintenance mode or remembered the magical incantations to reboot the old machine after a power failure.

Alas, nothing lasts forever. The current pattern seems to be pretty standard. The old wizards finally decide to retire. They’ve had enough and they’re ready to move to somewhere warm and enjoy not working. The management keeps the lights going because it’s not that hard, right? It would take way too much to rewrite the software or move people to a new platform. Until the day when the system stops working. The day when everything doesn’t come back up. Then it’s panic mode. Was that just the database? What if it’s the actual hardware. Do they still make parts for this? Does anyone even know what this button does? Eventually either the hard decision is made to cut over somehow or an exorbitant amount of money is paid to the former operations people to come back and get things running again long enough to figure out how to keep this from happening again. And if you think you’re going to be able to train a developer to just pick up where the grizzled old wizard left off, good luck. Go find a COBOL training course somewhere. I’ll wait.

Modern Makers Make Mistakes Too

If you think that the modern era of cloud development is any different than writing FORTRAN or COBOL on a mainframe you’ve got a nice set of rose-colored glasses. We’re locking ourselves into the same patterns of thought that brought on the monoliths we’re currently trying to tear down. Every time you enable a feature that only works on one cloud platform or you choose to develop in a hot new language that isn’t fully supported everywhere you’re putting up a barrier that will eventually lead to you making hard choices.

You know what’s different this time, though? You don’t have the luxury of a position where you get to be the wizard that knows how to keep the lights on. As the article above mentions, the race is on to get the COBOL migrated to a modern platform that allows integration with languages like C# and Java. Do you believe that having platforms like that means you’ll get to a point where you can be the last remaining person around that remembers what crazy setup you used to minimize the number of containers an app was using? Or do you think it’s more likely they’ll just fire you, figure out how to integrate your legacy code into a new platform, and go on painting themselves right back into corners?

Hardware is the last true driver to keep people moving along into a place where they are forced to do things the right way. If your hardware doesn’t support something you don’t do it. If you need to ensure that your code is portable you don’t bake in features that require specific hardware or you create a situation where you’re tied to that platform forever. That’s why cloud is a bit scary in my mind. Because you’re agnostic from the hardware. You can do whatever you want without limit.

Want to write software that requires the use of hundreds of processing threads? You can do it because why not? You aren’t limited to just one chip any longer. Want to eat up tons of memory and storage? Go for it. You get to use as much as your credit card can hold. Now the bounds of a programmer’s imagination is no longer limited to physical hardware limitations. If you don’t believe me then ask yourself why there are apps on the App Store today that are bigger then the entire amount of storage that the original iPhone was capable of. Sure, hardware brought us to this point. But ditching the hardware for the magic of the cloud means there isn’t anything holding back those that want to build the biggest, burliest, baddest application they can!


Tom’s Take

Somewhat ironically, I’m not really that worried about the cloud letting people build ugly things to their hearts’ content. Why? Just like terrible movie directors, once you’ve removed their limitations you expose their vulnerabilities and they build something that is unsustainable. Build the biggest app you can. You’ll find out that it collapses under its own weight. Even the promise of the mythical giant virtual machines with 1 TB of RAM haven’t made them materialize. Why? Because it turns out that removing restrictions just enforces them through trial and error. If you have to build small because you can’t get crazy due to hardware you’re held back by external forces. But when you are held back because you tried it that way the last time and you failed by creating an app that takes ten minutes to load you learned your lesson. You get leaner and better and more portable next time. And that’s the kind of driver that makes software and hardware better for us all.

Networking Isn’t Just A Tool

BlueFiberOptic

It’s another event week for me at Networking Field Day 25 and I’m continually impressed with the level of technology that we see in the networking world. I think back to how things looked when I was still deploying the networks I built and it seems like a hundred years ago instead of a decade. More software driving better outcomes for users. Easier collection of analytics and telemetry to understand how to tune things and make them faster and better. And, honestly, more need for advanced technical people to tune everything and make it work better.

When you consider that the last year has been done over the Internet for most of us it gets even crazier. Meetings, software productivity, and even food delivery has been driven by apps running in the cloud that we communicate with over the Internet. I can remember a time when I didn’t have a mobile phone in my pocket with Internet capabilities. Today I can barely imagine not having it at my fingertips. When the network is not doing things the way we want we quickly find out how dependent we’ve become on our connectivity.

Generational Differences

My children are amazed that dial-up networking used to be a thing. I remember rebuilding Winsock stacks in Windows 98 for Gateway in order to troubleshoot 56k modems not connecting to AOL at the beginning of the millennium. Today my cable modem gets me where I need to go when I’m home and my 5G phone does the heavy lifting for me when I’m on the road. Need to look up a price on something? Or know the temperature? Or just listen to a song you remember from your childhood? It’s all at your fingertips. I can’t imagine that kind of connectivity back when it took a minute or two for the phone to scream out the song of Compuserve at my house.

My in-laws have a DSL connection that suits them just fine for their needs. It’s painfully slow for me. I couldn’t live with their slow connection and inability to run multiple things at once, like video streaming and Zoom meetings at the same time. They don’t live very far from me either. The difference in their connectivity is shocking. And yet, we just expect to be able to get online any time we want.

Remember ATTWIFI at Starbucks? Remember when your iPhone would automatically connect to give you better speeds when your 3G was overwhelmed? I can recall getting into a situation where Cisco Live in Las Vegas made my phone unusable outside of the conference. Today that situation would be unacceptable. And we’re barely a decade removed from those days.

As I keep seeing technology moving along even faster, including things like silicon photonics promising speeds north of hundreds of gigabits on the uplink side, I wonder how our next generation is going to feel about not being able to watch 8k TV shows in a self-driving car on-demand because there’s not enough bandwidth. I laugh when I remember the need to swap out DVDs on car trips so my eldest son could have entertainment. Today my youngest is happy to binge watch shows on Disney without interruption because of the networks we’ve built.

Creating Dependence

What we’ve built has created the world we live in. But we also have made it a world dependent on what we’ve built. I realized that months ago when my network connection kept going out during a winter storm. Without connectivity people feel lost. I had a hard time getting things done offline without being able to look up information or get emails sent out. My kids are beside themselves without access to anything online. Their board games were boring. They couldn’t play video games offline because all the cool features were on the Internet. By the time the connection came back it was almost Lord of the Flies around here as the minutes ticked on.

We no longer have the luxury of shrugging our shoulders when the network goes down. It needs to be treated no differently than the electricity or water in a building. If we neglect it we risk alienating our users and stakeholders. We need to be firm when we need new equipment or better designs to ensure resilience. Instead of making everything cheap and barely usable we need to remind everyone how reliant they’ve become on the network. If it’s necessary it is absolutely worth investing in. Moving to the cloud or becoming more and more reliant on SaaS applications just reinforces those decisions.


Tom’s Take

Either the network is just a tool that doesn’t need investment or it’s a necessary part of your work that needs to be treated as such. While I would never suggest unplugging anything to prove a point I think you can point to specific outages that would do the same thing without the chaos. Every time you tell your stakeholders they need to invest in better switches or new access points and they push back about costs or try to suggest a cheaper alternative, you need to stand firm. In a world where everyone is dependent on Internet connectivity for all manner of their lives you have to treat it as a necessity in every possible way. You can tell your stakeholders to spend their day working from their phone hotspot if they don’t believe you. It’ll be like taking a trip back to the early parts of the millennium when networks weren’t as important.

Knowledge is Powerful and Needs to Be Shared

WritingPen

A tweet this morning from my friend Stephanie stood out in my timeline because she’s talking about something I’ve seen happen over and over again in my lifetime:

How many times have we seen this in our organizations? People want to hoard knowledge because they feel like it’s power. Maybe they’re worried that if anyone knew what they know it would mean they could get fired. Perhaps they enjoy holding the keys to the kingdom and not allowing anyone else to do something or know something they know. It could even be that they like the idea of mystery in the air and not allowing people to know the whole truth keeps things alive, as the founders of Coca-Cola and Colonel Sanders will happily tell you.

Over the years I’ve figured out that hoarding knowledge leads to ruin. I’ve been involved in so many scenarios were a lack of knowledge sharing ended up causing the kinds of problems that were easily avoidable if someone had just committed what they knew to memory. It wasn’t always malicious either. There is a lot of information that people collect incidentally and just flat out forget to write down until it’s too late.

The first time I truly realized how important good documentation was happened on a sad day in my career. My mentor, Wes Williams, tragically passed away during an otherwise uneventful workday during lunch. I wasn’t there but our office got the news shortly thereafter and we were stunned. No one in the office was even thinking about anything other than how tragic it was and how we were going to miss him. Eventually, one of the other engineers stood up and asked, “Does anyone know the password to Wes’s laptop?” We realized that Wes had a ton of knowledge about his clients and other operational things saved on his laptop, which was sitting unlocked on his desk. We rushed to keep it from going to sleep and to make sure that we changed the password to something we all knew. After that, the tedious process of copying all his data to a shared location began. Wes didn’t think to save it all somewhere else because he didn’t know he would need to until it was too late.

Information Is Currency

Since then I’ve tried very hard to share the knowledge that I have so that everyone benefits. The cynics in the organization may see information as a kind of currency to trade power. If you have to come to them to get the answers then they are important in the grand scheme of things. Kind of like a power broker of sorts.

My outlook is different. Having the knowledge written down somewhere means someone can learn what they need and do the job without needing to get me involved. Maybe they have other questions or need further explanation. That’s a great time to engage me. But I don’t need to be a gatekeeper for the basics or feel like I have to hoard it all to myself.

If you think that hoarding knowledge makes you impervious to firing or reassignment, you need to be careful with your hubris. No one is immune from reductions. External factors can limit your involvement with your career or organization before you know it. We use the idea of the “hit by a bus” test to explain how knowledge needs to be shared in the event of a sudden thing like the issue I mentioned above. But what if the situation develops slowly and you get caught up before realizing it’s too late to share what you know?

Power Corrupts

If you’re the person in the company that always calls the ISP to report outages because you know exactly how to navigate the automated trouble ticket system you have value. However, if you never write that down for your team members you’re going to limit your mobility out of your job. No matter how high you climb it will always be your responsibility. Worse yet, if you find yourself in a place where you can’t communicate what you know and someone else has to figure it out on their own your so-called “power” is useless now.

Information is a strange form of power because it loses all it’s inherent value as soon as it’s shared. If I know who the next supervisor is going to be before anyone else then I only have power as long as no one else knows. As soon as someone finds out I no longer control that information and therefore it’s useless. Instead of hoarding knowledge and information to wield as a cudgel to lord over others you should share it freely to ensure it’s not lost. How many advances in human history have been lost because no one wrote them down?

Knowing how something works isn’t a tool you should use to extract further value at the expense of capability. Even if the knowledge is something that shouldn’t be shared for reasons related to secrecy you still need to let people know there is a purpose and they either don’t need to know right now or it’s something that will be available to others. There are many times when I’ve been told that a decision-making process exists and I don’t need to understand it right now. I’m fine with that just so long as it’s written down somewhere. If it’s not then we may all find ourselves recreating a process that was a solved issue simply because someone took the secrets of how it worked with them when they moved on to a different role or out of the organization entirely.


Tom’s Take

There are times when I have to keep things secret. I know something and it’s not time to reveal it. But processes or knowledge about basic things should never fall into that category. It’s one thing to know that your supervisor is being promoted to section leader next week. It’s entirely different to hoard the knowledge of how the factory wireless network is configured because you are afraid if you don’t you’ll get fired. Commit every piece of knowledge you can to paper, physical or virtual, while you can. You never know when someone will need to know what you know. Knowledge is only valuable as currency when it’s shared. And that’s the kind of power that makes you valuable to us all.

Fast Friday Thoughts from the Woods

I’m at camp this week helping put on the second weekend of the Last Frontier Council Wood Badge course which is my idea of a vacation. I’m learning a lot, teaching a lot more, and having fun. But that does’t mean I’m not working too. Lots of fun conversations that make me recall the way people consume information, communicate what they know, and all too often overlook the important things they take for granted.

  • Why is IT one of the few disciplines that expects people to come in fully trained and do the job instead of learning while doing it? Is that because hiring managers don’t want to train people? Or is it because senior people are less likely to impart knowledge to protect their jobs? I don’t have a good answer but I know what the result looks like and it’s not something that’s positive, either for the people doing the job or how it’s perceived outside of IT.
  • There is a ton of value in doing something for real instead of just planning it and calling it good. DR plans need to be tested. Network changes need to be mocked up. No matter what kind of critical thing you’ll be doing you need to try it for real instead of just putting it on paper and hoping that it works. In theory, there’s no difference between theory and practice. In practice, there is.
  • Don’t forget to commit your knowledge to paper at some point in your career. You may know everything under the sun but when you’re not available you will be missed. You know more than you realize and you need to make sure that others benefit from what you’re capable of helping with. Those that hold the knowledge only have the power when they share it. Keeping it to yourself only hurts those that can benefit. Use your currency effectively.

Tom’s Take

These questions don’t have answers. They are designed to make you think about how you do things and what you can do to leave a legacy for the people that come behind you. Don’t be the story that everyone tells about that guy they hated. Be the inspiration for those that want to be like you in the future.

Tom’s Corner and Turning Another Corner

Thanks to everyone that popped in for Tom’s Virtual Corner at Cisco Live Global 2021. It was a great time filled with chats about nothing in particular, crazy stories about unimportant things, and even the occasional funny picture. It was just was Tom’s Corner has always been. A way for the community to come together and be around each other in a relaxing and low-key environment. Maybe we couldn’t meet in person but we got together when we needed it the most.

There was also something else that Tom’s Corner has represented for me for the last year that I didn’t even catch until it was pointed out to me by my wonderful wife Kristin (@MrsNetwrkingNerd). Tom’s Corner was the start of something that made me feel better about everything.

Get On Up and Move

After Tom’s Virtual Corner in 2020, I was energized. I needed to get up and get things done after sitting in a chair for hours talking to all my absent friends and getting the energy I needed to feel after months of being locked away during a pandemic. I felt on top of the world for the first time in quite a while. And I needed to do something with that burst of energy.

So I got up and went for a walk. Exciting, right? I recorded a quick Periscope (RIP) video thanking everyone in the community for turning out and being a part of things even behind the keyboard. Then I kept going. Walking a mile or two and then coming back home. I noticed that after I did I closed my Apple Watch fitness rings for calories burned and exercise.

Rather than just counting steps like a Fitbit, Apple came up with their own measure of activity. I had a love/hate relationship with it. Some days it was easy to get my goal for standing once a minute every hour. Other days was impossible thanks to airplanes and such. Calories could be a breeze on days at events due to all the walking. Or they could be a disaster on days when I barely got out of my chair. Exercise goals were almost impossible. How could I find time to hit those?

After the amazing energy of Tom’s Virtual Corner in 2020, I decided to get serious about fitness. No more excuses. No more giving up and calling it a day because it was too hot outside or raining. I was going to do this because I needed to do it and the people that I care for in the community gave their time to be a part of something special to me. If they could do that I could find the time to do it too.

June 1 became June 30th and I was still going strong. I kept moving. My walking turned into getting back into running with a Couch to 5k beginner program I’ve done several times before. This time I was determined not to give up so easily. I thought back to the year before when I was able to run for a bit but ended up mixing up the walking and running and ultimately going back to being all but sedentary. The key for me this time was moving every day. Not just getting out every other day but walking one day and running the next.

The Running Nerd

Before I knew it, I was running more than I was walking. I was going further with every exercise. I felt better. I also noticed that I was losing weight, which was something that hadn’t happened the last time I was exercising. My health was improving. My belt had to tighten an extra notch. By the time of Mobility Field Day 5 at the end of July someone pointed out that I looked slimmer. I didn’t really notice it that much.

As the months went on more and more people remarked that I looked slimmer and fitter. It took four months before I stepped on a scale to be sure. And yes, I really was losing weight. Not a few pounds but a lot of them. As best as I can guess I was somewhere just over 265 pounds in this picture from February 2020:

Looking back at it now I can’t even believe it. I didn’t change my diet. All that happened was that we started cooking more and more meals at home thanks to the pandemic. I got better at making things I wanted to eat and knowing what was going into them. My every-other-day runs became more frequent. I was only walking twice a week now and running the other five days. My runs went from just barely 3 miles on occasion to at least 5 miles a day. My walks were up to 6 miles a day too.

Here’s a side by side comparison of what I looked like in June of 2020 and what I look like right now (March 2021):

I’m down three pants sizes. I weigh 210 pounds, down 55 lbs from my highest weight just over a year ago. My running is getting faster. I feel better. My doctor has told me everything looks much better than it did when he saw me last year for my checkup. When I got my BSA medical form filled out for my activities right before the pandemic last year I was worried I was going to be in trouble because I was too heavy for my height. When I had it done again last month I was well under the weight limit for the first time in a very long time.

Tom’s Corner is what set me on this path. The positive energy of the community kicked me forward and helped me see that I needed to be healthy. I needed to get back to a place where I didn’t feel self-conscious and worried about weight gain or my fitness to go on hikes or be active. The response of people when they see how much weight I’ve lost or how much I run has boosted my confidence even more and encouraged me to keep going.

I’ve closed my rings for 303 straight days. The light at the end of the year-long tunnel is almost here. I’m going to make it there even if I have to walk at 11:00pm at night to get those last few calories or do an Apple workout in the middle of a blizzard to make sure I didn’t miss my exercise goal. It’s something that drives me to wake up and make plans for the day. When I didn’t get my run in during Tom’s Corner this year I made sure to get it done as soon as I could so I didn’t miss out. I need to stay healthy and happy to ensure there are more Tom’s Corner meetups in the future.


Tom’s Corner

Motivation is hard. I’ve learned that lesson my whole life both the easy way and the hard way. Like a shark, if I stop moving I’m not going to start again. Planning how I’m going to make this all happen takes cycles of my day but results in a happier, healthier, more confident me. Thanks to my friends and fellow community members at Tom’s Corner I’ve transformed myself into something to be proud of. I’m often told that Tom’s Corner gives people something they need. Maybe it’s a place in the community. Or a friendly conversation when they need it the most. For me, Tom’s Corner was the kick in the pants I needed to be a better Tom. Thank you all for helping me turn that corner and get better.

Tech Field Day Changed My Life

It’s amazing to me that it’s been ten years since I attended by first Tech Field Day event. I remember being excited to be invited to Tech Field Day 5 and then having to rush out of town a day early to beat a blizzard to be able to attend. Given that we just went through another blizzard here I thought the timing was appropriate.

How did attending an industry event change my life? How could something with only a dozen people over a couple of days change the way I looked at my career? I know I’ve mentioned parts of this to people in the past but I feel like it’s important to talk about how each piece of the puzzle built on the rest to get me to where I am today.

Voices Carry

The first thing Tech Field Day did to change my life was to show me that I mattered. I grew up in a very small town and spent most of my formative school years being bored. The Internet didn’t exist in a usable form for me. I devoured information wherever I could find it. And I languished as I realized that I needed more to keep learning at the pace I wanted. When I finally got through college and started working in my career the same thing kept happening. I would learn about a subject and keep devouring that knowledge until I exhausted it. Yet I still wanted more.

Tech Field Day reinforced that my decision to start a blog to share what I was learning was the right one. It wasn’t as much about the learning as it was the explanation. Early on I thought a blog was just about finding some esoteric configuration stanza and writing about it. It wasn’t until later on that I figured out that my analysis and understanding and explanation was more important overall. Even my latest posts about more “soft skill” kinds of ideas are less about the ideas and how I apply them.

Blogging and podcasting are just tools to share the ideas that we have. We all have our own perspectives and people enjoy listening to those. They may not always agree. They may have their own opinions that they want to share. However, the part that is super critical is that everyone is able to share in a place where they can be discussed and analyzed and understood. As long as we all learn and grow from what we share then the process works. It’s when we stop learning and sharing and try to protest that our way is right and the only way that we stop growing.

Tech Field Day gave me the platform to see that my voice mattered and that people listened. Not just read. Not just shared. That they listened and that they wanted to hear more. People started asking me to comment on things outside of my comfort zone. Maybe it was wireless networking. It could have been storage or virtualization or even AI. It encouraged me to learn more and more because who I was and what I said was interesting. The young kid that could never find someone to listen when I wanted to talk about Star Wars or BattleTech or Advanced Dungeons and Dragons was suddenly the adult that everyone wanted to ask questions to. It changed the way I looked at how I shared with people for the better.

Not Just a Member, But the President

The second way Tech Field Day changed my life was when I’d finally had enough of what I was doing. Because of all the things that I had seen in my events from 2011 to 2013, I realized that working as an engineer and operations person for a reseller had a ceiling I was quickly going to hit. The challenges were less fun and more frustrating. I could see technology on the horizon and I didn’t have a path to get to a place to implement it. It felt like watching something cool happening outside in the yard while I was stuck inside washing the dishes.

Thankfully, Stephen Foskett knew what I needed to hear. When I expressed frustration he encouraged me to look around for what I wanted. When I tried to find a different line of work that didn’t understand why I blogged, it crystallized in me that I needed something very different from what I was doing. Changing who I was working for wasn’t enough. I needed something different.

Stephen recognized that and told me he wanted me to come on board without him. No joking that my job offer was “Do you want to be the Dread Pirate Roberts? I think you’d make an excellent Dread Pirate.”. He told me that it was hard work and unlike anything I’d ever done. No more CLI. No more router installations. In place of that would be event planning and video editing and taking briefings from companies all over the place about what they were building. I laughed and told him I was in.

And for the past eight years I’ve been a part of the thing that showed me that my voice mattered. As I learned the ropes to support the events and eventually started running them myself, I also grew as a person in a different way. I stopped by shy and reserved and came out of my shell. When you’re the face of the event you don’t have time to be hiding in the corner. I learned how to talk to people. I also learned how to listen and not just wait for my turn to talk. I figured out how to get people to talk about themselves when they didn’t want to.

Now the person I am is different from the nerdy kid that started a blog over ten years ago. It’s not just that I know more. Or that I’m willing to share it with people. It has now changed into getting info and sharing it. It’s about finding great people and building them up like I was built up. Every time I see someone come to the event for the first time I’m reminded of me all those years ago trying to figure out what I’d gotten myself into. Watching people learn the same things I’ve learned all over again warms my heart and shows me that we can change people for the better by showing them what they’re capable of and that they matter.


Tom’s Take

Tech Field Day isn’t an event of thousands. It’s personal and important to those that attend and participate. It’s not going to stop global warming or save the whales. Instead, it’s about the people that come. It’s about showing them they matter and that they have a voice and that people listen. It’s about helping people grow and become something they may not even realize they’re capable of. I know I sound biased because the pay the bills but even if I didn’t work there right now I would still be thankful for my time as a delegate and for the way that I was able to grow from those early days into a better member of the community. My life was changed when I got on that airplane ten years ago and I couldn’t be happier.

Solutions In Search of a Problem

During a few recent chats with my friends in the industry, I’ve heard a common refrain coming up about technologies or products being offered for sale. Typically these are advanced ideas given form that are then positioned as products for sale in the market. Overwhelmingly the feedback comes down to one phrase:

This is a solution in search of a problem.

We’ve probably said this a number of times about a protocol or a piece of hardware. Something that seems to be built to solve a problem we don’t have and couldn’t conceive of. But why does this seem to happen? And what can we do to fix this kind of mentality?

Forward Looking Failures

If I told you today that I was creating software that would revolutionize the way your autonomous car delivers music to the occupants on their VR headsets you’d probably think I was crazy, right? Every one of the technologies I mentioned in the statement is a future thing that we expect may be big down the road. We love the idea of autonomous vehicles and VR headsets and such.

Now, let’s change the statement. I’m working on a new algorithm for HD-DVD players to produce better color accuracy on plasma TVs that use PowerPC CPUs. Hopefully that statement had you giggling a little no matter what your tech level. What’s the difference? Well, that statement was loaded with technology that no one uses any more. HD-DVD lost a format war against Blu-Ray. Plasma TVs are now supplanted by LCD, LED, and even more advanced things. PowerPC has been replaced with RISC architecture and more modern takes on efficient CPUs in mobile devices.

If you’d have bet on the second combination of things back in the heyday of those technologies you might have made yourself a bit of money. You’d ultimately find yourself without a product to sell now, though. Because technology always changes. Even the dominant form of tech eventually goes away. Blu-Ray may have beat HD-DVD but it couldn’t stop streaming services. LCD replaced plasma but now we’re moving beyond that tech into OLED and even more advanced stuff. You can’t count on tech staying the same.

Which leads to the problem of trying to create solutions for problems that haven’t happened yet or are so far out on the horizon that you may not be able to create a proper solution for it. Maybe VR headsets will have great software that doesn’t need a new music match algorithm. Maybe the passengers in your autonomous vehicle won’t wear VR headsets. Perhaps music as we know it will change and not even be as relevant in the future. There’s no telling which butterfly effects will impact what you’re trying to accomplish.

Solve the Easy Things

Aside from the future problems you hope to be solving with your fancy new product you also have to take into account human behavior. Are people more likely to buy something to solve an issue they don’t currently have? Or are they more apt to buy something to solve a problem they have now? Startups that are looking five years into the future are going to stumble over the problems people have today on their way to the perfect answer to a question no one has asked yet.

I wanted a tablet because it was cool when they first came out. After using one for a few weeks I realized that it was a solution that didn’t address my pressing issues. I didn’t need what it offered at the time. Today a tablet solves many other issues that have come up since then, such as note taking or having quick access to information away from my desk. However, those problems needed to develop over time instead of hoping that my solution would work for something I couldn’t anticipate. I didn’t need a word processor for my tablet because I wouldn’t by typing much with an on-screen keyboard. Today I write a lot on my tablet because of the convenience factor. I also take notes because I have a pencil to write with instead of my fingers.

Solving problems people have right now is a sure fire way to make your customers happy and give you the breathing room to look to the future. How many times have you seen a startup with a great idea that ends up building something mundane because they can’t build the first thing right or they realize the market isn’t quite there yet?

I can remember specifically talking to Guardicore when they were first out of stealth and discussing how their SDN-based offensive security systems worked. It was amazing stuff with very little market. When they looked around and realized they needed to switch it up they went full-on into zero trust security and microsegementation. They took something that could be a great solution later on and pivoted to solving problems that people have right now. The result is a healthy company that makes things people want to buy instead of trying to sell them a solution for a problem they may never have.

If you are looking at the market and thinking to yourself, “I need to build X because it will revolutionize the way we do things” stop and ask yourself how we get there. What steps need to be taken? Who will buy it and when? Are there problems along the way? If the answer to the last question is anything other than “no” you need to focus on those problems first. You may find that you don’t need to build your fancy new vision of perfect future success because you solved all the other problems people needed fixed first. Your development efforts will be rewarded with customers and income instead of the perfect solution no one wants to buy.


Tom’s Take

Solutions without problems to solve are a lot like one-off kitchen gadgets. I may have a use for an avocado slicer twice a year. I also have a knife that does the exact same thing a little slower that I can use for many other problems around my house. I don’t need the perfect avocado slicing solution for the future when I’m making guacamole and avocado toast every day. I need a solution that gets my problems of slicing, chopping, dicing, and cutting done today. Technology is no different. Build what solves problems now and you’ll be a success. Build for the future if and only if you have the disposable time and income to get there.

The Double-Edged Grindstone

Are you doing okay out there? I hope that you’re well and not running yourself thin with all the craziness still going on. Sometimes it seems like we can’t catch a break and that work and everything keep us going all the time. In fact, that specific feeling and the resulting drive around it is what I wanted to talk about today.

People have drive. We want to be better. We want to learn and grow and change. Whether it’s getting a faster time running a 5K or learning new skills to help our career along. Humans can do amazing things given the right motivation and resource availability. I know because I taught myself a semester of macroeconomics in a Waffle House the night before the final exam. Sure, I was groggy and crashed for a 10-hour nap after the final but I did pass!

It’s that kind of ability to push ourselves past our limits that both defines us and threatens to destroy us. I’m a huge fan of reading and fiction. Growing up I latched on to the Battletech novels, especially those written by Michael A. Stackpole. In his book Lost Destiny there is a great discussion about honing your skills and what it may end up costing you:

Were we to proceed, it would be the battle of the knife against the grindstone. Yes, we would get sharper, we would win great victories, but in the end, we would be ground away to nothing.

 

Running in Place

That quote hit home for me during my CCIE lab attempt training. I spent my free time after work labbing everything I could get my hands on. I would log on about 8pm every night after my eldest went to sleep and lab until midnight. Every night was the same chore. No time for television or reading anything other than a Cisco Press book. Instead, I drilled until I could provision EtherChannels in my sleep and could redistribute routes without a second thought in four different ways. I felt like I was getting so much accomplished!

I also felt tired all the time. I had no outlet to relax. I spent every waking minute of my free time focused on sharpening my skills. As above, you could practically hear the edge of the knife on the grindstone. I was razor-focused on completing this task. As I learned later in life, the joys of hyper focus in ADHD had a lot to do with that.

It wasn’t until I got through my lab that the true measure of this honing process hit home. I spent the weekend after my attempt hanging out with some friends away from technology and the whole time I felt unsettled. I watched horrible movies on the Sci-Fi channel and couldn’t get comfortable. I was antsy and wound up, even though I’d just completed a huge milestone! It wasn’t until about a week later that I was finally able to put a name to this anxiety. I listened to the little voice in my head repeating over and over again in my downtime: “You really should be studying something right now.”

I was in search of my next grindstone. My knife was sharp, but I knew it could be sharper still with the next certification or piece of knowledge. It took me a while before I could quiet that voice and focus on restoring some semblance of my life as I had known it before my lab attempts. Even now there are times when I feel like I should be studying or writing or creating something instead of unwinding with a book or a camping trip to the woods.

The rush of dopamine that we get from learning new things or performing skills to perfection cannot be understated. It makes us feel good. It makes us want to keep doing it to continue that stream of good feelings. We could focus on it to the detriment of our other hobbies or our social lives. And that was in the time before when we could go out whenever we wanted! In the current state of the pandemic it’s easy to get wrapped up in something without the ability to force yourself to walk away from it, as anyone with a half-filled room full of a new hobby can tell you.

Setting Goals with Limits

What’s the solution then? Do we just keep grinding away until there’s nothing left? Do we become the best left-handed underwater basket weaver that has ever existed? Do we keep forcing ourselves to run the same video game level over and over again until we’re perfect, even if that means not getting out of our house for weeks at a time? Can we do something repeatedly until we are destroyed by it?

The key is to set goals but to make them in such a way as to set limits on them. We do this all the time in the opposite direction. When we have a task we don’t like to do we force ourselves to do it for a set amount of time. Maybe it’s an hour of reconciling the checkbook or thirty minutes of exercise. But if it’s something you enjoy you have to set limits on it as well to avoid that burnout.

Find something you really like to do, such as reading a book. However, instead of devouring that book until it’s finished in one sitting and staying up until 4am to finish it, set an alarm to cut yourself off after an hour or two. Be honest with yourself. When the alarm goes off, stop reading and do something else. You could even pair it with a task you like less to use the little dopamine boost to help you through your other activity.

It’s not fun to stop doing something we like doing even when it’s something that is going to make us a better person or better employee. However, you’ll soon see that having that extra time to reinforce what you’ve learned or collect your thoughts does more for you that just binging something until you’re completely exhausted by it.

Remember when TV shows came out weekly? We had to wait until next Thursday for the new episode? Remember how much you looked forward to that day? Or maybe even the return of the new season in the fall? That kind of anticipation helps motivate you. Being able to consume all the content in one sitting for 8-10 hours leaves you feeling great at first. Later, when your dopamine goes back to normal you’re going to feel down. You’ll also realize you can’t get the same hit again because you have consumed your current resource of it. By limiting what you can do at one time you’re going to find that you can keep that great feeling going without burning yourself out.

Yes, this does absolutely apply to studying for things. Your brain needs time to lock in the knowledge. If you’ve ever tried to memorize something you know that you need to spend time thinking of something else and come back to that item before you really learn it. The only thing that transfers knowledge from short-term memory to long-term memory is time. There are no shortcuts. And the more you press the edge of the knife agains the stone the more you lose in the long run because the available resources are just gone.


Tom’s Take

I’m not qualified to delve into the psychoanalysis part of all this stuff. I just know how it works for me because that’s who I am. I can shut out the world and plow through something for hours at a time if I want. I’ve done it many times in my career with both work and personal tasks. But it’s taken a long time for me to finally realize that sprinting like that for extended periods of time will eventually wear you away to nothing. Even the best runners in the world need to rest. Even the smartest people in the industry need to not think about things for a while. You do too. Take some time today or tomorrow or even next week to set goals and limits for yourself. You’ll find that you enjoy the things you do and learn more with those limits in place and you’ll wind up a happier, healthier person. You’ll be sharp and ready instead of a pile of dust under the grindstone.

Planning For The Worst Case You Can’t Think Of

Remember that Slack outage earlier this month? The one that happened when we all got back from vacation and tried to jump on to share cat memes and emojis? We all chalked it up to gremlins and went on going through our pile of email until it came back up. The post-mortem came out yesterday and there were two things that were interesting to me. Both of them have implications on reliability planning and how we handle the worst-case scenarios we come up with.

It’s Out of Our Hands

The first thing that came up in the report was that the specific cause for the outage came from an AWS Transit Gateway not being able to scale fast enough to handle the demand spike that came when we all went back to work on the morning of January 4th. What, the cloud can’t scale?

The cloud is practically limitless when it comes to resources. We can create instances with massive CPU resources or storage allocations or even networking pipelines. However, we can’t create them instantly. No matter how much we need it takes time to do the basic provisioning to get it up and running. It’s the old story of eating an elephant. We do it one bite at a time. Normally we tell the story to talk about breaking a task down into smaller parts. In this case, it’s a reminder that even the biggest thing out there has to be dealt with in small pieces as well.

Slack learned this lesson the hard way. Why? Because they couldn’t foresee a time when their service was so popular that the amount of traffic rushing to their servers crushed the transit gateways. Other companies have had to learn this lesson the hard way too. Disney crawled on launch day because of demand. The release of a new game that requires downloading and patching on Day One also puts stress on servers. Even the lines outside of department stores on Black Friday (in less pandemic-driven years) are examples of what happens when capacity planning doesn’t meet demand.

When you plan for your worst case scenario, you have to think the unthinkable. Instead of asking yourself what might happen if everyone logs on at the same time you also need to ask when happens if they try and something goes wrong. I spent a lot of time in my former job thinking about simple little exercises like VDI boot storms, where office workers can push a storage system to the breaking point by simply turning all their machines on at the same time. It’s the equivalent of being on a shared network resource like a cable modem during the Super Bowl. There aren’t any resources available for you to use.

When we plan for capacity, we have to realize that even our most optimistic projections of usage are going to be conservative if we take off or go viral. Rather than guessing what that might be, take an hour every six months and readjust your projections. See how fast you’re growing. Plan for that crazy scenario where everyone decides to log on at the same time on a day where no one has had their coffee yet. And be ready for what happens when someone throws a wrench into the middle of the process.

What Happens When It Goes Wrong?

The second thing that came up in the Slack post-mortem that is just as worrisome was the behavior of the application when it realized there was a connection timeout. The app started waiting for the pathway to be open again. And guess what happened when AWS was able to scale the transit gateway? Slack clients started hammering the servers with connection requests. The result was something akin to a Distributed Denial-of-Service (DDoS) attack.

Why would the Slack client do this? I think it has something to do with the way the developers coded it and didn’t anticipate every client trying to reconnect all at once. It’s not entirely unsound thinking to be honest. How could we live in a world where every Slack user would be disconnected all at once? Yet, we do and it did and look what happened?

Ethernet figured this part out a long time ago. The CSMA/CD method for detecting collisions on a layer 2 connection has an ingenious solution for what happens when a collision is detected. Once it realizes that there was a problem on the wire it stops what is going on and calculates a random backoff timer based on the number of detected collisions. Once that timer has expired it attempts to transmit again. Because there has to be another station involved in a collision incident both stations do this. The random element of the timer calculation ensures that the likelihood of both stations choosing to transmit again at the same time is very, very low.

If Ethernet behaved like the Slack client did we would never resolve collisions. If every station on a layer 2 network immediately tried to retransmit without a backoff timer the bus would be jammed constantly. The architects of the protocol figured out that every station needs a cool off period to clear the wire before trying again. And it needs to be different for every station so there is no overlap.

Slack really needs to take this idea into account. Rather than pouncing on a connection as soon as it’s available there needs to be a backoff timer that prevents the servers from being swamped. Even a few hundred milliseconds per client could have prevented this outage from getting as big as it did. Slack didn’t plan beyond the worst case scenario because they never conceived of their worst case scenario coming to pass. How could it get worse than something we couldn’t imagine happening?


Tom’s Take

If you design systems or call yourself a reliability engineer, you need to develop a hobby of coming up with disastrous scenarios. Think of the worst possible way for something to fail. Now, imagine it getting worse. Assume that nothing will work properly when there is a recovery attempt. Plan for things to be so bad that you’re in a room on fire trying to put everything out while you’re also on fire. It sounds very dramatic but that’s how bad it can get. If you’re ready for that then nothing will surprise you. You also need to make sure you’re going back and thinking of new things all the time. You never know which piece is going to fail and how it will impact what you’re working on. But thinking through it sometimes gives you an idea of where to go when it all goes wrong.