Monthly Archives: March 2022

A Modest Proposal for Cisco Interface Naming

Posted on by
4

If you’re going to be configuring an interface in a switch, which one are you going to use? The interface has a name and a number based on where it is on the device. The numbering part is fairly easy to figure out. The module number comes first, followed by the slot, and finally the port. In the world of Cisco, which is the one I’m the most familiar with, that means a fixed configuration switch usually has interfaces labeled 0/24, with no module and the slot almost always being zero. With a modular switch the interface would be labeled 2/0/28 to indicate the 28th port on the second line card.

The issue arises when you factor in the first part of the interface naming convention. The nomenclature used in the Cisco world since the beginning of time has been the interface speed. If your interface is a 100Mbit Ethernet interface then the interface name is FastEthernet0/48. If you’re using a 1Gbit interface it’s GigabitEthernet0/48. If it’s a 10Gbit interface it becomes TenGigabitEthernet0/48. It’s a progression of interface speeds. Even if the port is capable of using 10/100/1000 the port is referred to at the highest speed. The 10Gbit ports don’t negotiate to a lower speed so they are always TenGigabit.

Now, with the new Catalyst 9000X Series switches you have the option of a port that can do 10/100/1000/2.5G/5G/10G. It’s a multi-gigabit capable port as outlined in this video from a recent Field Day event:

I don’t even want to think about the nightmare of trying to remember the name of the interface in a situation like that. It’s hard enough still to remember what the switches are capable of, whether it’s a gigabit port or a ten gigabit port. It’s hard enough for me but it’s impossible for anyone looking to make a script or automation that needs to be repeatable over a wide variety of switches.

Naming Convention Unity

The obvious solution here is actually present in another Cisco platform already. The Nexus series of switches running Cisco NX-OS already knows how to handle interface naming issues. It’s a problem that occurs when you could be running Gigabit Ethernet or Fibre Channel over Ethernet interfaces.

How does NX-OS fix this? The interface type is always Ethernet. It doesn’t matter what speed the interface is running at because it’s always Ethernet under the hood. So the every interface is labeled Ethernet(module/slot/port). It’s wonderful when you’re working on the device because you never have to make a typo based on the interface speed. It’s also consistent across every platform and module that could be installed.

The solution to the problem actually came from my friend Bruno Wollmann (@WollmanBruno) during the event. It’s well past time to adopt the NX-OS convention in every other version of Cisco IOS. You can alias the old interface names to the new nomenclature to ease the transition if you’d like. Otherwise it’s time to start making people use the basic convention of calling everything Ethernet.

This would ease new users into IOS by making all interfaces the same and focusing on the module/slot/port complexity as the primary place to learn about naming. It also helps with the development of automation and orchestration scripts. Rather than needing to detect device types or write different scripts for NX-OS or IOS-XE you can just use the same logic for both.

Tom’s Take

It’s time we move into the future of interface names. Bruno made an excellent point when he said that it’s time to adopt a standard across the whole line of switches. Cisco may not be ready to do a single OS but having unity between the features will go a long way to making it easier to work on the various platforms. You can do a lot with aliasing in the system to ease the transition but maybe by the time we get to IOS 20 we can use something that’s more manageable for the future.

No Is A Complete Sentence

Posted on by
5

Has someone asked you to do something recently that you know you don’t have time to do but felt like you needed to do anyway? Or has someone tried to get you to help with something and impressed upon you just how important it is? You probably told them “yes” out of guilt or obligation or some other kind of negative emotion. Sure, you could have declined but you thought about how bad you would feel if someone did the same to you.

Let me tell you clearly. “No” is a complete sentence. It requires no explanation or defense. It is the only thing you need to say when you know you won’t be able to do something no matter how much the other party tries to get you to agree.

Everything Sucking Equally

If you know anything about QoS, you know that once a given circuit reaches the limitation for bandwidth you can no longer send additional information. What’s counterintuitive about this is most people would assume that if you try to squeeze one more stream or packet into the mix that only that last packet would be affected and everything else would work perfectly fine, right? Only one of the many would suck.

Sadly, all of the packets or flows on the circuit are affected in this situation. Adding just one more thing to the mix means all things are affected because the system has to process the problems all at once. You suffer everywhere because the interface couldn’t say “no” to that last packet.

In fact, the whole mantra behind QoS is prioritizing information before it hits a given interface because the interface is just going to send whatever it is given. Those packets will queue up or eventually be dropped if they can’t be processed. Likewise, if you are trying to do too many things at once you will eventually miss something important and make a mistake because you couldn’t handle everything at once. You need to be able to prioritize all the things that are going on and know when you hit a limit you can’t get past.

Where the challenge comes into play is when others try to add more to the interface without knowing you’re at maximum capacity. It’s easy for someone to ask you to do just one more little thing. In their minds it’s not a huge ask, right? In your mind you’ve already begun trying to juggle all the things that need to be changed to make this one little thing work with all the other things you’ve spent so much time prioritizing. Unfortunately, it’s too easy to just say “yes” to avoid uncomfortable conversations and try to fix the problems as they come up rather than turning away something you know would put you over your limit.

Taking No For an Answer

It’s easy to tell people to say “no”, right? I mean it is possibly the shortest sentence in the English language. However, it’s not the person saying it that usually has an issue. The people doing the asking are the ones that almost invariably are the ones trying to convince, cajole, or outright impress upon the person how critical it is that whatever this is get done in addition to all the other things that are going on.

For those that are asking the question and expecting a positive answer, let me ask you a question of my own: Why does this person need to do the task? There usually are very good reasons for it. Perhaps it’s something this person specializes in. Maybe it’s something that only they have knowledge about. It could perhaps even be their job responsibility to get it done. But the next question you need to ask is even more important.

If they must do this task, what can I do to help them?

The calling card of bad management types is to show up, assign a bunch of tasks that only other people should do, and then fly away to measure their productivity and comment on how nothing seems to be getting done. Instead, management should be asking how they can help. Is this something that someone else could do? Do you have the bandwidth to do the task? Is there another task on your plate that I can do for you that would help you to take care of this?

People are often very fully tasked with all the things they have to take care of. Thinking that you can sneak one more thing onto their schedule without impacting all of it smacks of hubris. What if someone started adding tasks to your plate instead? I doubt you’d like that. Even if you said no, what if it was the CEO or the Chairman of the Board telling you that you had to do it? You’d probably be just as upset, especially if it was a task you couldn’t delegate.

For the people out there like me, try this next time someone asks you to do something and you’re fully tasked or unable to clear enough to accomplish it. Say “no”. Don’t explain. Don’t justify. Don’t defend. Just. Say. No. Watch the other person’s reaction. Are they getting frustrated because you’re not saying anything? Are they coming back in a helpful manner trying to understand why you declined? Or are they more upset that they now have to go find someone else to do it instead? If they come to you with respect and understanding you can tell them why you declined and are unable to help. They may offer suggestions or try to find a way to make it work. The ones that get upset are usually the ones that wouldn’t have taken “no” for an answer no matter what. They are the ones you don’t need to waste your breath on.

Tom’s Take

I will admit this post is as much for me as it is for others. I am the world’s worst for taking on additional things when I know I don’t have the bandwidth to do it all. I don’t like to disappoint. I really want to keep people happy. And instead I end up failing because I overestimate my ability to get work done and underestimate how much others could do if they knew I needed help. Next time you need to tell someone you can’t do something, don’t spend a ton of effort figuring out how to counter their eventual arguments. Just say “no” and move on. If it’s important or something that only you can do they will find a way to make it work. Otherwise, you will have saved so much more effort because you used a complete sentence instead of a needless paragraph.

Trust Will Do You In

Posted on by
1

If you’re a fan of the Gestalt IT Rundown that I do every week on the Gestalt IT YouTube channel, you have probably heard about the recent hacks of NVIDIA and Samsung. The original investigation into those hacks talked about using MDM platforms and other vectors to gain access to the information that was obtained by the hacking groups. An interesting tweet popped up on my feed yesterday that helped me reframe the attacks:

It would appear that the group behind these attacks are going after their targets the old fashioned way. With people. For illustration, see XKCD from 2009:

The Weakest Links

People are always the weakest link in any security situation. They choose to make something insecure through bad policy or by trying to evade the policy. Perhaps they are trying to do harm to the organization or even try to shine a light on corrupt practices. Whatever the reason, people are the weak link. Because you can change hardware or software to eliminate failures and bugs. You can’t reprogram people.

We’ve struggled for years to keep people out of our systems. Perimeter security and bastion hosts were designed to make sure that the bad actors stayed off our stage. Alas, as we’ve gotten more creative about stopping them we’ve started to realize that more and more of the attacks aren’t coming from outside but instead from inside.

There are whole categories of solutions dedicated to stopping internal attackers now. Data Loss Prevention (DLP) can catch data being exfiltrated by sophisticated attackers but it is more often used to prevent people from leaking sensitive data either accidentally or purposefully. There are solutions to monitor access to systems and replay logs to find out how internal systems folks were able to get privileges they should have.

To me, this is the crux of the issue. As much as we want to create policies that prevent people from exceeding their authority we seem to have a hard time putting them into practice. For every well-meaning solution or rule that is designed to prevent someone from gaining access to something or keep them secure you will have someone making a shortcut around it. I’ve done it myself so I know it’s pretty common. For every rule that’s supposed to keep me safe I have an example of a way I went around it because it got in my way.

Who Do YOU Trust?

This is one of the reasons why a Zero Trust Network Architecture (ZTNA) appeals to me. At its core it makes a very basic assumption that I learned from the X-Files: Trust No One. If you can’t verify who you are you can’t gain access. And you only gain access to the bare minimum you need and have no capability for moving laterally.

We have systems that operate like this today. Call your doctor and ask them a question. They will first need to verify that you are who you say you are with information like your birthdate or some other key piece of Personally Identifiable Information (PII). Why? Because if they make a recommendation for a medication and you’re not the person they think they’re talking to you can create a big problem.

Computer systems should work the same way, shouldn’t they? We should need to verify who we are before we can access important data or change a configuration setting. Yet we constantly see blank passwords or people logging in to a server as a super user to “just make it easier”. And when someone gains access to that system through clever use of a wrench, as above, you should be able to see the potential for disaster.

ZTNA just says you can’t do that. Period. If the policy says no super user logins from remote terminals they mean it. If the policy says no sensitive data access from public networks then that is the law. And no amount of work to short circuit the system is going to work.

This is where I think the value of ZTNA is really going to help modern enterprises. It’s not the nefarious actor that is looking to sell their customer lists that creates security issues. It does happen but not nearly as often as an executive that wants a special exception for the proxy server because one of their things doesn’t work properly. Or maybe it’s a developer that created a connection from a development server into production because it was easier to copy data back and forth that way. Whatever the reasons the inadvertent security issues cause chaos because they are configured and forgotten. At least until someone hacks you and you end up on the news.

ZTNA forces you to look at your organization and justify why things are the way they are. Think of it like a posture audit with immediate rule creation. If development servers should never talk to production units then that is the rule. If you want that to happen for some strange reason you have to explicitly configure it. And your name is attached to that configuration so you know who did it. Hopefully something like this either requires sign off from multiple teams or triggers a notification for the SOC that then comes in to figure out why policy was violated. At best it’s an accident or a training situation. At worst you may have just caught a bad actor before they step into the limelight.

Tom’s Take

Security isn’t perfect. We can always improve. Every time we build a good lock someone can build a better way to pick it. The real end goal is to make things sufficiently secure that we don’t have to worry about them being compromised with no effort while at the same time keeping it easy enough for people to get their jobs done. ZTNA is an important step because it creates rules and puts teeth behind them to prevent easy compromise of the rules by people that are trying to err on the side of easy. If you don’t already have plans to include ZTNA in your enterprise now you really should start looking at them. I’d tell you to trust me, but not trusting me is the point.

AI is a Promotion

Posted on by
1

When I worked at IBM as an intern, part of my job was writing a deployment script to help make our lives easier when installing new ThinkPads. In order to change an MTU setting on the token ring PCMCIA cards (long story), I had to write a script that iterated through all the possible combinations of adapters in the registry to find the one I was looking for and change the value.

Now, I was 22 at the time and green behind the ears, especially when it came to programming. I finally figured out that the most efficient way to do this in the language that I was using was a very deep nested if statement. It wasn’t my best work but it operated properly. I mentioned this to my mentors on my team with a remark of how hard it was to understand the logic at first. My comment was “You know, if it’s hard to read for anyone else then I never have to worry about gettin fired.”

To which the response was, “Yes, but you can never be promoted either.”

That sage wisdom brings me to the modern world and how AI can fix that issue.

Eternal Support

One of the first things you learn in IT is not to volunteer. If you know how to implement a technology or troubleshoot a specific issue then you become the go-to person for that problem. If you add a skill to the team like debugging Rust or configuring voice mail dial peers then you are the new owner for that particular skill.

This becomes annoying when you finally move past that skillset and you’re on to new and better things. If you are the person that knows how to extend database schemas or change auto attendant functions you are always going to be called upon to do those things. Rather than moving on to new challenges you will be called back to work on the things that no one else bothered to learn or couldn’t pawn off on someone else.

When you think about it, that’s part of the reason why so many entry-level positions start with some basics like VLAN programming or LUN configuration. Sure, you’re learning the basics and understanding how systems operate. But you’re also repeating the things that the junior person before you did. Why? Because they now have someone to do that work for them. And when the new person gets hired as a junior-level technician, you’re going to pass along those tasks to someone so they can learn and so you can move on. It’s the Circle of IT Life.

The reinforcement of knowledge is only so important to the organization. Yes, you need someone to keep the lights on and do the tasks regarded as menial by the senior teams. However, miring people in those tasks makes them feel undervalued after a while because no one wants to take them on. And, as above, if it’s something that no one else can be bothered to learn or take over you’re going to be stuck doing it until you leave or until the system in question is eventually replaced. And even then you’ll probably be asked to learn how to do the new thing because “this is your system”.

Learning, Not Thinking

The reason why I’m excited for applied AI in infrastructure and operations is because it will learn these menial tasks and do them as instructed every time without complaint or mistake. Yes, that means you need to train the algorithm correctly. Yes, that means you need to be on the lookout for changes and update accordingly when hardware changes or when procedure changes. Given those constraints AI will work for you, not against you.

When you move on to a new role you expect to face new challenges. If you move from a junior role to a senior role you want to work on harder things or solve new problems. You don’t want to get a new title with the same old work. If you change teams you want to experience new systems and implement new technology. You don’t want to be the cloud person that still deploys campus switches in wiring closets.

In a way, having AI take over the basic tasks of your job functions doesn’t remove your job. It promotes you to a new role. It could be that the first promotion is running the system that you implemented to do the work. A “train the trainer” kind of role, if you will. As soon as you can ensure that it is working correctly and that the rest of the team is aware of how to keep the engine running you can decide where you want to go next. You essentially have the freedom to learn and do and not have to be concerned about going back to doing the old things.

Is AI the Enemy?

AI doesn’t take away jobs. It takes away tasks. It streamlines processes and provides consistent, repeatable outcomes. If your job is a collection of tasks that need to be done then it is worth asking why it’s so easy for it to be replaced by an AI system. I’ve said on a number of occasions that there will always been room for jobs that AI can’t do, such as those that involve physical labor, as well as advanced thinking and creative problem solving.

Where AI does succeed is in the repeatable task department. If a role is comprised of repeatable tasks then the question becomes “why does this role exist?” If an algorithm can determine the best way to implement VLANs or offload clients to other devices then the job that was doing those roles needs to be examined. Can that person be encourage to do something else for the company? Can that role be reduced and given to someone with less experience? If those questions make you or someone on your team bristle, is that the fault of the AI? Or the fact of the job being less than ideal for the business?

Tom’s Take

AI can take away some of the unimportant and forgettable parts of your job and give you the freedom to do things that are more important to the business, possibly in a different role. Isn’t that the textbook definition of a promotion? Change makes people uncomfortable. It makes sense that no one wants to have their job taken away. But if you ask anyone in IT right now if they could get rid of 10-20% of the tasks in their job that they don’t like to do they’d agree in a heartbeat. AI isn’t going to remove your job. It’s going to give you the freedom to do something different. It’s also going to ensure you don’t have to keep getting called back to do that thing over and over again because no one else wants to learn it. I think it’s time we let new technologies like AI give everyone on the team the promotion they deserve.