Choosing the Least Incorrect Answer

My son was complaining to me the other day that he missed on question on a multiple choice quiz in his class and he got a low B grade instead of getting a perfect score. When I asked him why he was frustrated he told me, “Because it was easy and I missed it. But I think the question was wrong.” As usual, I pressed him further to explain his reasoning and found out that the question was indeed ambiguous but the answer choices were pretty obviously wrong all over. He asked me why someone would write a test like that. Which is how he got a big lesson on writing test questions.

Spin the Wheel

When you write a multiple choice test question for any reputable exam you are supposed to pick “wrong” answers, known as distractors, that ensure that the candidate doesn’t have a better than 25% chance of guessing the correct answer. You’ve probably seen this before because you took some kind of simple quiz that had answers that were completely wrong to the point of being easy to pick out. Those quizzes are usually designed to be passed with the minimum amount of effort.

This also extends to a question that includes answer choices that are paired. If you write a question that says “pick the three best answers” with six options that are binary pairs you’re basically saying to the candidate “Pick between these two three times and you’re probably going to get it right”. I’ve seen a number of these kinds of questions over the years and it feels like a shortcut to getting one on the house.

The most devious questions come from the math side of the house. Some of my friends have been known to write questions for their math tests and purposely work the problem wrong at a critical point to get a distractor that looks very plausible. You make the same mistake and you’re going to see the correct answer in the choices and get it wrong. The extra effort here matters because if you see too many students getting the same wrong distractor as the answer you know that there may be confusion about the process at that critical point. Also, the effort to make math question distractors look plausible is impressive and way too time consuming.

Why Is It Wrong?

Compelling distractors are a requirement for any sufficiently advanced testing platform. The professionals that write the tests understand that guessing your way through a multiple choice exam is a bad precedent and the whole format needs to be fair. The secret to getting the leg up on these exams is more than just knowing the right answer. It’s about knowing why things are wrong.

Take an easy example: OSPF LSAs. A question may ask you about a particular router in a diagram and ask you which LSAs that it sees. If the answer choices are fairly configured you’re going to be faced with some plausible looking answers. Say the question is about a not-so-stubby-area (NSSA). If you know the specifics of what makes this area unique you can start eliminating choices from the question. What if it’s asking about which LSAs are not allowed? Well, if you forgot the answer to that you can start by reading the answer choices and applying logic.

You can usually improve your chances of getting a question right by figuring out why the answers given are wrong for the question. In the above example, if LSA Type 1 is listed as an answer choice ask yourself “Why is this the wrong answer?” For the question about disallowed LSA types you can eliminate this choice because LSA Type 1 is always present inside an area. For a question about visibility of that LSA outside of an area you’d be asking a different question. But if you know that Type 1 LSAs are local and always visible you can cross off that as a potential answer. That means you boosted your chances of guessing the answer to 33%!

The question itself is easy if you know that NSSAs use Type 7 LSAs to convey information because Type 5 LSAs aren’t allowed. But if you understand why the other answers are wrong for the question asked you can also check your work. Why would you want to do that? Because the wording of the question can trip you up. How many times have you skimmed the question looking for keywords and missing things like “not” or “except”? If you work the question backwards looking for why answers are wrong and you keep coming up with them being right you may have read the question incorrectly in the first place. Likewise, if every answer is wrong somehow you may have a bad question on your hands.

What happens if the question is poorly worded and all the answer choices are wrong? Well, that’s when you get to pick the least incorrect answer and leave feedback. It’s not about picking the perfect answer in these situations. You have to know that a lot of hands touch test questions and there are times when things are rewritten and the intent can be changed somehow. If you know that you are dealing with a question that is ambiguous or flat-out wrong you should leave feedback in the question comments so it can be corrected. But you still have to answer the question. So, use the above method to find the piece that is the least incorrect and go with that choice. It may not be “right” according to the test question writer, but if enough people pick that answer you’re going to see someone taking a hard look at the question.


Tom’s Take

We are going to take a lot of tests in our lives. Multiple choice tests are easier but require lots of work, both on the part of the writer and the taker. It’s not enough to just memorize what the correct answers are going to be. If you study hard and understand why the distractors are incorrect you’ll have a more complete understanding of the material and you’ll be able to check your work as you go along. Given that most certification exams don’t allow you to go back and change answers once you’ve moved past the question the ability to check yourself in real time gives you an advantage that can mean the difference between passing and retaking the exam. And that same approach can help you when everything on the page looks wrong.

A Decade of CCIE Certification

I was notified this week that I’m eligible for the 10-year CCIE plaque. Which means that it’s been a decade since I walked out of Cisco’s Building C in San Jose with a new number and a different outlook on my networking career. The cliche is that “so many things have changed” since that day and it’s absolutely accurate because the only constant in life is change.

Labbing On the Road

I think the first thing that makes me think about the passage of time since my certification is the fact that the lab where I took the exam no longer exists. Building C was sold to the company that owns and operates the San Francisco 49ers stadium just down Tasman drive from the old letter buildings. Those real estate locations were much more valuable to the NFL than to Cisco. I can’t even really go and visit my old stomping grounds any more because the buildings were gutted, renovated, and offered to other operations that aren’t from Cisco.

Now, you don’t even go to San Jose or RTP for the lab. Three years ago the labs in the US moved to Richardson, TX. The central aspect of the location is pretty appealing when you think about it. A part of me wishes I would have had the opportunity to take the lab there since I wouldn’t have to jump on a plane and burn three days of my work schedule. The costs of my lab attempts would have been a lot less if I only had to drive down for one night in a hotel and got to come back and sleep in my bed that same night. I realize that it’s equally inconvenient for people to need to fly to the middle of the country when they used to be closer to the lab when it was on either coast. However, real estate in RTP and San Jose is beyond crazy when it comes to price. Moving the lab to somewhere more reasonable means Cisco is getting value out of their buildings elsewhere.

The mobile lab is another aspect of the changes in the CCIE certification program that are a welcome change. By putting the lab on the road and giving people in countries far away from a lab location the opportunity to get certified the program can continue to be relevant. This is due in large part to the changes in the lab that allow a large part of it to be virtualized or operated remotely from a rack located somewhere else. I remember starting my lab studies and thinking to myself that the rack that I was working on was just across the room. Not that there was much that I could do about it. The idea that there could be something going on that was just out of my reach was an itch I had to get over. Today, you would never even start to believe that you had a hardware issue in your lab because of the streamlining of the process. That can only happen when you optimize your offerings to the point where you can just virtualize the whole thing.

The Next Ten Years

Right now, I still have a year to go on my certification before I have to make the decision to keep it current or go to Emeritus retirement. My role on the CCIE Advisory council doesn’t matter either way. I’m likely going to just go Emeritus when the opportunity presents itself because I don’t use those lab skills every day. I’m not configuring BGP filter lists and port channels like I used to. The technical skills that I honed in Building C serve me more now to understand technology at an architecture level. I can see how people are using tools to solve problems and offer commentary when they are making poor decisions or when a better protocol exists.

The CCIE itself is still a very valuable certification to hold and study for. IT certification on the whole has been trending away from being the gold standard for hiring. Cloud and DevOps focus more on skills instead of papers hanging on a wall. However, operations teams still need ways to differentiate their people. If nothing else the CCIE is a great forcing function for you to figure out how deeply into networking you really want to get. It’s not enough to be curious about BGP or Frame Relay and traffic shaping QoS. You have to understand it at a level that would bore most others to tears. If you’re not prepared to know the minutia of a protocol the way that some people memorize batting averages or random movie trivia than you might not be up for this particular challenge.

The CCIE also isn’t going away any time soon. I remarked to someone the other day that the CCIE is a technology bellweather. I can remember the clamor to introduce the “new” SDN changes into the program so many years ago. I also chuckle when I think about the CCIE OpenFlow that more than a couple of people proposed. The certification program exists to refine and highlight the technology solutions that people are using today. It’s not a sneak peak at things that might be important later on in life. Think about how long it took for them to remove ISDN, ATM, and even frame relay from the test. And even frame relay was debated heavily because more than a few claimed they still used it in production.

The CCIE is a testament to the way that people study for and build networks at a high level. It’s not a cool badge to keep on your list like a hunting trophy. It’s a testament to the commitment that it takes to attain something like that. The JNCIE and the VCDX are much the same. They represent an investment of time and energy into something that proves your capabilities. More than any other certification, the CCIE challenges people. It creates study habits and builds communities. It makes people ask themselves hard questions about desire and commitment and helps the best rise to the occasion. It’s more than just a certification.


Tom’s Take

I wouldn’t change a thing about my CCIE journey. I learned as much from the failures as I did from the success. The opportunities afforded to me because of that number have been immeasurable. But through it all I realized that the process of getting my lab has helped shape me into who I am today. A decade past late night study sessions and soul-crushing failures I know that it was all worth it because it helped me take technology more seriously and form the habits and process that have served me well from then on. I’m happy to get the new plaque that marks me as a veteran of the lab plus ten years. My status as a CCIE might pass into Emeritus but the lessons I learned along the way will always be there.

Basics First and Basics Last

This week I found my tech life colliding with my normal life in an unintended and somewhat enlightening way. I went to a store to pick up something that was out of stock and while I was there making small talk the person behind the counter asked me what I did for a living. I mentioned technology and he said that he was going to college for a degree in MIS, which just happens to be the thing I have my degree in. We chatted about that for a few more minutes before he asked me something I get asked all the time.

“What is the one thing I need to make sure I pay attention to in my courses?”

It’s simple enough, right? You’ve done this before and you have the benefit of hindsight. What is the one thing that is most important to know and not screw up? The possible answers floating through my head were all about programming or analytical methods or even the dreaded infrastructure class I slept through and then made a career out of. But what I said was the most boring and most critical answer one could give.

“You need to know the basics backwards and forwards.”

Basics Training

Why do we teach the basics? Why do we even call them that? And why are people so keen on skipping over all of them so fast to get to the cool stuff? You have to understand the basics before you even move on and yet so many want to get the “easy” stuff out of the way because memorizing the OSI model or learning how an array works in programming is mind-numbing.

The basics exist because we all need to know how things work at their most atomic level. We memorize the OSI model in networking because it tells us how things should behave. Sure, TCP/IP blows it away. However, if you know how packets are supposed to work with that model it informs you how you need to approach troubleshooting and software design and even data center layouts.

I’ll admit that I really didn’t pay much attention when I took my Infrastructure class twenty years ago. I was hell-bent on being a consultant or a database admin and who needed to know how a CPU register worked? What was this stupid OSI model they wanted me to know? I’ll just memorize it for the test and be done with it. Needless to say that the intervening years have shown me the folly of not paying attention in that class. If I went back today I’d ace that OSI test with my eyes closed.

The basics seem useless because we can’t do much with them right now. They’re just like Lego bricks. We need uniform pieces with predictable characteristics to help us understand how things are supposed to work together. Without that knowledge of how things work you can’t build on it. If you don’t understand the different between RAM and a hard disk you won’t be able to build systems that rely on both. Better yet, when technology changes to incorporate solid state disks and persistent memory storage you need the basics to understand how they are different and where you need to apply that knowledge.

I once picked up a Cisco Press CCIE study guide for the written exam to brush up on my knowledge before retaking the written. The knowledge in the book seemed easy to me. It was all about spanning tree configurations and OSPF area types and what BGP keepalives were. I felt like it was a remedial text that didn’t give me any new knowledge. That’s when I realized that they knowledge in the book wasn’t supposed to be new. It was supposed to be a reminder of what I already learned in my CCNA and CCNP courses. If anything in the text was truly new, was it something I should have already known?

It’s also part of the reason the CCIE is such a fun exam in the lab. You should already know the basics of how things like RIP and OSPF work. So let’s test those basics in new ways. Any of the training lab you can take from companies like INE or Micronics are filled with tricky little scenarios that make you take the basics and apply them outside the box. That’s because the instructors don’t need to spend time teaching you how RIP forms neighbor relationships or adjacencies. They want to see if you remember how that happens so you can apply it to a question designed to stretch your knowledge. You can only do that when you know the basics.

Graduation Day

Basics aren’t just for learning at the beginning. You should also brush up on them when you’re at the top of your game. Why? Because it will answer questions you might not know you had or explain strange things that rely on the architecture we long-ago forgot about because it seemed basic.

A fun example was years ago in the online game City of Heroes. The players can earn in-game currency to buy and sell things. Eventually the game economy got to the point where players were at the maximum amount of currency for a player. What was that number? It was just over two billion. Pretty odd place to stop, right? What made them think that was a good stopping point? Random chance? Desire to keep the amount of currency in circulation low? Or was there a different reason?

That’s when I asked a simple question: How would you store the currency value in the game’s code? The answer for every programmer out there is an integer. And what’s the maximum value for an integer? For a 32-bit value it’s around four billion. But what if you use a signed integer for some reason? The maximum value is just over two billion in each direction. So the developers used a 32-bit signed integer and that’s why the currency value was capped where it was.

Over and over again in my career I find myself turning back to the basics to answer questions about things I need to understand or solve. We really want the solutions to be complex and hard to understand and solve because that shows our critical thinking skills being applied. However, if you start with the basics approach you’ll find that the solutions to problems or the root causes are often defined by something very basic that has far-reaching consequences. And if you forget how those basics work you’re going to spend a lot of time chasing your tail looking for a complex solution to a simple problem.


Tom’s Take

I don’t think my conversation partner was hoping for the answer I gave him. I’m sure he wanted me to say that this high level course was super important because it taught all the secrets you needed to know in order to succeed in life. Everyone wants to hear that the most important things are exciting and advanced. Finding out that the real key to everything is the basics you learn at the beginning of your journey is disappointing. However, for those that master the basics and remember them at every step of their journey, the end of the road is just as advanced and exciting as it was when you stepped on it in the first place. And you get there with a better understanding of how everything works.

How Long Should You Practice

A reporter once asked boxing legend Muhammad Ali how many sit-ups he did each day. I’m sure the reporter wasn’t expecting Ali’s answer. Ali replied with:

I don’t know. I don’t start counting them until it hurts. Those are the only ones that count. That’s what makes you a champion.”

Ali knew that counting things is just a numbers game. Five hundred poor sit-ups don’t count as much a fifty done the right way. With any practice that you do the only things that count are the things that teach your something or that push you to be better.

Don’t Practice Until It’s Right

People used to ask me how long I would spend at night studying for the CCIE lab. I told them I usually spent between five and seven hours depending on what I was studying. Sometimes those people would say things like “I’m not talking about setup time. I’m talking about actual lab work.” I always countered by making them explain why the setup isn’t part of the “real” work. That’s usually when they went quiet.

It’s far too easy to fall into the trap of overlooking things that you think are unimportant. A task you’ve done a hundred times is no big deal until you do it wrong the next time. Like Ali above, the things you do that require no effort don’t count. If you’re practicing a skill for a certification or a lab you need to put the same effort into it every time to ensure you’re doing it correctly. Lack of attention means you are doing it without gaining something from it.

I spend a lot of my time teaching things to people all over the place. I teach IT and networking skills to professionals. I teach outdoor skills to scouts of all ages. I teach merit badges and other things to a variety of youth. And I teach my kids life skills they will need. Every one of these lessons comes with instruction in the little details that matter. Every lesson also includes guidance that it needs to be practiced properly until it’s right. And then some.

Until You Can’t Get It Wrong

I tell my students and kids all the time, “Don’t practice until you get it right. Practice until you can’t get it wrong.” The level of involvement that it takes to get past the part where something finally works up to the level where it works every time is as wide as the gap at the lower end of the spectrum.

Too often people are content to work on something until they get it once. Whether it’s tying a knot or programming a router interface or even cooking a grilled cheese sandwich. Once you’ve done it right once you’re done with learning, right? Most of you are already shaking your head because you know that’s not right.

Once you get it right the first time you’ve already made a list of all the wrong ways to do something and you avoid them in the future. However, that list doesn’t include the entirety of all the wrong ways to do a thing. Amateurs make somewhat predictable mistakes because they’re working from the same basic knowledge. It’s when someone says they know what they’re doing that the real crazy stuff starts coming out of the woodwork.

Once you’ve practiced a skill you need to keep going. You need to work a variety of different angles to make sure you’ve covered all the ways you could get it wrong. If you’re tying a knot you need to practice with different kinds of ropes or in different positions. If there are two ways to tie something, practice them both. You don’t want to be an expert at a clove hitch over the end of a pole only to find out you have to tie it around the middle with no way to use the loop method you have memorized.

In IT, we lab things up to make sure we understand them. For these labs, try out the things in wrong ways. Click buttons before you’re supposed to. Put the wrong numbers in the field. See how the system will try to correct your errors. Maybe it doesn’t even bother? It’s easy to figure out you typed something in wrong when you hear a bell and see a message. It’s harder to troubleshoot when you don’t see anything right away and it all falls over later.

The extra practice above and beyond the first success is just like Muhammad Ali’s sit-ups. The hard ones count. The tasks that stretch your mind are the ones that build your skillset. You can’t give up when the answer isn’t right at your fingertips. Going that extra mile is the key to making yourself a better professional in whatever you do.


Tom’s Take

As we wind down 2020 we’re all looking to be better at things. Hobbies, skills, or professional talents are all calling to us to work on in whatever down time we have available to us. Make that practice count. Work hard to get it right every time. If you want to learn to make hollandaise sauce or write a novel or do a forward flip you have to keep practicing even after your success. Get to the point where you have no other choice but to get it right every single time. That’s the perfect amount of practice you need. Anything less counts as much as Muhammad Ali’s sit-ups before they start hurting.

Iron Chef: Certification Edition

My friend Joshua Williams (@802DotMe) texted me today with a great quote that I wanted to share with you that made me think about certifications:

You’ve probably already thought through this extensively, and maybe even written about it, but after sitting through another 8 hour practical exam yesterday I’m more convinced than ever that expert level exams from technical companies are more analogous to a gimmicky Food Network TV show than real world application of technical acumen. They don’t care so much about my skill level as they do about what kind of meal I can prepare in 30 minutes using Tialapia, grapes, and Dr. Pepper syrup with my salt shaker taken away halfway through.

I laughed because it’s true. And then I thought about it more and realized he’s way more than right. We know for a fact that companies love to increase the level of challenge in their exams from novice to expert. It’s a way to weed out the people that aren’t committed to learning about something. However, as the questions and tasks get harder it becomes much more difficult to get a good sense of how candidates are going to perform.

Boiling Water Isn’t Hard?

When you look at something like the CCNA, they’re trying to make sure you know how networks actually work. The simulations and lab exercises are pretty basic. Can you configure RIP correctly? Do you know the command to enable a switch port? There isn’t a need to get crazy with it. Using Joshua’s analogy from above, it’s not unlike a show like Worst Cooks in America, where the basics are the challenge that needs to be overcome. Not everyone is a superstar chef. Sometimes getting the building blocks right is more than half the battle.

As you move up the ladder, the learning gets harder. You dive deep into protocols and see how technologies build on each other. You need to configure BGP, but you also need to have some kind of other IGP running to distribute the routes. You need to remember that this spice goes in while the dish is cooking and this other goes on at the end so the flavor isn’t destroyed. I would liken this to a “fun” challenge cooking show, where the expert Food Network Chef faces off against someone that isn’t in the food business at a high professional level. Maybe they run a diner or are a short-order cook in a hotel restaurant. They aren’t looking to create their own signature dish. They know enough to cook what tastes good. But ask them to make hollandaise sauce or make pufferfish sashimi and they’re out.

Which brings us to the highest level of learning. The expert certification tracks. These are the crowing achievements of a career. They are the level that you have to be at to prove you know the technology inside and out. How do you test that, exactly? Microsoft had a great way of doing it back in the day with some of the mastery programs. You went to Redmond and you spent a couple of months learning the technology with the people that wrote it. It was very similar to a doctor’s internship in a hospital. You did the work with people that knew what you needed to know. They corrected you and helped you grown your knowledge. Even though you were an expert you understood what needed to be done and how to get there. At the end you took an exam to cover what you had learned and you earned your mastery.

Other certification programs don’t do that. Instead, they try to trip you up with tricky scenarios and make you make mistakes if you’re not paying attention. This is the Iron Chef round. You know your stuff, eh? Face off against this hard challenge. And by the way, here’s your curveball: You have to use this crazy extra ingredient. A show like Chopped does this a lot too. You need to make a meal using chicken, soy sauce, and candy corn. Are they testing your ability to prepare food? Or trying to figure out how creative you can be with a set of constraints that don’t make sense?

Ala Config!

The theory behind this kind of challenge is sound on paper. You never know what you’re going to walk into and what you’ll be forced to fix. I’ve had some real interesting problems that I’ve needed to solve over my career. But in every crazy case I never had to deal with the kinds of constrained setups that you get in lab-based exams. Configure this protocol, but don’t use these options. Make this connection work this way using one of these options but know that picking the wrong one will wreck your configuration in about two hours. Make trout-flavored ice cream. You name it and it’s a huge challenge for no good reason.

In theory, this is a great way to challenge your experts. In practice, it’s silly because you’re putting up barriers they will never see. Worse yet, you force them to start looking for the crazy constraints that don’t exist. One of my favorites is the overarching constraint in the CCIE lab that you are not allowed to use a static route to anything unless explicitly allowed in the question. Why? Because static routes don’t scale? Because they create administrative overhead? Or is it because a single static route fixes the problem and doesn’t require you to spend an hour tagging routes when redistribution happens? Static routes cut the Gordian Knot in the lab. So they can’t be allowed. Because that would make things too easy.


Tom’s Take

We need to move away from trivia and Iron Chef-style certifications. Instead of making our people dependent on silly tricks or restricting them from specific tools in their kit, we need to ensure their knowledge is at the right level. You would never ask a chef to cook an entire meal and not be able to use a saucepan. Why would you take away things like static routes or access lists from a network engineer’s arsenal? Instead of crafting the perfect tricky scenario to trap your candidates, spend the time instead teaching them what they need to know. Because once someone learns that trout is a horrible ice cream flavor we all win.

Thanks to Josh Williams for this great post idea!

The Certification Ladder

Are you climbing the certification ladder? If you’re in IT the odds are good that you are. Some people are just starting out and see certifications as a way to get the knowledge they need to do their job. Others see certs as a way to get out of a job they don’t like. Still others have plenty of certifications but want to get the ones at the top of their field. This last group are the ones that I want to spend some time talking about.

Pushing The Limit

Expert-level certifications aren’t easy on purpose. They’re supposed to represent the gap between being good at something and going above and beyond. For some that involves some kind of practical test of skills like the CCIE. For others it involves a board interview process like the VCDX. Or it could even involve a combination of things like the CWNE does with board review and documentation submissions.

Expert certifications aren’t designed to be powered through in a short amount of time. That’s because it’s difficult to become an expert at something without putting in the practice time. For some tests, that means meeting some minimum requirements. You can only attempt your VCDX when you have already passed the VCAP-DCA and VCAP-DCD test, for example. Or you may have a minimum requirement of time in the industry, such as the CISSP requirement of four years in the security industry.

But, more importantly, the requirement is that you truly are an expert. How many times have you bumped into someone that has a certification that you think to yourself, “How on earth did they pass that?” It should be fairly uncommon to run into a CCIE that you feel that way about. The test is rigorous and requires everyone to pass a very similar version of the practical exam. Sure, you still run into people that say the old 2-day exam was harder. But by and large, most CCIEs have had to endure the same kind of certification requirements.

Now, what people do after they get there is an entirely different matter altogether. There are a lot of people that get to the pinnacle of their certification journey and sit there on top of their mountain. They take time to survey the lands that they now watch over and they relax. They don’t see any need in going any further. They’ve done what they came to do. And for many that’s the way to go. Congratulations on your ride.

Still others use this opportunity negatively. They expect people to kiss the brass certificate and pay deference to them because of it. This can affect almost anyone. I remember years ago back to a time when I had just gotten my CCIE lab out of the way. I was working on a proposal for a customer. We had just gotten an email from the customer asking why we didn’t include a particular switch in the design. I told our team that we didn’t need it because the requirements of the design didn’t need something that cost three times over what we recommended. The customer’s response was, “Well, this other partner guy is a CCIE and he says we need that switch.” I replied back with, “Well, I’m a CCIE too, so let’s cut that crap and talk about the hardware.”

I’m not sure how many times that person had used the “I’m a CCIE” justification for their recommendations, but it shows me that some people believe a piece of paper speaks louder than their track record. Those people are usually the ones that fall back into the pattern of “listen to me because I passed tests” not “listen to me because I did the studying”. It’s important to ascribe value to passing a test, but remember that the test is a way to prove you have knowledge. It reminds me of this scene from Tommy Boy:

Throwing up a certification as justification for a recommendation is no different that just tossing a worthless guarantee on a box. Prove you know what you’re talking about instead of just saying you do.

Exceeding Your Reach

The last type of person that climbs the certification ladder is like the one in this tweet from my friend Hank Yeomans:

He looks at the ascent to the top of his certification ladder as a chance to do more. To build more. It’s not the end of the journey. It’s not bad to stop and look around at the new view from the top of your ladder when you’ve climbed it. But if you look at the journey as the start of something that you need to finish, you’re going to start immediately looking around to find the next thing that you need to do. Perhaps it’s learning a new technology related to the one that you just finished. Or maybe it’s that you want to figure out how to get even better at what you do.

People that never rest in their attempts to be better at the ones that ultimately change the way things are done. They don’t just accept that this is the way that things need to be. Instead, they use the top of their ladder to stretch out and see what they can reach. They realize that everything we do in life it just building on something else we’ve already done. We use Crawl, Walk, Run as a metaphor for building through a project or a process all the time. That’s because we know that you have to make steps all the time to progress. But what if someone just said, “You know what, I’ve mastered walking. I don’t need to run. All you people who only crawl listen to me because I’m better than you!” It would show how short-sighted they are when it comes to continuing the journey.


Tom’s Take

Many times, I’ve talked about the fact that I relaxed after I passed my CCIE and enjoyed not studying into the wee hours of the night. But after a while I started getting uncomfortable around 8-9pm. Because there was a little voice in the back of my head that kept telling me “You should be studying for something.” Instinctively, that voice knew that I needed to continue my journey. I would never be content resting on my laurels and I could never bring myself to use my certification as a crutch to make myself look important to others. Instead, I needed to push myself to build on what I’ve already done and make myself better. As Hank said, it’s just a foothill on a greater journey. Once you’ve learned how to use your ladder to increase your reach, even the sky isn’t the limit any longer.

The CCIE Times Are A Changing

Today is the day that the CCIE changes. A little, at least. The news hit just a little while ago that there are some changes to the way the CCIE certification and recertification process happens. Some of these are positive. Some of these are going to cause some insightful discussion. Let’s take a quick look at what’s changing and how it affects you. Note that these changes are not taking effect until February 24, 2020, which is in about 8 months.

Starting Your Engines

The first big change comes from the test that you take to get yourself ready for the lab. Historically, this has been a CCIE written exam. It’s a test of knowledge designed to make sure you’re ready to take the big lab. It’s also the test that has been used to recertify your CCIE status.

With the new change on Feb. 24th, the old CCIE written will go away. The test that is going to be used to qualify candidates to take the CCIE lab exam is the Core Technology exam from the CCNP track. The Core Technology exam in each CCNP track serves a dual purpose in the new Cisco certification program. If you’re going for your CCNP you need the Core Technology exam and one other exam from a specific list. That Core Technology exam also qualifies you to schedule a CCIE lab attempt within 18 months.

This means that the CCNP is going to get just a little harder now. Instead of taking multiple tests over routing, switching, or voice you’re going to have all those technologies lumped together into one long exam. There’s also going to be more practical questions on the Core Technologies exam. That’s great if you’re good at configuring devices. But the amount of content on the individual exam is going to increase.

Keeping The Home Fires Burning

Now that we’ve talked about qualification to take the lab exam, let’s discuss the changes to recertification. The really good news is that the Continuing Education program is expanding and giving more options for recertification.

The CCIE has always required you to recertify every two years. But if you miss your recertification date you have a one year “grace period”. Your CCIE status is suspended but you don’t lose your number until the end of the one-year period. This grace period has informally been called the “penalty box” by several people in the industry. Think of it like a time out to focus on getting your certification current.

Starting February 24, 2020, this grace period is now formalized as an extra year of certification. The CCIE will now be valid for 3 years instead of just 2. However, if you do not recertified by the end of the 3rd year, you lose your number. There is no grace period any longer. This means you need to recertify within the 3-year period.

As far as how to recertify, you now have some additional options. You can still recertify using CE credits. The amount has gone up from 100 to 120 credits to reflect the additional year that CCIEs get to recertify now. There is also a new way to recertify using a combination of CE credits and tests. You can take the Core Technologies exam and use 40 CE credits to recertify. You can also pass two Specialist exams and use 40 CE credits to recertify. This is a great way to pick up skills in a new discipline and learn new technologies. You can choose to pass a single Specialist exam and use 80 CE credits to recertify within the three-year period. This change is huge for those of us that need to recertify. It’s a great option that we don’t have today. They hybrid model offers great flexibility for those that are taking tests but also taking e-learning or classroom training.

The biggest change, however, is in the test-only option. Historically, all you needed to do is pass the CCIE written every two years to recertify. With the changes to the written exam used to qualify you to take the lab, that is no longer an option. As listed above, simply taking the Core Technologies exam is not enough. You must also take 40 CE credits.

So, what tests will recertify you? The first is the CCIE lab. If you take and pass a lab exam within the recertification period you’ll be recertified. You can also take three Specialist exams. The combination of three will qualify you for recertification. You can also take the Core Technologies exam and another professional exam to recertify. This means that passing the test required for the CCNP will recertify your CCIE. There is still one Expert-level exam that will work to recertify your CCIE – the CCDE written. Because no changes were made to the CCDE program in this project, the CCDE written exam will still recertify your CCIE.

Also, your recertification date is no longer dependent on your lab date. Historically your recert date was based on the date you took your lab. Now, it’s going to be whatever date you pass your exam or submit your CEs. The good news is this means that all your certifications are going to line up. Because your CCNA and CCNP dates have always been 3 years as well, recertifying your CCIE will sync up all your certifications to the date you recertify your CCIE. It’s a very welcome quality of life change.

Another welcome change is that there will no longer be a program fee when submitting your CE credits. As soon as you have amassed the right combination you just submit them and you’re good to go. No $300 fee. There’s also a great change for anyone that has been a CCIE for 20 years or more. If you choose to “retire” to Emeritus status you no longer have to pay the program fee. You will be a CCIE forever. Even if you are an active CCIE and you choose not to recertify after 20 years you will be automatically enrolled in the Emeritus program.

Managing Change

So, this is a big change. A single test will no longer recertify your number. You’re going to have to expand your horizons by investing in continuing education. You’re going to have to take a class or do some outside study on a new topic like wireless or security. That’s the encouragement from Cisco going forward. You’re not going to be able to just keep learning the same BGP and OSPF-related topics over and over again and hope to keep your certification relevant.

This is going to work out in favor of the people that complain the CCIE isn’t relevant to the IT world of today. Because you can learn about things like network automation and programmability and such from Cisco DevNet and have it count for CCIE recertification, you have no excuse not to bring yourself current to modern network architecture. You also have every opportunity to learn about new technologies like SD-WAN, ACI, and many other things. Increasing your knowledge takes care of keeping your CCIE status current.

Yes, you’re going to lose the ability to panic after two and a half years and cram to take a single test one or two times to reset for the next three years. You also need to be on top of your CCIE CE credits and your recert date. This means you can’t be lazy any longer and just assume you need to recertify every odd or even year. It means that your life will be easier without tons of cramming. But it means that the way things used to be aren’t going to be like that any longer.


Tom’s Take

Change is hard. But it’s inevitable. The CCIE is the most venerable certification in the networking world and one of the longest-lived certifications in the IT space. But that doesn’t mean it’s carved in stone as only being a certain way forever. The CCIE must change to stay relevant. And that means forcing CCIEs to stay relevant. The addition of the continuing education piece a couple of years ago is the biggest and best thing to happen in years. Expanding the ability for us to learn new technologies and making them eligible for us to recertify is a huge gift. What we need to do is embrace it and keep the CCIE relevant. We need to keep the people who hold those certifications relevant. Because the fastest way to fade into obscurity is to keep things the way they’ve always been.

You can find more information about all the changes in the Cisco Certification Program at http://Cisco.com/nextlevel

Home on the Palo Alto Networks Cyber Range

You’ve probably heard many horror stories by now about the crazy interviews that companies in Silicon Valley put you though. Sure, some of the questions are downright silly. How would I know how to weigh the moon? But the most insidious are the ones designed to look like skills tests. You may have to spend an hour optimizing a bubble sort or writing some crazy code that honestly won’t have much impact on the outcome of what you’ll be doing for the company.

Practical skills tests have always been the joy and the bane of people the world over. Many disciplines require you to have a practical examination before you can be certified. Doctors are one. The Cisco CCIE is probably the most well-known in IT. But what is the test really quizzing you on? Most people will admit that the CCIE is an imperfect representation of a network at best. It’s a test designed to get people to think about networks in different ways. But what about other disciplines? What about the ones where time is even more of the essence than it was in CCIE lab?

Red Team Go!

I was at Palo Alto Networks Ignite19 this past week and I got a chance to sit down with Pamela Warren. She’s the Director of Government and Industry Initiatives at Palo Alto Networks. She and her team have built a very interesting concept that I loved to see in action. They call it the Cyber Range.

The idea is simple enough on the surface. You take a classroom setting with some workstations and some security devices racked up in the back. You have your students log into a dashboard to a sandbox environment. Then you have your instructors at the front start throwing everything they can at the students. And you see how they respond.

The idea for the Cyber Range came out of military exercises that NATO used to run for their members. They wanted to teach their cyberwarfare people how to stop sophisticated attacks and see what their skill levels were with regards to stopping the people that could do potential harm to nation state infrastructure or worse to critical military assets during a war. Palo Alto Networks get involved in helping years ago and Pamela grew the idea into something that could be offered as a class.

Cyber Range has a couple of different levels of interaction. Level 1 is basic stuff. It’s designed to teach people how to respond to incidents and stop common exploits from happening. The students play the role of a security operations team member from a fictitious company that’s having a very bad week. You learn how to see the log files, collect forensics data, and ultimately how to identify and stop attackers across a wide range of exploits.

If Level 1 is the undergrad work, Cyber Range Level 2 is postgrad in spades. You dig into some very specific and complicated exploits, some of which have only recently been discovered. During my visit the instructors were teaching everyone about the exploits used by OilRig, a persistent group of criminals that love to steal data through things like DNS exfiltration tunnels. Level 2 of the Cyber Range takes you deep down the rabbit hole to see inside specific attacks and learn how to combat them. It’s a great way to keep up with current trends in malware and exploitive behavior.

Putting Your Money Where Your Firewall Is

To me, the most impressive part of this whole endeavor is how Palo Alto Networks realizes that security isn’t just about sitting back and watching an alert screen. It’s about knowing how to recognize the signs that something isn’t right. And it’s about putting an action plan into place as soon as that happens.

We talk a lot about automation of alerts and automated incident response. But at the end of the day we still need a human being to take a look at the information and make a decision. We can winnow that decision down to a simple Yes or No with all the software in the world but we need a brain doing the hard work after the automation and data analytics pieces give you all the information they can find.

More importantly, this kind of pressure cooker testing is a great way to learn how to spot the important things without failing in reality. Sure, we’ve heard all the horror stories about CCIE candidates that typed in debug IP packet detail on core switch in production and watched it melt down. But what about watching an attacker recon your entire enterprise and start exfiltrating data. And you being unable to stop them because you either don’t recognize the attack vector or you don’t know where to find the right info to lock everything down? That’s the value of training like the Cyber Range.

The best part for me? Palo Alto Networks will bring a Cyber Range to your facility to do the experience for your group! There are details on the page above about how to set this up, but I got a great pic of everything that’s involved here (sans tables to sit at):

How can you turn down something like this? I would have loved to put something like this on for some of my education customers back in the day!


Tom’s Take

I really wish I would have had something like the Cyber Range for myself back when I was fighting virus outbreaks and trying to tame Conficker infections. Because having a sandbox to test myself against scripted scenarios with variations run by live people beats watching a video about how to “easily” fix a problem you may never see in that form. I applaud Palo Alto Networks for their approach to teaching security to folks and I can’t wait to see how Pamela grows the Cyber Range program!

For more information about Palo Alto Networks and Cyber Range, make sure to visit http://Paloaltonetworks.com/CyberRange/

Certifications Are About Support

You may have seen this week that VMware has announced they are removing the mandatory recertification requirement for their certification program. This is a huge step from VMware. The VCP, VCAP, and VCDX are huge certifications in the virtualization and server industry. VMware has always wanted their partners and support personnel to be up-to-date on the latest and greatest software. But, as I will explain, the move to remove the mandatory recertification requirement says more about the fact that certifications are less about selling and more about supporting.

The Paper Escalator

Recertification is a big money maker for companies. Sure, you’re spending a lot money on things like tests and books. But those aren’t usually tied to the company offering the certification. Instead, the testing fees are given to the testing center, like Pearson, and the book fees go to the publisher.

The real money maker for companies is the first-party training. If the company developing the certification is also offering the training courses you can bet they’re raking in the cash. VMware has done this for years with the classroom requirement for the VCP. Cisco has also started doing in with their first-party CCIE training. Cisco’s example also shows how quality first-party content can drive out the third parties in the industry by not even suggesting to prospective candidates that this is another option to get their classroom materials.

I’ve railed against the VCP classroom requirement before. I think forcing your candidates to take an in-person class as a requirement for certification is silly and feels like it’s designed to make money and not make good engineers. Thankfully, VMware seems to agree with me in the latest release of info. They’re allowing the upgrade path to be used for their recertification process, which doesn’t necessarily require attendance in a classroom offering. I’d argue that it’s important to do so, especially if you’re really out of date with the training. But not needing it for certification is a really nice touch.

Keeping the Lights On

The other big shift with this certification change from VMware is the tacit acknowledgement that people aren’t in any kind of rush to upgrade their software right after the newest version is released. Ask any system administrator out there and they’ll tell you to wait for a service pack before you upgrade anything. System admins for VMware are as cautious as anyone, if not moreso. Too often, new software updates break existing functionality or cause issues that can’t be fixed without a huge time investment.

How is this affected by certification? Well, if I spent all my time learning VMware 5.x and I got my VCP on it because my company was using it you can better believe that my skill set is based around VCP5. If my company doesn’t decide to upgrade to 6.x or even 7.x for several years, my VCP is still based on 5.x technology. It shouldn’t expire just because I never upgraded to 6.x. The skills that I have are focused on what I do, not what I’m studying. If my company finally does decide to move to 6.x, then I can study for and receive my VCP on that version. Not before.

Companies love to make sure their evangelists and resellers are all on the latest version of their certifications because they see certifications as a sales tool. People certified in a technology will pick that solution over any others because they are familiar with it. Likewise, the sales process benefits from knowledgable sales people that understand the details behind your solution. It’s a win-win for both sides.

What this picture really ignores is the fact that a larger number of non-reseller professionals are actually using the certification as a study guide to support their organization. Perhaps they get certified as a way to get better support terms or a quicker response to support calls. Maybe they just learned so much about the product along the way that they want to show off what they’ve been doing. No matter what the reason, it’s very true that these folks are not in a sales role. They’re the support team keeping the lights on.

Support doesn’t care about upgrading at the drop of a hat. Instead, they are focused on keeping the existing stuff running as long as possible. Keeping users happy. Keeping executives happy. Keeping people from asking questions about availability or services. That’s not something that looks good on a bill of materials. But it’s what we all expect. Likewise, support isn’t focused on new things if the old things keep running. Certification, for them, is more about proving you know something instead of proving you can sell something.


Tom’s Take

I’ve had so many certifications that I don’t even remember them all. I got some of them because we needed it to sell a solution to a customer. I got others to prove I knew some esoteric command in a forgotten platform. But, no matter what else came up, I was certified on that platform. Windows 2000, NetWare 6.x, you name it. I was certified on that collection of software. I never rushed to get my certification upgraded because I knew what the reality of things really was. I got certified to keep the lights on for my customers. I got certified to help the people that believed in my skills. That’s the real value of a certification to me. Not sales. Just keeping things running another month.

The Magic of the CCIE

I stumbled across a great Reddit thread this week: Is the CCIE as impossible as it seems? There are a lot of great replies on that thread about people passing and the “good old days” of Banyan Vines, Appletalk, and more. It’s also a fascinating look into how the rest of the networking industry sees exams like the CCIE and JNCIE. Because those of us that have the numbers seem to be magicians to some.

Sleight of CLI Hand

Have you ever seen the cups and balls magic trick? Here’s an excellent example of it from the recently departed Ricky Jay:

Impressive, right? It’s amazing to behold a master craftsman at work. Every time I watch that video I’m amazed. I know he’s doing sleight of hand. But I can’t catch it. Now, watch this same video but with annotations turned on. SPOILER ALERT – The annotations will tell you EXACTLY where the tricks are done:

Is it more impressive now that you know how the tricks are done? Check out this demonstration from Penn and Teller that shows you exactly how they do the tricks as well:

Okay, so it’s a little less mystifying now that you’ve seen how all the sleight of hand happens. But it’s still impressive because, as a professional, you can appreciate how the execute their tradecraft. Knowing that it’s not magic doesn’t mean it’s not an impressive feat. It must means you appreciate something different about the performance.

Let’s apply that to the CCIE. When you’re just starting out in networking, every piece of knowledge is new. Everything you learn is something you didn’t know before. Subnet masks, routing tables, and even just addressing an interface are new skills that you acquire and try to understand. It’s like learning how to take a coin from someone’s ear. It’s simple but it provides the building blocks for future tricks.

When you reach the level of studying for the CCIE lab, it does look like a daunting task. If you’ve followed Cisco’s guidelines you probably have your CCNP or equivalent knowledge. However, there is still a lot you don’t know. If you don’t believe that, go pick up Jeff Doyle’s Routing TCP/IP Volume 1 book. That book taught me I still had a lot to learn about networking.

But, as I slogged through the CCIE, I realized that I was acquiring skills. Just like the magicians that practice the cups and balls every day to get it right, I was picking up the ability to address interfaces quickly and see potential routing loops before I made them like I did in my first lab attempt. Each thing I learned and practiced not only made me a better engineer but also made the CCIE seem less like a mountain and more like a hill that could be climbed.

And I truly realized this when I was thumbing through a copy of the CCIE Official Exam guide. Someone had given me a copy to take a look at and I was happy with the depth of knowledge that I found. I wanted to pass it along to another junior engineer because, as I said to myself, “If only I had this book when I started! I could have skipped over all those other books!”

Practice, Practice, Practice!

That’s where I went wrong. Because I jumped right to the end goal instead of realizing the process. Magicians don’t start out making the Statue of Liberty disappear. They start out pulling coins from your ear and finding your card in a deck. They build their basic skills and then move on to harder things. But they most grand tricks in the magician’s top hat all still use the basic skills: sleight of hand, misdirection, and preparation. To neglect those is to court folly on stage.

CCIEs are no different. Every person that asks me about the test asks “How hard is it to pass?” I usually respond with something like “Not hard if you study.” Some of the people I talk to pick up on the “not hard” part and get crushed by the lab their first time out. They even end up with a $1,500 soda for their efforts. The other people, the ones that focus on “study” in my answer, they are the people who pass on the first attempt or the ones that get it right pretty quickly thereafter.

The CCIE isn’t a test. It’s a course in studying. It’s the culmination of teaching yourself the minutia of protocols and how they interact. The exam itself is almost perfunctory. It tests specific combinations of things you might see in the real world. And if you ask any CCIE, the real world is often ten time stranger than the lab. But the lab makes you think about the things you’ve already learned in new ways and apply that knowledge to find ways to solve problems. The lab isn’t hard because it’s easy. The lab becomes easier when you practice enough to not think the knowledge is hard any longer. I think Bruce Lee said it best:

I fear not the man who has practiced 10,000 kicks once, but I fear the man who has practiced one kick 10,000 times.

Most people would agree that Bruce Lee was one of the best martial artists of all times. And even he practiced until his fingers bled and he body was exhausted. Because he knew that being the best wasn’t about passing an exam for a belt or about showing off for people. It was about knowing what you needed to know and practicing it until it was second nature.


Tom’s Take

The CCIE has a certain magical aura for sure. But it’s not magical in and of itself. It’s a test designed to ensure that the people that pass know their skills at a deep level. It’s a test designed to make you look deeper at a problem and exhaust all your options before throwing in the towel. The CCIE isn’t impossible any more than sawing someone in half is impossible. It’s all about how your practice and prepare for the show that makes the trick seem impressive.