What Happened To The CCDE?

Studying for a big exam takes time and effort. I spent the better part of 3 years trying to get my CCIE with constant study and many, many attempts. And I was lucky that the CCIE Routing and Switching exam is offered 5 days a week across multiple sites in the world. But what happens when the rug gets pulled out from under your feet?

Not Appearing In This Testing Center

The Cisco Certified Design Expert (CCDE) is a very difficult exam. It takes all of the technical knowledge of the CCIE and bends it in a new direction. There are fun new twists like requirements determination, staged word problems, and whole new ways to make a practical design exam. Russ White made a monster of a thing all those years ago and the team that continues to build on the exam has set a pretty high bar for quality. So high, in fact, that gaining the coveted CCDE number with its unique styling is a huge deal for the majority of people I know that have it, even those with multiple CCIEs.

The CCDE is also only offered 3-4 times a year. The testing centers are specialized Pearson centers that can offer enhanced security. You don’t have to fly to RTP or San Jose for your exam, but you can’t exactly take it at the local community college either. It’s the kind of thing that you work toward and get ready for, not the kind of spur-of-the-moment sales test that you have to take today to certify on a new product.

So, what happens if the test gets canceled? Because that’s exactly what happened two weeks ago. All those signed up to take the May 17, 2017 edition of the exam were giving one week’s notice that the exam had been canceled. Confusion reigned everywhere. Why had Cisco done this? What was going to happen? Would it be rescheduled or would these candidates be forced to sit the August exam dates instead? What was going on? This was followed by lots of rumors and suggestions of impropriety.

According to people that should know, the February 2017 exam had a “statistically signification pass rate increase”. For those that don’t know, Cisco tracks the passing exam scores of all their tests. They have people trained in the art of building tests and setting a specific passing rate. And they keep an eye on it. When too many people start passing the exam, it’s a sign that the difficulty needs to be increased. When too few people are able to hit the mark, a decision needs to be made about reducing difficulty or examining why the pass rate is so low. Those are the kinds of decisions that are made every day when the data comes back about tests.

For larger exams like the CCDE and CCIE that have fewer test taking opportunities, the passing rate is a huge deal. If 50% of the people that take the CCIE pass, it doesn’t say very much for the “expert” status of the exam. Likewise, if only 0.5% of test takers pass the CCDE on a given attempt, it’s an indicator that the test is too hard or overly complex and less about skill demonstration and more about being “really, really tough”. The passing rate is a big deal to Cisco.

According to those reports, there was a huge spike in the passing rate for the February CCDE. Big enough to make Cisco shut down the May attempts to find out why. If you have a few percentage points variance in passing or failing rates now and then, it’s not a huge deal. But if you have a huge increase in the number of certified individuals for one of the most challenging exams ever created, you need to find out why. That’s why Cisco went into damage control mode for May and maybe even for August.

The Jenga Problem

So, if your still with me at this point, you’ve probably figured out that there’s a good possibility that someone got their hands on a copy of the CCDE exam. That’s why Cisco had to stop the most recent date in order to ensure that whatever is out in the wild isn’t going to cause issues with the exam passing rates going forward. It makes sense from a test giver perspective to plug the leak and ensure the integrity of the exam.

Yes, it does suck for the people that are taking the exam. That’s a lot of time and effort wasted. One of the things I kept hearing from people was, “Why doesn’t Cisco just change the test?” Sadly, changes to the CCDE are almost impossible at this point in the game.

Even written exams with hundreds of test bank questions are difficult to edit and refresh. I know because I’ve been involved the process for many of them in the past. Getting new things added and old things removed takes time, effort, and cooperation among many people on a team. Imagine the nightmare of trying to get that done for a test where every question is hand-crafted and consists of multiple parts. Where something that is involved in Question 2 of a scenario has an impact on Question 8.

Remember, exams like the CCIE and CCDE are infamous for word choices mattering a lot in the decision making process of the candidate. Changing any of the words is a huge deal. Now, think about having to change a question or two. Or a scenario or two out of four. Or a whole exam revision. Now, do it all in two weeks. You can see what the CCDE team was faced with and why they had to make the decision they did. Paying customers are angry, but the possibility that the integrity of a complicated exam is compromised is worth more to Cisco in the long run.

Tom’s Take

What’s going to happen? It’s difficult to say. Cisco has to find out what happened and stop it immediately. Then, they have to assess what can be done to salvage the exam as it exists today. Scenarios must be created to replace known bad versions. Evaluation of those new scenarios could take a while. And in the interim, Cisco can’t just shut down the testing and certification process. It would be the end of the CCDE and cause a huge backlash in the elite community that exists to spread the good word of Cisco throughout networking circles. Cisco isn’t acting from a position of confusion, but instead from a position of caution. The wrong step at the wrong time could be disastrous.

There Won’t Be A CCIE: SDN. Here’s Why

There’s a lot of work that’s been done recently to bring the CCIE up to modern network standards. Yusuf and his team are working hard to incorporate new concepts into the written exam. Candidates are broadening their horizons and picking up new ideas as they learn about industry stalwarts like OSPF and spanning tree. But the biggest challenge out there is incorporating the ideas behind software defined networking (SDN) into the exam. I don’t believe that this will ever happen. Here’s why.

Take This Broken Network

If you look at the CCIE and what it’s really testing, the exam is really about troubleshooting and existing network integration. The CCIE introduces and tests on concepts like link aggregation, routing protocol redistribution, and network service implementation. These are things that professionals are expected to do when they walk in the door, either as a consultant or as someone advising on the incorporation of a new network.

The CCIE doesn’t deal with the design of a network from the ground up. It doesn’t task someone with coming up with the implementation of a greenfield network from scratch. The CCIE exam, especially the lab component, only tests a candidate on their ability to work on something that has already exists. That’s been one of the biggest criticisms of the CCIE for a very long time. Since the knowledge level of a CCIE is at the highest level, they are often drafted to design networks rather than implementing them.

That’s the reason why the CCDE was created. CCDEs create networks from nothing. Their coursework focuses on taking requirements and making a network out of it. That’s why their practical exam focuses less on command lines and more on product knowledge and implementation details. The CCDE is where people that build networks prove they know their trade.

The Road You Must Design For

When you look at the concepts behind SDN, it’s not really built for troubleshooting and implementation without thought. Yes, automation does help implementation. Orchestration helps new devices configure themselves on the fly. API access allows us to pull all kinds of useful information out of the network for the purposes of troubleshooting and management. But each and every one of these things is not in the domain of the CCIE.

Can SDN solve the thorny issues behind redistributing EIGRP into OSPF? How about creating Multiple Spanning Tree instances for odd numbered VLANs? Will SDN finally help me figure out how to implement Frame Relay Traffic Shaping without screwing up the QoS policies? The answer to almost every one of these questions is no.

SDNs major advantages can only be realized with forethought and guidelines. Orchestration and automation make sense when implemented in pods or with new greenfield deployments. Once they have been tested and proven, these concepts can be spread across the entire network and used to ease design woes.

Does it make more sense to start using Ansible and Jinja at the beginning? Or halfway through a deployment? Would you prefer to create Python scripts to poll against APIs after you’ve implemented a different network monitoring system (NMS)? Or would it make more sense to do it right from the start?

CCIEs may see SDN in practice as they start using things like APIC-EM to roll out polices in the network, but CCDEs are the real SDN gatekeepers. They alone can make the decisions to incorporate these ideas from the very beginning to leverage capabilities to ease deployment and make troubleshooting easier. Even though CCIEs won’t see SDN, they will reap the benefits from it being baked in to everything they do.

Tom’s Take

Rather than asking when the CCIE is going to get SDN-ified, a better question would be “Should the CCIE worry?” The answer, as explained above, is no. SDN isn’t something that a CCIE needs to study for. CCDEs, on the other hand, will be hugely impacted by SDN and it will make a big difference to them in the long run. Rather than forcing CCIEs into a niche role that they aren’t necessarily suited for, we should instead let them do what they do best. We should incorporate SDN concepts into the CCDE and let them do what they do best and make the network a better place for CCIEs. Everyone will be better in the long run.

Designer or Architect? It’s A Matter Of Choice


I had a great time at ONUG this past week. I got to hear a lot of great presentations from some great people, and I got a chance to catch up with some friends as well. One of those was Pete Lumbis (@PeteCCDE) who had a great presentation this past spring at Interop. We talked a lot about tech and networking, but one topic he brought up that made me stop and think for a moment was the wide gulf between design and architecture.

Binary Designers

Design is a critical part of an IT project. Things must fit and make sense before the implementors can figure out how to put the pieces together. Design is all about building a list of products and describing how they’ll interact once turned on. Proper design requires you to step away from the keyboard for a moment and think about a bigger picture than just hacking CLI commands or Python code to make some lights start blinking in the right order.

But design is inherently limited. Think about the last design you did, whether it be wireless or networking or even storage. When you start a design, you automatically make assumptions about what’s going on in the scenario. Perhaps they want to expand their near-line storage capacity. That brings a set of products into play that you choose from. But what if the goal is something different? What if they want a fast caching tier? What if the goal is to create a new pod for object storage?

All of these scenarios are broad enough to require a designer to come up with a good mix of products to fulfill the goals of the project. But the designer has already had assumptions put down for them: The scope and the requirements are pre-determined for them before they ever start thinking about the technology that will be involved in the setup.

Design is all about choices. You have to choose the right product to meet the goals. Once you know the product, you have to make the right choices about which set of products to use? The orange ones or the blue ones? The cheap ones or the expensive ones? Design is about making good choices so implementers can focus on making those choices work.

Visionary Architects

Architecture, on the other hand, has very little to do with choice. Architects are idea people. They look at a problem faced by an organization and try to narrow the focus of the issue to make the designer’s choices easier. Architects don’t worry about individual products or even minor solution sets. They focus on technology areas.

Think back to our storage problem from above. How did the designer arrive at the near-line storage decision? Or the object storage idea? It’s because an architect is the one driving those ideas from a higher level. Architects may not know how to build an object storage bill of materials or how to assemble a chassis switch but they do know what those are used for. Architects instead know that you should be using flash storage in lower density, faster reaction systems when cost is sensitive. They know that a rack may only need a 1U ToR switch instead of a chassis if that ToR switch doesn’t have to provide power or advanced features. They won’t know the specific part number, but they know the technology.

Architects have vision. Designers know products. They need each other to make solutions work and designs happen. The same person can fulfill both roles provided they understand how things break down in the end. A designer architect needs to know that the solutions to customer problems should come before any decisions are made about products. Too often, we find ourselves cornered in a mess because the product mix was decided before the solution was determined.

It’s like trying to bake a cake when all you have in the house is flour, eggs, and swiss cheese. Maybe a cake isn’t what you should be making. The architect would realize that the problem is a limited set of ingredients. instead of deciding on a cake, the architect can work with the designer to find a solution to the problem of food with limited ingredients. Perhaps the designer realizes what’s needed is a soufflé instead. The team figures out the problem with the best design instead of deciding on a design before knowing what the problem is.

Tom’s Take

I was a designer in my past life at a VAR. I still had to implement my designs at the end of the day, but I was the one making the decisions about the products that were needed to meet the solutions my customers had to have. Now, at Tech Field Day I understand the technology at an architecture level. I know why you need this solution for that problem. My ability to hack CLI has gone down a bit but my understanding of the bigger picture has increased several times over that. I now think that I have a better idea of what needs to happen to make tech work the right way and be implemented easier when the architect’s vision can solve the problems that allows the designers to make the right choices.

CCDE and CCAr – Why All The Hate?

Cisco Live 2012 gave me an opportunity to sit in a session dedicated to the newer Cisco expert certfications.  BRKCRT-8862 is for CCIEs that are looking at moving to the Cisco Certified Design Expert (CCDE) and maybe even the Cisco Certified Architect (CCAr).  The CCDE is a pretty well known certification at this point.  Developed in large part by Russ White, the CCDE tests a candidate on their knowledge of taking a set of requirements and producing a valid design for a given scenario.  Originally envisioned as a board certification exam not unlike the VMware Certified Design Expert (VCDX), the CCDE is instead an 8-hour exam with some multiple choice and some fill-in-the-blank type questions.  The CCDE is a prerequisite for the CCAr, which is the culmination of something Cisco is trying to do with focusing on solutions.  The CCAr tends to focus more on the Planning and Preparation areas of the PPDIOO model.  Cisco tends to see them as “big picture” solutions engineers that focus on more conceptual ideas that revolve around things like business contraints and specific use cases.  From what Cisco was describing, it appears that the role of the CCAr is to gather information about the customer desires that will then be given to the CCDEs to generate a design.  The CCAr is a 5-month long board exam that is graded by three judges (mostly existing CCArs) that are with you during the entire process, from the initial submission of your application up until the final board review.  Note that not all those that apply to the program will be selected for review.

The BRKCRT session highlighted a lot of hesitation in the CCIE ranks where the CCAr is concerned.  Cisco has spent a lot of time over the last three years attempting to have the CCDE reach parity with the CCIE in terms of importance.  Had they simply called it the CCIE: Design it would likely have been much more accepted in the community.  However, there is a legacy of the original failed CCIE: Design track from a decade ago, so I’m sure that Cisco wanted to avoid carrying the negativity forward.  Instead, they’ve had to fight the reputation that the exam has gotten for being too focused on very specific technologies or being a bad representation of what a design test should be.  Much of this criticism focuses on the major test developer, Russ White.  When I first heard of the exam going live, many people said it was easy so long as you asked yourself “What Would Russ White Do?”  With the new version of the exam being recently released, as well as Cisco offering the exam at new locations, the CCDE may very well be on the road to gaining a little more respect.

The CCAr, on the other hand, is a pretty big target.  CCIEs are upset that the CCDE is the only prerequisite for the exam.  After almost twenty years of being told that the CCIE is the most important certification inside of Cisco, if not the world, now we’re told that the CCIE isn’t even good enough for us to get our foot in the door of the Architect board.  I think some of this comes from the reality that many CCIEs are called upon to do designs in their every day work.  Often, after a CCIE goes through all the training necessary to pass the lab exam, they have a very good idea of the capabilities of the product set within their particular track.  Therefore, many companies call on them to produce designs, as they are usually the best suited to make the decision between using a particular model of switch or router or firewall.  However, not all CCIEs are good at design.  Many of them have a “bottom up” view of things that tends to lead them down the path of point solutions without regard for higher-level thinking.  Call it a “forest for the trees” type of mentality.  They get so bogged down on the decisions between what line cards to use or why they’d rather use a 4500 in place of a 6500 that they lose sight of the bigger goals.  There’s also no guarantee that a CCIE will be able to produce a valid design from a pile of non-technical interviews and business requirements instead of data sheets and performance specs.  The CCDE teaches engineers how to keep a bigger view of things in mind when planning a design.

The problem, however, is that both the CCDE and CCIE are still focused on providing their respective documents, whether they be for design or implementation.  Someone still has to lay the groundwork for the project and figure out how to focus the task of the designers.  Without an even bigger picture, design is just throwing things at the wall until something sticks.  Some designers understand that and ask specific questions before diving into their work.  These are the folks that are the target of the CCAr program.  Cisco doesn’t just want a bill of materials or a pretty Visio document handed to the customer.  They want a cohesive plan and design delivered to sell a vision, whether it be an architecture like a connected sports stadium or a connected energy grid.  Architects take into account more than just technology.  They are constantly thinking about esoteric things like regulatory laws and other logistic restraints.  These are the kinds of things that CCIEs and CCDEs either shy away from or would rather not think about.

Look at it like this:  The CCAr is like the CEO of the team.  They have the vision and the desire to go out and kickstart things by looking at the big picture.  They have to play the role of project manager and pre-sales at the same time.  They keep a handle on the non-technical aspects of the project.  Once they’ve determined the direction, the send in the CCDEs.  These guys take all the documentation the CCAr has generated and meld it with the best practices needed to create a valid, working design.  Once the CCDEs have everything in order, it’s up to the CCIEs to go out and make it all work.  They are the technical piece that gets the hard work accomplished.  The CCAr may not be typing commands in on the CLI, but they are the ones running interference from the other side by keeping customers appraised and kicking over rocks to find things the designers need to know.

If you’d like to read a few more takes on the CCDE, check out Russ White’s Why CCDE? post at the Packet Pushers site.  Also, read about the journey to CCDE success from CCDE 2012::1, Ronnie Angello (@rangello)

Tom’s Take

I’ll be shocked if there are ever more than a hundred Cisco Certified Architects.  The level of thinking required for this exam isn’t something that can be taught.  You are either born to be a technical architect or you aren’t.  With that being said, I think that the skills that are crucial to having a well rounded view of architecture are best served by requiring both the CCIE and CCDE as a prerequisite for the CCAr.  Design without technical know-how is a dicey proposition at best, but trying to attain and architecture role without knowing how to design is equally capable of colossal folly.  Just like any recipe, you need a good mix of both to make the final product come out right.