I’m having a great time at Cisco Live this year talking to networking professionals about the state of things. Most are optimistic about where their jobs are going to fit in with networking and software and the new way of doing things. But there is an undercurrent of dissatisfaction with one of the most fundamental pieces of network training in the world. The discontent is palpable. From what I’ve heard around Las Vegas this week, it’s time to fix the CCIE Written Exam.
Whadda Ya Know?!?
The CCIE written is the bellwether of network training. It’s a chance for network engineers that use Cisco gear to prove they have what it takes to complete a difficult regimen of training to connect networks of impressive size. It’s also a rite of passage to show others that you know how to study, prep, and complete a difficult practical examination without losing your cool. But all that hard work starts with a written test.
The CCIE written has always been a tough test. It’s the only barrier to entry to the CCIE lab. Because the CCIE has never had prerequisites and likely never will due to long standing tradition, the only thing standing in the way of you ability to sit the grueling lab test is a 100 question multiple choice exam that gauges your ability to understand networking at a deep technical level.
But within the last year or so, the latest version of the CCIE written exam has begun to get very bad reviews from all takers of the test. There are quite a few people that have talked about how bad the test is for candidates. Unlike a lot of “sour grapes” cases of people railing against a test they failed, the feedback for the CCIE written is entirely different. It tends to fall into a couple of categories:
The Test Is Poorly Written
The most resounding critique of the exam is that it is a poorly constructed and executed test. The question quality is subpar. There are spelling mistakes throughout and test questions that have poor answer selections. Having spent a large amount of time helping construct the CCNA exam years ago, I can tell you that you will spend the bulk of your time creating wrong answers as distractors to the right ones. Guidelines say that a candidate should have no better than a 25% chance to guess the correct answer from all the choices. If you’ve ever taken a math test that has four multiple choice answers with three being correct for various mistakes in working the problem, you know just how insidious proper distractors can be (and math teachers too).
The CCIE written is riddled with bad distractors according to reports. It also has questions that don’t have a true proper answer or a set of answers that are all technically correct with no way to select them all. That frustrates test takers and makes it very difficult to study for the exam. The editing and test mechanics errors must be rectified quickly in order to restore confidence to the people taking the test.
The Test Doesn’t Cover The Material
Once people stop telling me how bad the test is constructed, they start telling me that the questions are bad on a conceptual level as well. No NDAs are violated during these discussions to protect everyone involved, but the general opinion is that the test has skewed in the wrong direction. Cisco seems to be creating a test that focuses more on the Cisco and less on the Internetworking part of the CCIE.
The test has never been confused for being a vendor-neutral exam. Any look at the blueprint will tell you that there a plenty of proprietary protocols and implementation methods there. But the older versions of the exam did do a good job of teaching you how to build a network that could behave itself with other non-Cisco sections. Redistributing EIGRP and OSPF is a prime example. But the focus of the new exam seems to be skewed toward very specific Cisco proprietary protocols and the minutia around how they operate. I’ve always thought that knowing the hello and dead timers of OSPF NBMA areas is a huge time sink and really only justified for test takers, but I also see why knowing that would be important in multi-vendor operations. But knowing the same thing for an EIGRP DMVPN seems a bit pointless.
The other problem is that, by the admission of most test takers, the current CCIE Written Exam study guide doesn’t cover the areas of the blueprint that are potentially on the test. I feel very sorry for my friend Narbik Kocharians here. He worked very hard to create a study guide that would help test takers pass the exam with the knowledge necessary to do well on the lab. And having a test over a completely different area than his guide makes him look bad in the eyes of testers without good cause. It’s like a college class when the professor tells you to study the book but gives you a test over his or her lectures. It’s not fair because you studied what you were told and failed because they tested something else.
CCIEs Feel There Are Better Recert Options
This is the most damaging problem in my mind. About half the test takers for the CCIE written are candidates looking to qualify for the lab. That requires them to take the written exam for their specific track. But the other half of the test takers are CCIEs that have passed the lab and are looking to recertify. For these professionals, any CCIE written exam is valid for recertification.
Many CCIE candidates look to broaden their horizons by moving to different track to keep their CCIE current while they study for service provider, data center, or even collaboration as a topic area of study. For them, the CCIE is a stepping stone to keep the learning process going. But many CCIEs I’ve spoken to in the past few months are starting to take other exams not because they want to learn new things, but because the CCIE Routing and Switch written exam is such a terrible test.
Quite a few CCIEs are using the CCDE written to recertify. They feel it is a better overall test even though it doesn’t test the material to the level that the CCIE R&S written exam does. They would even be willing to take the chance of getting a question on an area of technology that they know nothing about to avoid having to deal with poor questions in their areas of study. Still more CCIEs are choosing to become Emeritus and “retire” so as to avoid the pain of the written exam. While this has implications for partner status and a host of other challenges for practicing engineers, you have to wonder how bad things must be to make retirement of your CCIE number look like a better option.
I took the CCIE R&S written last year at Cisco Live. I was so disgusted with the exam that I immediately switched to the CCDE written and recertified my number while simultaneously vowing never to take the R&S written again. From what I’ve heard this year, the test quality is still slipping with no relief in sight. It’s a sad state of affairs when you realize that the flagship test for Cisco engineers is so horribly broken that those same engineers believe it can’t be fixed. They feel that all the comments and feedback in the world are ignored and their expertise in taking exams is pushed aside for higher cut scores and a more exclusive number of candidates. The dark side of it all is the hope that there isn’t an agenda to push official training materials or other kinds of shortcuts that would help candidates while charging them more and/or locking out third party training providers that work hard to help people study for the lab.
Cisco needs to fix this problem now. They need to listen to feedback and get their written problems under control. If they don’t, they may soon find the only people taking the R&S written test are the same kinds of dumpers and cheaters they think they are trying to keep out with a poorly constructed test.
NOTE: I have published an update to this post here: Fixing The CCIE Written – A Follow Up