So Long To The CCIP

The Cisco Certified Internetwork Professional (CCIP) certification has always been the goal of those network professionals that wanted to march to the beat of a different drummer.  People like me that concentrate on the enterprise/campus side of things revel in our use of OSPF and EIGRP.  We live and die by IOS and get cold sweats at night when someone mentions IS-IS.  The ideal CCIP candidate, on the other hand, loves all of this service provider oriented talk.  They want to spend all their time talking about ingress QoS policies.  They cackle with glee when the subject of MPLS-TE comes up.  They think users are just a myth that exist on the other side of the mythical CPE Wall.

The problem, though, is that the CCIP hasn’t really been focused on the service provider arena for a while now.  While the other professional level exams have received overhauls in the recent past, no one touched the CCIP.  When the CCVP and CCSP became the CCNP: Voice and CCNP: Security, no one wanted to make the CCNP: Internetwork.  The coursework for the CCIP has always relied heavily on other tracks to exist.  QoS is a big part of the SP world, so the QoS exam was borrowed from the voice track.  Routing is another huge part, so the old Building Cisco Scalable Internetworks (BSCI) test was repurposed as well.  The only pure CCIP exams were over BGP and MPLS.  You could even take a composite exam if you were feeling up to the challenge of getting your teeth kicked in for twice as long.  However, the routing exam has caused some consternation.  When I originally studied for my CCNP three years ago, the BSCI book was a handbook of enterprise and service provider routing.  It contained a lot of information about every routing protocol.  While it focused on OSPF and EIGRP, there was a touch of BGP and IS-IS as well.  It served as the foundation for the CCNP, CCDP, and the CCIP.  This made sense with Cisco’s foundation being the router.  However, when Cisco changed the tests and courseware for the CCNP with their latest refresh, the new ROUTE test was a shell of its former self.  Based on the blueprint (login required), it still tests on OSPF, EIGRP, and BGP somewhat.  It even throws in IPv6 routing as well, which is a sorely needed topic.  However, there’s no IS-IS.  None. Nada. Zilch.  How’s that supposed to help the SP engineer that might use IS-IS all the time and never see EIGRP?  Something needed to be done.  And every passing day that the CCIP relied upon tests that didn’t fulfill the criteria of the people being certified was a day that it passed closer to irrelevance.

Thankfully, Cisco decided in May 2012 to overhaul the entire CCIP track.  Now known as the CCNP: Service Provider, it finally focuses on the things that service provider network professionals will be doing.  The four new tests are specific to the SP track.  There are no overlapping tests.  The prerequisite for the CCNP: SP is the CCNA: SP, which is two SP-specific tests of it’s own.  Cisco has finally figured out that most SP engineers exist in a world all their own with very little in common with enterprise/campus folks.  A quick glance at Mirek Burnejko’s excellent IT Certfication Master page for the CCNP:SP shows that the SPROUTE test will focus on IS-IS, OSPFv2 and v3, and BGP.  No EIGRP to be found.  It also tests these topics on IOS-XR and IOS-XE, the new flavors of IOS that run on the equipment that would be found in an SP environment.  If you’d like to see more about the ins and outs of IOS-XR, check out Jeff Fry’s (@fryguy_pa) IOS-XR posts.  The SPADVROUTE test focuses on BGP and multicast, the two odd ducks of routing.  This means that you can spend your time reading Jeff Doyle’s Routing TCP/IP Volume 2 and take a test basically over that whole book.  The SPCORE covers QoS and MPLS functionality such as MPLS-TE.  That’s where I’d expect to see the TE stuff, since it’s usually configured in the network core and not on the edges.  The SPEDGE test covers MPLS VPNs, as well as VPN technologies in general.  I like that Cisco chose to split the core and edge pieces of the CCNP: SP, as there are people that may spend their entire careers working on P routers and never see a piece of CPE equipment.  Conversely, there are those that want to stay as far away from the core as possible and would prefer to make the PE router their device of choice.

The CCNP: SP is available today at any Prometric/VUE testing center.  You can find out more about the certification from Cisco’s website or by visiting Mirek’s site above.

Tom’s Take

Cisco has done a great job of breaking the CCIP up into bite-sized chunks that have clearly defined topic boundaries.  I can choose to focus on interior routing without worrying about multicast.  I can focus on MPLS VPN without thinking too much about MPLS-TE.  I can focus on the important parts one at a time.  The new CCNP: SP also addresses the shortcomings I’ve seen with the old CCIP test.  By giving the SP track a dedicated testing platform all by itself, Cisco no longer has to worry that test changes in one area will carry over to a separate track and cause confusion and delay.  As well, with the new branding and focus on the service provider arena, Cisco has shown that it has not forsaken those that want to spend their time working behind the scenes at ISPs.

Double NAT – NAT$$$

Welcome to my first NAT post of 2012.  After spending some time during the holidays unwrapping new tech toys and trying to get them to work on my home network, I’m full of enough vitriol that I need to direct it somewhere.  Based on the number of searches for “double NAT” that end up on my blog, I thought it was only fitting that I direct some hate toward NAT444, also called carrier-grade NAT or large-scale NAT.

Carrier-grade NAT is the brainchild of the ISP world.  It turns out that we may be running out of IP addresses.  Shocking, right?  We’ve all known for at least a year that we were on the verge of running out of IPv4 addresses.  I even said as much last February.  The ISPs seem to have decided that IPv4 is still a very important business model for them and the need to continue using it over IPv6 is equally important.  My best guess is that many consumer-oriented ISPs looked at their traffic patterns and found that the majority of them were dominated by outbound connections.  This isn’t shocking when you consider that the majority of devices in the home aren’t focused around serving content.  In fact, many residential ISPs (like mine) tend to block connections on well-known server ports like 25 and 80.  This serves to discourage consumer users from firing up their own mail and web servers and forces them to use those of the ISP.  It also makes the traffic patterns outflow dominant.

With the lack of availability of IPv4 addresses, the ISP need to find a way to condense their existing and new traffic onto an ever-dwindling pool of available resources.  Hence, NAT444.  Rather than handing the customer an global IPv4 address for use, the ISP NATs all traffic between their exit points and the customer premise equipment (CPE):

In this example, the subscribers may have an address space on their devices in the 192.168.x.x/24 space.  The ISP would then assign an address to the CPE device in the 172.16.x.x./16 space or the 10.x.x.x/8 space.  That traffic would then bent sent through some kind of NAT gateway device or cluster of devices.  Those devices would function in the same way that your home DSL/Cable router functions when translating addresses, only on a much larger scale.  The amount of addresses the ISP current has in their pool would not need to be significantly increased to compensate for a larger number of subscribers, just as if buying a new XBox doesn’t require you to get a new IP address from your ISP.

NAT444 has its appealing points.  It’s helpful in staving off the final depletion of the IPv4 address space from the provider side of things.  It will help keep IPv4 up and running until IPv6 can be implemented and reduce the pressure on the address space.  Yeah, that’s about it…

NAT444 has drawbacks.  Lots of them.  First, you are adding a whole new layer of complexity onto your ISP’s network.  Keeping track of all those state tables and translations for things like lawful intercept is going to be a pain.  Not to mention that the NAT gateway devices are going to need to be huge, or at the very least clustered well.  Think about how many translations are going through your CPE device at home.  Now multiply that by the number of people on your ISP’s network.  Each of those connections now has to have a corresponding translation in the NAT table.  That means RAM and CPU power.  Stupidly big boxes for that purpose.  What about applications?  We’ve already seen that things like VoIP don’t like NAT, especially when SIP hardcodes the IP address of the endpoint into all of its messages.  Lucky for me, a group already did some testing and published their results as a draft RFC.  Their findings?  Not so great if you like using SIP or seeding files with BitTorrent (hey, it has legitmate uses…).  They also tested things like XBox Live and Netflix.  Those appear to have been bad as late as last year, but may have gotten better as of the last test.  Although, I don’t think testing Netflix streaming for 15 minutes was a fair assessment.  You can also forget about hosting anything from your own network.  No web, no email, no peer-to-peer gaming sessions over a NAT444 setup.  I’m sure your ISP will be more than happy to provide you with a non-NAT444 setup provided you want to upgrade to “premium” service or move to a business account with all the associated fees.

I leave you with a this small reminder…


Tom’s Take

I had one of those funny epiphanies when writing this post.  I kept holding down the shift key when typing, so NAT444 kept turning into NAT$$$.  That’s when it hit me.  NAT444 isn’t about providing better service for the customers.  It’s about keeping the whole mess running just a little while longer with the same old equipment.  If the ISPs can put off upgrading to IPv6 for another year or two, that’s one more year they don’t have to spend their budgets on new stuff.  Who cares if it’s a little harder to troubleshoot things?

In the end, I think NAT444 will be dead on arrival, or at the most shortly thereafter.  Why?  Because too many things that end users depend on today will be horribly broken.  Sure, I can grouse about how NAT444 breaks the Internet and is horrible from a design perspective.  I am the I Hate NAT Guy, after all.  But try telling the average suburban household that they won’t be able to watch a streaming Netflix movie or play Call of Duty over XBox live anymore because we didn’t plan to keep the Internet running with a new set of addresses.  Those people won’t wax intellectual about their existential quandary on a blog.  They’ll vote with their dollars and go to an ISP that doesn’t use NAT444 so all their shiny new technology works the way they want it to.  In the end, NAT444 will end up costing the ISPs big $$$.

Aerohive Branch on Demand – Bring Your Own Office

Bring Your Own Device (BYOD) is enabling people to provide their own equipment for work.  But what happens when people aren’t just satisfied bringing their own Macbook to the party?  What happens if they want to bring their office to your office as well?  With the large surge in teleworkers and contractors being brought on inside companies and their ability to do the majority of their jobs without having to step foot into the corporate office, the need to provide connectivity and security for a home workspace is now becoming paramount if the Bring Your Own Office (BYOO) movement is going to take off.

The current solutions to this problem either involve using some off-the-shelf consumer product to address the issue or buying an enterprise grade solution to implement.  Both have their strengths and weaknesses.  Consumer-grade devices are dirt cheap and get the job done.  However, there is very little in the way of scalability and configuration management.  Unless your remote worker is good at configuring Linksys or D-Link, you could be in for a fight.  Also, consumer grade equipment doesn’t have the service and support necessary to run an enterprise on a regular basis.  On the flip side, enterprise equipment does have a great degree of manageability and support to provide robust service for your teleworkers.  Provided, that is, you are willing to invest the large amount of money that it takes to get it setup.  In fact, the investment is usually so high that reclaiming the equipment is top priority in the event that the teleworker leaves the company or completes the contract.  How then do we as network rock stars balance our need for cheap remote connectivity with our desire to have manageability and security?

Enter Aerohive.  I saw Aerohive at Wireless Field Day back in March of this year and was pretty impressed by their HiveManager product that they use to provide configuration and management for their controller-less access points.  They’ve also given me a briefing about the 4.0 release of their HiveOS firmware.  They were kind enough to give me a sneak peak at their Branch on Demand product that was announced November 15th.

Aerohive Branch on Demand utilizes Aerohive’s experience with creating cloud based management for devices and couples it with a new branch router device that can provide simple connectivity for your branch/remote offices or teleworkers.  All of the provisioning for these devices is done in HiveManager, so the only instructions your remote workers need is “plug the yellow cable into the yellow slot and plug the other end into the Internet”.  I think even my mom could do that.  Afterwards, the router checks in with HiveManager and pulls down the configuration so your teleworker can connect back to the home office.  Your user connects via SSL IPSec VPN to allow any device to access corporate resources, whether it be a desktop, laptop, tablet, or smartphone (EDIT – Stephen Phillip was kind enough to notice that I mixed up SSL and IPSec in my notes on this.  The BR series use IPSec to connect back to the central site due to the increased performance for special traffic like voice).   The same polices that you have in place in your corporate office are extended to the remote worker as well.  You can either choose to tunnel all traffic back to the home office to be scanner and permitted, or you can split tunnel the traffic so that non-corporate packets exit locally.  There is a bit of apprehension on the part of most network rock stars for a setup like this, as splitting the traffic does introduce the capability for nasty things to infect the remote machine and then be introduced back into the corporate network.  Aerohive thought of this too and uses a cloud proxy to redirect the split tunneled traffic to a filtering service such as Websense or Barracuda to ensure that all those packets are “cloud washed” before they are permitted back into the network.  That alleviates the stress of not knowing where your branch users are going as well as preventing large amounts of traffic from being needlessly tunneled back to the corporate sites just to go out to the Internet.

All of these features come with HiveOS 5.0, which means that current users of the AP 330 and AP 350 gain the ability for those devices to function as routers.  You can even connect a 3G/4G USB modem to the USB port on the device and turn it into a backup interface for connectivity in the event the primary WAN link goes down for some reason.  At launch, the branch routers will support a small list of USB modems such as the AT&T Shockwave or Momentum, but as the software matures and drivers become available a wider variety of these devices will be supported.  This would be a great idea for those that live in areas where solid Internet connectivity isn’t always a given or for a user that spends a lot of time on the road and needs corporate VPN capabilities where they aren’t always available, such as in the middle of an oilfield or a parking lot.  No need to setup a cumbersome VPN client or worry about usernames and passwords and tokens.  Just give them an Aerohive branch router and let them go.

There are two models of branch routers available.  The BR100 is a 10/100 5-port device that includes a 2.4GHz 802.11n radio and a USB port for 3G/4G backhaul.  It retails for $99, or if you’d like to use the Network-as-a-Service subscription, you can get the device for the same $99 price point, only it includes software updates as well as tech refreshes for two years, so when a new update to the BR100 comes out, you’ll get that device for nothing.  There is also a BR200 that will have 5 GigE ports and dual 2.4/5GHz 3×3:3 802.11n radios as well as two PoE ports and crypto acceleration.  The BR200 will be out sometime next year.


Tom’s Take

I think Aerohive has finally found a good use case for the cloud.  Having your hardware managed by a cloud-based application means that you can always find it no matter where it might be.  If you are already an Aerohive customer that finds yourself in need of a branch router solution, this is a no-brainer.  The same management platform now allows you to control your access points as well as your branch users.  The ability to push the same policies from desktop to Destin, FL is very powerful and cuts down on a lot of stress.  If you aren’t a current Aerohive customer but know that you are going to need to add some teleworking capacity in the future, you can’t go wrong looking at this solution.  For $99 a device (and $999 for the VPN termination software) the solution is very inexpensive and gives you a lot of flexibility to build out instead of needing to worry about scaling straight up.  After all, letting your users bring their own office should cost you yours.

If you’d like to learn more about Aerohive’s new solutions, head over to http://www.aerohive.com.  There’s also a nice short introduction to the product over at the Packet Pushers site.


Disclaimer

Aerohive provided me with an advanced briefing on the Branch on Demand product for the purposes of preparing this blog post.  The did not ask for nor were they promised any consideration in the creation of this article.  Any and all opinions expresses within are mine and mine alone.

Bogon Poetry

I was thinking the other day that I’ve used the term bogon in several Packet Pushers podcasts and never really bothered to define it for my readers.  Sure, you could go out and search on the Internet.  But you’ve got me for that!

Bogon is a term used in networking to describe a “bogus address”.  According to Wikipedia, Fount of All Knowledge, the term originated from a hacker reference to a single unit of bogosity, which is the property of being bogus.  I personally like to think of it as standing for BOGus Network (forgive my spelling).  Not that this refers to undesirable packets, which is not to be confused with vogon, which is a class of undesirable bureaucrats that run the galaxy or a bogan, which is an undesirable class of socioeconomics in Australia (if you’re American, think “redneck” or “white trash”).

Bogons are addresses that should never be seen as the source of packets that are entering your network.  The most stable class of bogon isn’t actually a bogon.  It’s a martian, so called because they look like they are coming from Mars, which is a place packets clearly cannot be sourced from…yet.  Martians include any address space that is listed as reserved by RFC1918 or RFC5735.  It’s a pretty comprehensive list, especially in RFC5735 so take few moments to familiarize yourself with it.  You’ll see the majority of private networks along with APIPA addressing and a few lesser-known examples of bogus networks as well.

The other component of a bogon is an address that shouldn’t exist on the public Internet.  Beyond the aforementioned Martians, the only other bogons should be IP blocks that haven’t yet been allocated by IANA to the RIRs.  However, that list should be almost empty right now, as IANA has exhausted all its available address space and given it over to the 5 RIRs.  The folks over at Team Cymru (that prononunced kum-ree for those not fortunate to be fluent in Welsh) have put together a list of what they call “fullbogons” which lists the prefixes assigned to RIRs but not yet handed out to ISPs for consumption by customers.  Traffic being sourced from this range should be treated as dubious until the range is allocated by the RIR.  The fullbogon list is updated very frequently as the hungry, hungry Internet gobbles up more and more prefixes, so if you are going to use it please stay on top of it.

How Do I Use Bogons?

My preferred method of using a bogon list is in an edge-router access list (ACL) designed to filter traffic before it ever lands on my network.  By putting the ACL on the very edge of the network, the traffic never gets the chance to hop to my firewall for evaluation.  I’d prefer to save every spare CPU cycle I could on that puppy.  My access list looks something like this (taken from Team Cymru’s bogon list today):

!
access-list 1 deny 0.0.0.0 0.255.255.255
access-list 1 deny 10.0.0.0 0.255.255.255
access-list 1 deny 127.0.0.0 0.255.255.255
access-list 1 deny 169.254.0.0 0.0.255.255
access-list 1 deny 172.16.0.0 0.15.255.255
access-list 1 deny 192.0.0.0 0.0.0.255
access-list 1 deny 192.0.2.0 0.0.0.255
access-list 1 deny 192.168.0.0 0.0.255.255
access-list 1 deny 198.18.0.0 0.1.255.255
access-list 1 deny 198.51.100.0 0.0.0.255
access-list 1 deny 203.0.113.0 0.0.0.255
access-list 1 deny 224.0.0.0 15.255.255.255
access-list 1 deny 240.0.0.0 15.255.255.255
access-list 1 permit any
!
!
interface FastEthernet 0/0
description Internet_Facing
ip access-group 1 in
!

That should wipe out all the evil bogons and martians try to invade your network.  If you want to use the fullbogon list, obviously your ACL would be considerably longer and need to be updated more frequently.  The above list is just the basic bogon/martian detection and should serve you well.

Tom’s Take

Blocking these spoofed networks before they can make it to you is a huge help in preventing attacks and spurious traffic from overwhelming you as a Network Rock Star.  Every little bit helps today with all of the reliance on the Internet, especially as we start moving toward…The Cloud.  If you sit down and block just the regular bogon list I’ve outlined above, you can block up to 60% (Warning: Powerpoint) of the obviously bad stuff trying to get to your network.  That should be a big relief to you and let you have a few minutes of free time to take up a new hobby, like poetry.

Thanks to Team Cymru for all the information and stats.  Head over to http://www.team-cymru.org to learn more about all those nasty bogons and how to stop them.

Network Field Day 2: Network Boogaloo

Guess who’s back?

I’m headed to yet another Tech Field Day event!  This time, I’ll be attending Network Field Day #2 in San Jose, CA on October 27th and 28th.  I read about the first Network Field Day last year and learned a lot about the vendors and presentations from the delegates.  Now, it’s up to me to provide that same kind of response for Net Field Day 2.  The delegate list this time around is quite awe-inspiring for a guy like me:

Kurt Bales Network Janitor @NetworkJanitor
Ethan Banks PACKETattack @ECBanks
Tony Bourke The Data Center Overlords @tbourke
Brandon Carroll BrandonCarroll GlobalConfig @BrandonCarroll
Greg Ferro EtherealMind PacketPushers @EtherealMind
Jeremy L. Gaddis Evil Routers @JLGaddis
Ivan Pepelnjak Cisco IOS Hints and Tricks @IOSHints
Mrs. Y. Packet Pushers @MrsYisWhy

I am humbled to be included in such good company.  The two Packet Pushers, Mr. MPLS himself, the man that beat IOU, the Aussie JNCIE/CCIE candiate, the walking security dictionary Brandon Carroll, and the Network Security Princess herself.  I think my invitation must have gotten confused with someone else’s.

Odds are good that if you are involved in networking at all you already follow all of these people on Twitter and read their blogs daily.  If not, stop what you are doing and follow them RIGHT NOW.  You won’t be sorry.  In fact, this is the first time I haven’t had to start following a Tech Field Day delegate on the list of attendees since I’ve been following these folks for quite a while.

Getting Inolved with Tech Field Day

Tech Field Day is always looking for amazing people to attend events and share in the wealth of knowledge.  There are lots of ways you can add your voice to the gestalt:

1.  Read the TFD FAQ and the Becoming a Field Day Delegate pages first and foremost.  Indicate your desire to become a delegate.  You can’t go if you don’t tell someone you want to be there.  Filling out the delegate form submits a lot of pertinent information to Tech Field Day that helps in the selection process.

2.  Realize that the selection process is voted upon by past delegates and has selection criteria.  In order to be the best possible delegate for a Tech Field Day, you have to be an open-minded blogger willing to listen to the presentations and think about them critically.  There’s no sense in bringing in delegates that will refuse to listen to a presentation from Brocade because all they’ve ever used is Arista and they won’t accept Brocade having good technology.  If you want to learn more about all the products and vendors out in the IT ecosystem, TFD is the place for you.

3.  Write about what you’ve learned.  One of the hardest things for me after Tech Field Day was consolidating what I had learned into a series of blog posts.  TFD is a fire hose of information, and there is little time to process it as it happens.  Copious notes are a must.  As is having the video feeds to look at later to remember what your notes meant.  But it is important to get those notes down and put them up for everyone else to see.  Because while your audience may have been watching the same video stream you were watching live, they may not have the same opinion of things.  Tech Field Day isn’t just about fun and good times.  Occasionally, the delegates must look at things with a critical eye and make sure they let everyone know where they stand.


Be sure to follow Tech Field Day on Twitter (@TechFieldDay) for information and updates about Network Field Day 2 as the date approaches.  There will also be streaming video of the presentations at the Tech Field Day website.  The videos will also be posted in their entirety shortly afterwards.  If you want to follow along on Twitter, you can use the hastags #TechFieldDay or #NFD2 to make comments or ask questions during the presentations.  I usually have a TweetDeck window open and will relay your questions along if no one else beats me to it.  I try to tag all my posts with the #TechFieldDay and #NFD2 hashtags, so if I’m overwhelming you with commentary feel free to filter that hashtag from your feed to keep me quiet.  In the past, I’ve tried to have an IRC channel open during the presentations to allow for real-time communications and feedback for those of you out there that prefer an alternative to Twitter.  Once I have the room setup I will post the details.

Tech Field Day Sponsor Disclaimer

Tech Field Day is made possible by the sponsors.  Each of the sponsors of the event is responsible for a portion of the travel and lodging costs.  In addition, some sponsors are responsible for providing funding for the gatherings that occur after the events are finished for the day.  However, the sponsors understand that their financing of Tech Field Day in no way guarantees them any consideration during the analysis and writing of reviews.  That independence allows the delegates to give honest and direct opinions of the technology and the companies that present it.

You Don’t Need Gigabit, But We Do

Stacy Higginbotham wrote a thought-provoking article last week entitled “The Elephant in the Gigabit Network Room”.  Therein, she talks about how many providers are starting to bring gigabit connectivity to residential areas for prices in the $200-$300 range.  She also discusses that this is overkill for most customers, as many devices today can’t reach sustained transfer rates above 500 Mbps as well as the majority of the content being provided are low speed, bandwidth non-intensive services like Twitter.  She goes on to discuss that while there may be applications for using gigabit broadband, they are few and far between now and don’t equate to the cost when something like a 25 Mbps downstream cable modem would suffice just as well.

Allow me to disagree here.

I think one of the reasons why this article sounded flawed to me is because is sounds based on the idea that people still use one computer at a time.  The more I thought about it, the more I realized that the supposition that gigabit residential service for a single machine is overkill is indeed correct.  However, that’s where my opinion diverges.  I would argue that today’s residential networks are staring to resemble small enterprise networks with regard to bandwidth usage.

Think about all the things that you are doing with your home networks right now.  Sure, there’s a fair amount of low bandwidth web surfing going on.  We use Twitter to and Facebook to post status updates.  We check email.  We look up things on Wikipedia to win Internet arguments.  If that was it, I would say that even 100 Mbps or 25 Mbps service would be more than you’d ever need.  But go deeper.  We now use Netflix to stream movies to our televisions.  We use iTunes to download content to all manner of devices.  Hulu, Boxee, and Vudu are all clamoring for attention and bandwidth.  Even simple Bittorrent transfers can suck up an entire pipe.  Now imagine all this couple with the blah blah cloud services coming down the pipe.  We even use cloud-ish services today.  Gigabytes of pictures uploaded to Picasa and Flickr.  Video uploaded to Youtube and Vimeo.  Music streaming coming from Google, Amazon, Apple, and anyone else with a handheld device with a headphone jack.  We can even run our household phone system over the Internet.  Not to mention Facetime, Telepresence, and all manner of real-time video communications.  Sounds to me like that little cable modem is starting to get a bit crowded.

Another argument against gigabit networking is the inability of devices to use the full bandwidth.  Specifically, the lack of gigabit wireless networking is pointed out in the article.  Right now, she’s right.  However, with 802.11ac coming down the pipe and WiGig coming to the 60 Ghz spectrum sooner rather than later, I think it’s better if we have the broadband infrastructure in place sooner rather than later.  In the article, it is stated that a generic laptop only hit 420 Mbps downstream in a test.  Okay, so with a little optimization we could probably hit 600 Mbps easy.  Did they test several sites to be sure it wasn’t a transit network issue?  Did they pull from a close FTP server with a high-speed backbone?  Or were they clocking Windows Update?  Most machines will eat any amount of bandwidth you throw at them.  Even if you peaked at 500 Mbps out of the box, that’s still 5 times faster than a 100 Mbps network.  Think about what would happen in your enterprise if you granted users the ability to run gigabit all the way to the desktop.  Files could be transferred faster internally.  Content could be pushed with little effort.  Imagine again what might happen if you then brought those same users back down to 100 Mbps.  You’d have a mutiny on your hands.  When driving on the highway, 80 MPH only seems fast when you get going.  Once you’ve been cruising there for a while, 60 MPH seems like a standstill.  I think that even half a gigabit connection per machine is still amazingly fast, especially when that pipe starts getting crowded as I’ve outlined above.

The final argument is that there is no killer app that necessitates paying such high fees for gigabit service.  One service that is discussed by the author is online backup.  This, however, is dismissed as being too infrequent to be useful to a customer paying a monthly charge.  Let me ask this of you out there: how crazy did the idea of downloading music on the Internet seem when the fastest connection we could muster was 56k?  How about watching movies in our house solely over the internet when 128k ISDN was the fastest kid on the block (that was exorbitantly high priced for its time too)?  Why code an app if you know it can’t work to its fullest potential today?  What about continuous online backup?  If you’ve already got the pipe to handle it why not keep a running backup of your files out in the blah blah cloud?  HD streaming video to multiple devices simultaneously?  What about the burgeoning website designs that seem to be taking more and more bandwidth every day with Flash landing pages, Flash adds, Shockwave menus and more?  If we start running gigabit to our house, I can promise you that there will be apps written to take advantage of those big fat pipes.

Tom’s Take

Yes, running a gigabit pipe into my house would probably be overkill right now.  Despite my protestations to the contrary, my wife realizes that I don’t need to have the ability to instantly download anything and everything on the Internet.  But I also see that as we start placing more and more content and information outside of our computers and in the blah blah cloud, we’re going to get very impatient to get that content quickly.  HD video, 27 megapixel images, and enough MP3s to sink an aircraft carrier stored somewhere in an online vault and we have to have it NAO!  Just because 100 Mbps would do anyone just fine today doesn’t mean that there isn’t a market for gigabit residential service.  It’s like saying that just because we can only drive 65-75 MPH on the highway there’s no need for sports cars that can do 130.  Someone out there will find a use for it if it’s available.  If nothing else, the blah blah cloud providers should be championing us to get the fastest available connections and start storing everything we have with them.  That way, we don’t have to spend so much time worrying about where our stuff is being stored.  We just click it and go.

Rollover Beethoven – USB’s In Town

Every Cisco engine…rock star in the world should have a rollover cable or two stashed away in their bag/car/pocket just in case.  The rollover serial cable is the hallmark of access to a Cisco device.  The console port is the last resort for configuration when all else has gone wrong.  It is the first thing you should plug into when you boot up a router for the first time and the best way to get info you couldn’t otherwise find.  However, the days of the serial cable are quickly becoming numbered.

It wasn’t all that long ago that every PC manufactured included a 9-pin serial connection.  These ports were handy for all kinds of devices, including printers and modems.  However, with the introduction of Universal Serial Bus (USB) connections, the usefulness of the serial (and parallel) ports has been waning quickly.  By utilizing a higher speed connection that more tightly integrates into the system, the need to configure devices with DIP switches and play COM port roulette have long since passed.  As it is with any transition though, there have been some holdouts in the movement to retire serial ports.  While some of these are understandable due to outdated single-purpose technology, others have never made any sense to me, like the Cisco rollover console cable.  Surely there must be a better way to connect to the serial port of a device than with an outdated technology holdover from the 80s?  I myself am a victim of this kind of thinking, having used an IBM T30 Thinkpad well past its useful life simply because it had an integrated serial port and my replacement laptop wouldn’t.

When Cisco developed the new ISR G2 line of routers, someone in the console access department finally decided to wake up and get with the 2000′s.  Thanks to their efforts, the Cisco routers and switches manufactured today have started including a new console access option:

In the picture above, you can see the familiar RJ-45 console port to the right and the newer USB console port to the left, indicated with the USB icon.  This new port allows those of us that have spent most of our lives using the flat blue rollover serial cables to add a new, exciting cable to our bag, the USB A-to-mini cable.

The new USB port allows the user to access the router’s console with a newer cable instead of relying on the standby rollover cable.  However, you need to take a few steps first.  You have to head out to the Cisco Connections Online (CCO) download page and pull the driver for your particular operating system if you’re running on Windows.  Make sure you specify 32-bit or 64-bit, since this driver will be masquerading as a COM port on your system.  You don’t want to waste time downloading a driver that won’t work.  Once you’ve installed the driver, you can plug in your USB connection to any USB port and then to the router.  It will look like an additional COM port on your system, probably with a high number like COM6 or COM7, so make sure you’ve got a terminal emulator that allows you to choose your COM port.  I tend to use TeraTerm for this very reason, but your terminal program of choice should do nicely.  For those of you in the audience with Macbooks, you don’t need to download any drivers at all.  Seems like OS X already has the right driver built in, so just plug and and get cranking.  As a quick aside, Cisco will attempt to sell you a $30 USB console cable when you order the router. JUST. SAY. NO.  This is a regular USB A-to-Mini cable that can be purchased at Walmart for about $10.  You can even use the USB cable that came with your digital camera or Blackberry or old Motorola RAZR.

Once you get attached to the USB console port, you’ll find that it works pretty much the same as the RJ-45 port that you’ve become attached to over the years.  You can also plug in a regular old serial cable into the RJ-45 port if you need a second connection.  The RJ-45 console port will mirror what’s going on with the USB console port.  However, since their both Console 0, only one of them will have preference on the input.  In this case, that’s the USB port.  So if you have a terminal access server plugged in for reverse telnet connections and someone comes in and attaches a USB connector, you can watch what’s going on but you can’t do anything about it.  You can specify a timeout value if you’d like so you can force a logout after inactivity.  You can do that with the following command:

Router(config)# line con 0
Router(config-line)#usb-inactivity-timeout <value in minutes>

Note that this command doesn’t work on the 2900 series ISR G2 routers for some strange reason.  Oh well, feature request down the road.  For those of you out there that don’t feel comfortable with the idea of having just anyone off the street walking up and consoling into your router via USB, you can always disable the USB console port in favor of the RJ-45 connection as follows:

Router(config)# line con 0
Router(config-line)# media-type rj45

Bingo.  USB port locked out.  Now only those people in possession of a serial-to-USB adapter or a Redpark iOS Console Cable will have access.

Tom’s Take

I have three rollover cables in my various laptop bags.  I keep two for emergencies and one in case someone doesn’t have theirs.  I passed out console cables to all my engineers and technicians once and told them the next time I asked them for their console cable, they’d better present this one to me.  A console cable is an indispensable tool for anyone that works on Cisco equipment.  Having the USB option is always welcome since I no longer have to fumble for my USB-to-serial adapter or worry that the dodgy drivers are going to bluescreen my Windows 7 64-bit laptop over and over again.  Still, there is a lot of Cisco equipment out there with the older RJ-45 cable setup as the only console option.   So you can’t just throw out the old rollover serial cables just yet.  Better to throw a USB cable in your bag for those glorious days where you get to access a newer device.  Then you can await the day when you can bury your rollover cable alongside Beethoven.

Cut Me Some SLAAC, Or Why You Need RA Guard

The Internet has been buzzing for the last couple of weeks about a new vulnerability discovered in IPv6 and the way it is interpreted by networking devices.  Firstly, head over to Sam Bowne’s excellent IPv6 site and read his assessment of the attack HERE.  What is being exploited is a “feature” in IPv6.

Since IPv6 doesn’t use Address Resolution Protocol (ARP), it relies on ICMP and Neighbor Discovery messages to determine neighbors on the network.  It also uses Router Advertisements (RA) to build a picture of how to get off the local network.  When the Stateless Address AutoConfiguration (SLAAC) flag is set in the RA, the local host will choose an address from the announced address space and begin using it.  This is a great addition to the protocol, since it allows a network admin to setup an automatic addressing protocol that isn’t reliant on a server like DHCPv6.  However, from a security standpoint, it introduces some possible problems.  If a host on the network were to start sending RA packets to the LAN, that man-in-the-middle could start influencing packet routing.  Worse still, if the attacker isn’t really interested in rerouting packets, they could just take the anarchist’s approach and burn the whole network down with a specially-crafted DoS attack.  By flooding a ton of RA messages onto the local network with different network address spaces, the attacker can cause the CPUs on Windows and FreeBSD boxes to spike to 100% and stay there indefinitely.  This is because the host system continually tries to process the RAs flooding in from the network and starts trying to pick address space in every network announcement that it hears.  This consumes all its resources with updating the routing table and addressing the adapters on the system.  This could cause problems for your end users should an attacker get into a position to launch RAs into your LAN.  Right now, there are a couple of ways to fix this:

1. Disable IPv6 – Okay, this doesn’t fix the problem it just makes it go away.

2.  Disable RAs on the local network – Again, not a fix, just hiding it.  Plus, this breaks SLAAC, which I see is a real advantage to IPv6.

3.  Install a firewall or ACL on your host-facing ports to block RAs or filter out the ICMPv6 packets carrying them.

What I find even more interesting about this whole affair is the response of the three biggest players in the game in regards to the issue.  Let me sum it all up using their words:

Microsoft – This is Working As Intended (WAI).  We don’t plan on fixing this.

Juniper – We need to work with the IETF and figure out a standard solution to address this problem, and until we do we aren’t patching against it.

Cisco – We fixed this last year, and by the way have you heard of RA Guard?

Cisco has implemented a solution very similar to what they do with DHCP snooping on IPv4 switches.  They call it RA Guard.  As defined in RFC 6105, RA Guard can be enabled on all host-facing switchports to filter RAs from non-trusted sources.  In this case, the trusted source would be a switchport you know to contain a valid router, so you wouldn’t enable RA guard on it.  The RFC defines a discovery method using Secure Neighbor Discovery (SeND) that made me chuckle because the four states of the discovery are the same as 802.1w Rapid Spanning Tree.  Seems we’re never going to get rid of it.  When you enable SeND-based RA Guard discovery, it can dynamically scan the network for devices broadcasting RAs and block or allow them as necessary.  That way, you don’t have to worry about misconfiguring a switchport and killing all the advertisements coming from it. By enabling RA guard on a Cisco switch with firmware 12.2(50)SY, you can effectively mitigate the possibility of an unauthorized attacker DoSing your entire network with what amounts to a script-kiddie style attack.

Tom’s Take

Take a vulnerability that has been known about for two years but swept under the rug, add in a dash of vendor disregard, and shake until the Internet security community is frothing at the mouth to tell you that you should turn off IPv6 on your entire network.  Sounds like a recipe for overreaction to me.  I’m not denying that it is a serious vulnerability.  In fact, given the fact that IPv6 is enabled by default on the current Windows version, it could cause issues.  That is, unless you are smart and take measures to fix the issue rather than sweeping it back under the rug.  Rather than just turning off IPv6 until someone other than Microsoft releases a patch, we need to work through the issue and fix the underlying security issue.  At the same time, this needs to be agreed upon by the major networking vendors sooner rather than later.  The longer this issue exists in its present form, the more security sensationalists can point to it and decry one of the advantages of IPv6 on your network when in fact they should be focusing on the lack of security in the software that allows anyone to masquerade as an IPv6 router.  Then, maybe we can cut IPv6 a little bit of slack.

My Thoughts on IOU-For-Learning

This week, Learning@Cisco announced a new program designed to help those people out there that want a virtualized router platform upon which to study for the CCNA and CCNP.  While the idea behind an emulated IOS platform is one that has been desired for a long time, what Cisco released today isn’t quite what we’ve been clamoring for.  The new programs use the now-famous IOS on Unix (IOU) setup that has been used internally at Cisco for a while now and was made famous by Jeremy Gaddis in this post.  This is also the same platform that is used in the troubleshooting section of the CCIE Routing & Switching Lab.

The new program is completely hosted by Cisco.  All of your access to the IOU environment is done via web and SSH.  You, as the end user, have no access to the files that comprise IOU.  Since the emulator is presented as a component of a learning package, there is no opportunity to modify the topologies presented.  They are canned and align with the courseware you purchase.  This is great for people that are just starting out in the networking world that have no access to the proper gear to learn how to enable telnet sessions and address an interface.  By limiting the access you have to a topology, you get rid of some of the confusion that surrounds tools such as GNS3, namely the dearth of options that tend to confuse the first-time users.

I have a couple of problems with what Cisco’s released so far:

1.  IOU isn’t a true layer 2 emulator.  The software that comprises IOU is great at simulating IOS running on a router.  That’s because it’s essentially an IOS image that has been modified to run on a different “hardware” platform.  So long as all you are worried about is working with routers, IOU is a great resource.  However, if you really want to dive into the second layer of the OSI model, you’re going to come up short rather quickly.  Basic layer 2 configuration is fine for a CCENT/CCNA type of student, but by the time you reach the CCNP level of switching, you’re going to find the interface of IOU wholly unsuitable.  Since IOU emulates a router, it has to emulate switching as it would be on a router with an ESM switch module.  That means that anything that relies on an ASIC to function, such as QoS, is right out the window.  Which means that some of the more esoteric and hard-to-learn parts of using IOS on a switch remain off-limits.  I’ve been able to use 16-port switching modules in GNS3 to emulate switches for some of my studies, but I quickly reached the limits of this configuration with things like advanced spanning tree configuration or specialized tasks like Storm Control.  I think that Cisco needs to put a little more effort into providing an emulated environment for switching.  Finding a way to emulate the ASICs of the QoS functions would make those learning VoIP QoS on 3560/3750 switches much happier.

2.  There’s still no proof-of-concept for engineers.  As luck would have it, I have a small lab at $employer to test some of the things customers ask me about.  It’s been cobbled together with bits and pieces of cast off equipment over the years.  Where I run into trouble are those cases where the customer has a setup that I can’t quite reconstruct with the equipment I have.  What would be nice is a kind of emulation environment that allows me to reconstruct this setup quickly.  This is the perfect scenario for something like IOU.  Being able to quickly reconstruct a customer’s environment or duplicate your own environment for things like change control and internal testing would be a dynamite idea.  By utilizing a Cisco UCS cluster with the right topology files, I could have my WAN configuration duplicated and run several sample configs for maintenance window changes quickly with the capability to roll them back if something horrible breaks.  That’s where the true power of having an emulator lies for the advanced engineer.

3.  Strict control of IOU cuts out the “gray market”.  It’s no big shock that Cisco has taken the stance with the 360 Program that you’re either with us or you’re the “gray market”.  Vendors like Internetwork Expert (INE) and IPExpert have their own courseware and rack space designed to aid their students.  These racks use real routers and switches to allow students the ability to do practical studying.  However, these kinds of study aids are prohibitively expensive for a training provider to get into.  Now, imagine if you could fire up and virtual rack of routers and switches for your students at the touch of a button.  The barrier to entry becomes much lower to those companies wishing to get involved in the training market.  The possibility then exists that you could have some bad apples in the bunch that might dilute the training offered to students and put a black mark against your name.  By holding all the cards in the IOU discussion, Cisco ensures that the technology never leaves their house, so any training partners wishing to leverage the power behind the emulated IOS platform must abide by Cisco’s rules if they want to keep playing.  Cisco can then force training partners to use 360 materials or the equivalent for CCNP/CCNA/CCENT training.  That forces the non-Cisco approved partners out of the space sooner rather than later.

Tom’s Take

Cisco’s getting to the educational platform party ahead of some of the other network vendors, like HP and Juniper, but they’re doing it with baby steps.  High level engineers have been hoping for a truly unlimited emulator for testing things for quite a while now.  I think they’re still going to be waiting for a while to come.  This new learning program is leveraging IOU to replace aging programs like the Boson Network Simulator or the NetSim products.  By tailoring it toward the entry-to-mid learner, it allows them to work out the kinks in the presentation while still keeping control over the platform for the time being.  I’ve heard that they will expand this idea to encompass security offerings and one day the CCIE as well.  I think that the IOU Learning Platform will be integrated into the 360 program and will only be offered as a part of the materials that you receive from your subscription to it.  I seriously doubt that even a CCIE-level student will have unfettered access to IOU in their own lab, since the possibility of a non-crippled version of IOU being readily available creates too many complications for Cisco support.  It’s already fairly easy to get a copy of IOU if you know where to look.  Imagine what would happen if a copy from a CCIE candidate got out into the wild without fixed configurations or limitations that you face in the hosted CCNA version?  I applaud Cisco for the steps they’ve taken in the right direction for allowing students access to emulated educational software.  Now it’s time to observe what happens and meet the needs of those of us on the other end of the scale.

If you think that Cisco needs to offer a full IOS platform for educational purposes, please head over to Greg Ferro’s site and put your digital signature on the educational IOS petition.  The more signatures that are gathered, the more pressure that can be brought to bear on Cisco to show them the will of the engineer.

These /8s Are Now Diamonds

The end times are upon us.  According to many reliable sources, the allocation of IPv4 addresses is quickly reaching it’s conclusion.  Of the final seven /8s available to be allocated by IANA, APNIC is about to exercise its option on two of them.  When that happens, the final 5 will be allocated as planned to each of the 5 Regional Internet Registrars (RIRs) and completely deplete the source pool of allocatable IPv4 addresses.  Now, before Chicken Little starts screaming about what this means, let’s take a step back and examine things.

I liken the total allocation addresses from IANA to the RIRs to a resource exhaustion.  For the sake of discussion, lets pretend the IPv4 addresses are diamonds.  The announcement of of total allocation would be like De Beers announcing that all of the diamonds in the earth’s crust have been mined and there are no more available.  That doesn’t necessarily mean that all the engagement rings and tennis bracelets for sale will disappear tomorrow.  What it means is the primary source for this resource is now depleted.  Just like we can’t manufacture any more IPv4 addresses, in this example we can’t mine any more diamonds.  So what happens next?

Usually, when a resource starts becoming scarce, the cost to acquire that resource will be driven up.  In this particular case, people like Greg Ferro have already suggested that there will be a “run” on addresses.  Again, this is a behavior that is typically seen when the announcement is made that a resource is becoming hard to come by.  I think that the RIRs will start putting policies in place to prevent ISPs and other parties from requesting more IPv4 addresses than they currently need.  That will prevent the pool that the RIRs currently possess from becoming depleted faster than necessary.  It will also hopefully stave off the resale of these addresses on the black market, which is a distant possibility but possible nonetheless.  Just like in our fictional diamond example, the price of the diamonds at the jewelry store will go up, and most stores will implement policies restricting the sale of large numbers of stones to single parties, so as to prevent hoarding and help keep prices high.  This will also stave off the onset of a diamond secondary market, where speculators will sell stockpiles of stones for exorbitant prices.

So, now that IANA has run out of addresses, what’s next?  Well, the next countdown becomes the date the first RIR runs out of it’s allocation.  Right now, that’s projected to be APNIC sometime in October 2011.  APNIC has been burning through its addressing at blinding speed, and so they find themselves at the head of the IPv4 exhaustion line.  Once APNIC runs out of addresses, the only thing they can offer their customers going forward is IPv6 address space.  For some customers, this will be rather unsavory.  These types of customers will feel that IPv6 hasn’t penetrated deep enough into the market.  They’ll pay any price for those precious v4 prefixes.  So, I imagine that APNIC and the other RIRs will hold a small portion of IPv4 address blocks in reserve for those customers that are willing to pay big bucks for them.  For the customers without the pocketbook or that don’t care about the address space they receive, they’ll get IPv6 and likely won’t think twice about it.  Just like in our diamond example, when the first jewelry supply runs out of stones purchased from De Beers the price will start going up.  Perhaps they’ll offer lab-created diamonds of similar quality.  But there will be customers that feel the lab-created stones are inferior and those same customers will pay a significant amount of money to get the “real” stones.  In these cases, the smart jewelry supplier will hold back some of the best gems in order to get a much better price for them.

Once the first RIR runs out of address, things will accelerate from there.  Depending on the level of IPv6 preparedness in the market, you may start seeing customers hosting equipment in locations where RIRs still have address space to assign.  I would sincerely hope that by the end of 2011 that most everyone has either begun their IPv6 prep in earnest or completed it with flying colors.  Otherwise, I predict there will be a migration of data centers to locations served by AfriNIC, which is the RIR with the largest block of unallocated /8s.  The final exhaustion of IPv4 addresses isn’t predicted to occur until July 2012.  That gives customers and plenty of time to decide how to implement IPv6 rather than moving data centers around to mop up what little IPv4 address space remains.  In our fictional example, as the jewelery suppliers start running out of diamonds to give to their customers, their customers will begin shopping around to find suppliers that still have stock, even if they have to start importing stock from supplies located overseas.

Once the last RIR runs out of addresses to give to its customers, then it will be a matter of time before the last of the IPv4 addresses are allocated to the final end users by the ISPs and other middle men.  Customers that deal directly with ARIN and RIPE and the other RIRs will be out of luck, instead needing to move to IPv6 to continue growing their Internet presence.  Hopefully by the time this occurs in late 2012, IPv6 will be firmly entrenched and the drive to allocate the final IPv4 address space will be greatly lessened.  With end users concentrating on their shiny new IPv6 address blocks, the last of the IPv4 addresses can be handed out to those truly in need.  After that, the Internet can go forward operating on a dual-stack of IPv4 and IPv6 until the last of the IPv4-only hosts go dark, leaving us totally on IPv6.  I doubt that day will ever truly come, but it’s a possibility that’s out there.  And in our fictional diamond example, people will still show off their fancy jewelry, but the world at large will start to turn to the next precious stone like rubies or sapphires.  While diamonds will never truly be gone, the demand for that which can no longer be obtained will be lessened greatly, only pursued by those with the resources to expend to obtain something so expensive.

The final and total allocation of IPv4 isn’t something that’s going to happen overnight.  It will be a death by degrees.  Just like boiling a frog one degree at a time, there was a time we might not have known what was going on until it was too late.  Thankfully, enough people have been getting the word out to cause everyone to start making their plans early and get ready for what is coming.  This was no more apparent to me as when I contacted a local technology group putting on a conference to request a presentation slot to speak about IPv4 exhaustion and IPv6 planning.  The person on the other end of the phone was a rather technical person, yet I had to spend some time explaining just what IPv6 was and why getting the word out was so important.  So, to those of you that have influence on communication and/or blogging and tweeting avenues, keep talking about what’s going on with IPv4 depletion as we approach the true end of the address space.  That way, we don’t find ourselves scrambling for diamonds at the last minute or wondering if we need to upgrade to Diamondv6.