The Century Club

Posted on by
2

I’m not usually one for milestones on things, but I wanted to take a second for my 100th blog post.  In truth, when I started this thing back in September, I never thought I’d have a hundred posts in me.  I just wanted to get a few things off my chest that wouldn’t fit into 140 characters on Twitter.  I never expected anyone to take an interest in my ramblings about stuff.  But, for some reason, you all came to see what was floating around in my mind and it’s really taken off from there.  This blog has been more than just an outlet for harping on about NAT or being funny some of the time.  It’s also helped me to hear from vendors I might not otherwise talk to, get a weekend job running my mouth, and enjoy the fame and fortune of being a semi-famous technology blogger.  Still waiting on the fame and certainly the fortune, though.

Since making my very first post on September 23rd, I’ve taken 234 days to post 100 entries.  That works out to be a post about every 2.5 days, which is much better than the unofficial standard I hold myself to of trying to post about once a week.  I’ve had about 31,000 visitors so far, on average about 190 a day.  Not bad for someone that thought they’d be lucky to get 10 readers.

I thought it might be fun to revisit the most popular posts out of my first hundred.  I was a little shocked by some of them:

#5 – The Recertification Treadmill Aside from my huge Wall of Shame behind my desk, I get a lot of questions about certifications in the industry.  People want to pass the CCNA or the JNCIA and then run right out and get a job.  This post was a way to let people know that there is another side to doing nothing but taking tests all the time.  It was also my hope that certification organizations would perk up and start offering CPE-like credits for recertifying.  Eh, maybe next year…

#4 – Fruit Company Console: My Review of the Cisco Console Companion for the iPad/iPhone I really liked this one.  I got to review a piece of technology that people were curious about and put it through its paces.  I wrote a long article with tons of pictures and explanations about the inner workings of the cable and the software.  I even got the developer to leave a comment about upcoming features!  This post was responsible for my largest amount of traffic in one day, 523 page views.  It was kind of humbling to see that people wanted to hear my opinion about something.

#3 – Hooray for Bruno! My discussion of the addition of layer 2 troubleshooting to the CCIE lab and my summation of the Cisco discussion threads about it.  Not that big a deal when it came out, but it consistently gets hits from search engines each week.  My post right before it about the announcement of the addition of the layer 2 stuff was a little more emotional and perhaps a little over the top, but I think getting grounded back in the reality of things and posting a follow up with hard facts as opposed to screaming was a little better overall.  I guess CCIE candidates are still curious about what to expect inside that little room, and Google sends them my way for some reason.

#2 – God Help Us, We’re In The Hands Of Engineers Without a doubt, the post that really launched my blog.  Crafted because of a comment someone left on Jeremy Stretch’s blog, this post took off like wildfire.  I got tons of DMs about it from my fellow enginee…rock stars telling me they loved it.  My post was result of  my irritation at the idea that people in the IT industry can’t use the term “engineer” to describe themselves.  My comment leveled at P. Eng was my frustration at the fact people think it’s a title that should be put on a pedestal.  At the end of the day, I might have taken the troll bait, but I felt better about things, and from what I’ve been told a few other people did as well.  I still use the term “rock star” on here to refer to network engineers as a way to poke a bit of fun at this post.  This was the first post on my blog to reach 1,000 views.

#1 – CUVA Windows 7 64-bit Support To be completely honest, this is the most shocking thing I’ve seen in my first hundred posts.  I wrote this post when I was trying to get my old CUVA camera working with my new Windows 7 64-bit laptop.  I found the answers scattered across a Cisco forum and buried 6 or 7 pages into the thread.  I decided to clarify them a little and post them here so that I could find them again if I needed them.  This wasn’t an instant success like the Engineer or Console Cable posts, but it does receive the most consistent traffic stats each week.  I guess that there are enough CUVA cameras out there that don’t work with the new versions of Windows that people want to start searching the Internet to make them operational again.  This post only recently overtook the Engineer post as the #1 viewed post on my site, and at the rate it gets hits, I doubt anything will overtake it in the foreseeable future.

Those are the highlights of my first hundred posts.  How about the next hundred?  Well, I considering starting a sort-of weekly link posting.  I read a lot of stories during the week, and I discuss some of them on the Packet Pushers podcast, but I don’t get to talk about all the things I digest.  I figure by highlighting a few of them, especially the good commentaries, it will help share some of the things I encounter during my scouring of the Internet.  What else?  I’m really curious to hear what you have to say about when I’ve written.  More humor?  Less humor?  More technical deep dives into things?  More CCIE stuff?  Less CCIE stuff?  I write whatever pops into my head most of the time, but if I know there are things that my audience likes to read, I can focus more on those.

In the end, I want to thank each and every one of you that read my blog.  Committing thoughts to phosphors doesn’t really impact me one way or the other.  I would probably do it even if I had zero readers, as it kind of cathartic to be able to get this stuff out of my head and down someone semi-permanent.  But my readers take time out of their busy lives to read through things and post comments, discuss, and share my thoughts with others.  It goes without saying that my attendance at Tech Field Day and my regular spots on Packet Pushers would not have been possible without all of you believing in me enough to stick with me while I figured this whole blogging thing out.  Once again, thank you from the bottom of my heart.

Stay tuned for the next hundred posts.  We’ll see what happens from here.

Unity In Your Community – Unified Messaging

Posted on by
1

Voice mail is a funny thing in the IT industry.  Some people live by it and use it as their primary form of communication.  Others would prefer to take voice mail and throw it into a river.  Regardless of your views, configuring voice mail to integrate with other forms of communication has been an interesting task in the past to say the least.  I’d like to take a few words to talk about unified messaging as I understand it and why I find it so critical in my infrastructures that I design and build.  Note that I’ll be discussing Cisco unified messaging for the most part, with a little Microsoft flavor thrown in here and there.

If you’ve been using a Cisco phone system at any time in the past 10 years, you are probably intimately familiar with Cisco Unity.  Unity has been the flagship voice mail system for Cisco from the very start of their journey down the road of unified communications.  Unity at its heart is a place to store voice mail messages and retrieve them, usually via phone.  It allows you to provision a server for many different kinds of voice mail interfaces besides Cisco CallManager, such as Octel or PIMG.  However, the thing that has most attracted people that I talk to is the promise of what Cisco calls “Unified Messaging”.  This is the idea that your voice mail messages are stored in the same container as your other forms of communication, in this case emails.  By locating the voice messages in the same area as your email, you only need to look at one program or portal to receive all of your messages, voice or electronic.  That is the real promise of unified communications, and the one that I managed to sell my users on.  Couple in the fact that you could receive email on a smartphone and you can get your office voice mail no matter where you might be.  However, Unity is not without it’s drawbacks.

How exactly do you pull off unified messaging?  Well, the key lies in the fact that Unity on supports unified messaging on Microsoft Exchange or Lotus Domino platforms.  You need to specify your platform when you order.  Why’s that?  Because the Unity server you load is actually an Exchange or Domino platform that integrates with your existing environment.  Even if you order a Unity server as a stand-alone voice mail server, you are still loading Exchange and using it to store voice mail messages, except you can only retrieve them via telephone (unless you are crafty…).  Unified messaging can only work if you are storing the messages on a “partner” server, which is a box other than the Unity server.  Essentially Unity serves as a gateway to move messages to the Exchange or Domino server.  This isn’t without its challenges.  Unity requires your directory schema to be extended in order to support unified communications.  Since the latest versions of Unity still load Exchange 2003, you must be able to support that version in your environment, which can be difficult for those that have been running Exchange 2007 or 2010 for a while.  Any time you install a major patch to your partner Exchange server, you have to patch Unity to work with it.  When we upgraded our Exchange 2003 server to 2007, it took two major patches and a few hours to sort out the resulting mess.  I didn’t even want to think about what might happen once we upgraded to Exchange 2010.  Unity 8 has dropped all support for Domino, so if you want to keep unified messaging with Domino you’re kind of stuck.  You still need to use Windows Server 2003 to install and support Unity.  The list could go on for a while, but all you really need to do is go find a Cisco voice person and casually mention Unity to them.  Odds are good you’ll see them twitch, followed by horror story after horror story about Unity.

Cisco has been developing a different form of server for the past few years in parallel to the Unity development.  Originally designed to be a smoother integration with Cisco CallManager, Cisco Unity Connection was positioned to those customers that didn’t need full unified messaging and didn’t need thousands of subscribers.  While the 1.x versions still relied on Windows as the server OS, the message store didn’t not need Exchange or Domino to function.  As Cisco started migrating the CallManager platform from Windows to a Linux-based OS, so too did Unity Connection slowly migrate to the same Unified OS under the hood.  This helped Cisco avoid paying license fees to Microsoft for each CallManager or Connection system sold.  The shift to Linux came in part because Cisco could release security patches for the whole platform at once without the need to wait on Microsoft to patch OS vulnerabilities only to then have Cisco test those same patches to ensure the CallMangager or Connection software still functioned.  Also, when Microsoft introduced Exchange 2007 they included a role for Exchange called Unified Messaging.  This was the beginning of Microsoft’s push toward voice software, and many say that it came because Cisco was having so much success using Exchange as a voice message store that Microsoft wanted to jump in the game.  Hence, Cisco had a great desire to move their voice messaging platforms to a non-Microsoft OS.  There was still a problem, however.

Without Exchange or Domino, there is simply no way to provide true Unified Messaging.  Cisco attempted to do something similar in Connection 6.x and 7.x with what they called “Integrated Messaging”.  This allowed a customer to attach to Connection as an IMAP mailbox or to have Connection deliver a copy of the message to your Exchange mailbox as a forwarded email.  This presented a couple of problems for customers such as the users that I support.  Firstly, my users wanted the ability to check their messages from outside the office.  With Unity, all their messages are delivered into their mailboxes, so they just check the one email account.  With Connection, they needed to log into the Connection server, so I would have needed to expose the Connection server to the outside Internet, something I wasn’t exactly comfortable with.  Secondly, with Unity when a message is checked in Outlook or on the web, it changes the state of the message waiting indicator (MWI) to reflect that the message has been listened to.  For some of my users, the volume of voice mail they receive would cause a panic attack if I told them they had to listen to each message on the phone to clear that light.  Because of things like these, no matter how much I may have hated Unity, I couldn’t get rid of it because Connection didn’t do everything I needed it to do.

Enter Unity Connection 8.5.  Cisco has gone to great lengths to create a true unified messaging platform, sans Exchange.  Sorry for those Domino users out there, but Connection 8.5 only support unified messaging on Exchange.  I think both of you Domino people left in the world will have plenty of time to think about where you’ve gone wrong.  Anyway, Unity Connection 8.5 uses an agent to track the message state in the message store and synchronize it with the copy of the message that is deposited in your Exchange mailbox.  Listen to the voice mail on your phone and the message is marked as read in your inbox.  Delete the message from your inbox and it is removed from your phone and placed in the trash, which is a lot better than it just being outright deleted like it was in Unity.  This is the kind of unified messaging behavior that finally got me to the point of migrating my users off of Unity.  After judicious use of COBRAS, I was able to get my users up and running on Connection and shut off Unity once and for all.  The only “change” was a couple of users asking me if the Cisco Lady’s voice had changed for some reason, since the Connection voice prompts were a little newer than the old ones found on Unity.

Tom’s Take

Unified messaging is a great thing, but the infrastructure that needs to be in place to support it isn’t.  It gives us headaches and indigestion from all the moving parts needed to make the whole thing work correctly.  Those that have survived a Unity install and have labored to support it would do well to take a look at Unity Connection 8.5.  It provides almost every feature you might want from Unity while leaving all the old Microsoft cruft behind.  For those that have never had the ability to use unified messaging, you should definitely give it a try.  You’ll find your users thanking you.

The Land Of Opportunity

Posted on by
1

It’s been *checks watch* about 38 minutes since I last thought about NAT, so it’s probably time to sit down and write some more about it.  I’ve got an image to uphold after all.  I wrote about my position on NAT a couple of posts ago, and in it I discussed NAT64.  I railed against it much the same as introducing any kind of NAT into IPv6.  Ivan Pepelnjak, a very well respected and frighteningly intelligent networking rock star, listened to the Packet Pushers show that fueled my rant and read my blog post.  He then posted an article where he takes NAT64 and puts it into proper perspective.  I read through it, and guess what?

He’s absolutely right.

Yep, Ivan nailed that one.  NAT64, which is the process of translating IPv6 addresses into IPv4 addresses, has a use when it comes to allowing the IPv6 Internet to access content that is still stuck on the IPv4 Internet.  Without some sort of translation mechanism, those IPv6-only hosts will be walled off from the IPv4 Internet in general.  The other methods he discusses are either completely insane, like Carrier-grade NAT (NAT444), or are impractical from a support standpoint, like dual-stacking.  I happen to think dual-stacking is the way to go with this, but I don’t want to be the local ISP trying to support a D-Link router running a dual stack when someone’s mom calls for support.

Ivan’s final point of the article hits home in a way that should make people sit up and pay attention.  “The proper way to tackle this issue is to make your content available over IPv4 and IPv6. IPv4 clients won’t notice it and IPv6 clients will use native IPv6 connectivity bypassing NAT64.”  He’s spot on with this one too.  If you build your content aware of both IPv4 and IPv6, you won’t have any real issues when it comes to IPv6-only hosts.  I’m going to take this one step further, though.  Let me ask a question that I think really cuts to the heart of the Internet in general when it comes to content creation and consumption:

What content would you access that is IPv4-only?

Right now, that question would be answered with “everything”, since IPv6 adoption is spotty at best.  However, with World IPv6 Day looming in less than a month, the sponsor list is growing quickly.  Google, Yahoo, & Facebook are already on board, and 163 more companies are ready to flip the switch as well.  If you think about it, there’s a great importance that should be attached to making your content IPv6 accessible.  In my IPv6 presentation, I talked about the impact of new content providers in the APNIC region creating things that you won’t be able to see if you are on the IPv4 Internet, since they are out of IPv4 addresses totally at this point, for all intents and purposes.  However, look at it from the perspective of an IPv4-only content provider.  You’ve got all this great content that you want to serve out to your audience.  Many of your audience members are starting to hear about IPv6 and wonder how it will be implemented.  More still, like those in the APNIC region, are unable to view your IPv4 content right now.  For those hopping on the IPv6-only Internet, it probably looks a lot like it did back in Vint Cerf’s day – a whole lot of nothing.  If you want to stake your claim for your content, are you going to wait for someone to come out with a NAT64 appliance?  Are you going to sit around until an IPv6-to-IPv4 transition is possible on a load balancer?  If you are, you had best start packing up your office, because you won’t stick around for long.  The handwriting is already on the wall.

World of Warcraft, the largest Massively Multiplayer Online Role-Playing Game (MMORPG) enabled IPv6 connectivity in their recent v4.1.0 patch.  They’ve been talking about it since March.  Now, those 10 million subscribers won’t have to worry about NAT64 if they get assigned an IPv6-only prefix.  They’ll just keep on playing the same way they’ve always played.  I think it’s going to end up being the same for 75% of the services that you use today.  Things like e-mail, calendaring, and popular websites will be IPv6 ready well ahead of any kind of IPv4 exhaustion.  Those that rely on advertising revenue won’t hesitate to get IPv6 enabled so as to not lose out on an untapped market of IPv6-only hosts.

Tom’s Take

NAT isn’t pretty.  It’s a necessary evil.  It has uses, and as Ivan pointed out they can be pretty important to keep the content-rich Internet from looking like a barren desert.  But at the same time, there is a path away from the need to slap a NAT64 band-aid on things.  By enabling IPv6 now, you avoid the expense and hassle of needing to wait for NAT64 to be finalized and argued about until the vendors are blue in the face.  You, as a content provider, can serve your content and ads and subscriptions to a whole new world of consumers in this new land of opportunity while the IPv4-only world gets left by the wayside, trying to figure out how to patch the problem rather than fixing it right the first time.

As a side note here, if you are at all interested in IPv6 and its implementation and impact, you need to sign up for Ivan’s excellent webinars.  He’s a genius when it comes to MPLS, IPv6, and datacenter networking.  In fact, do yourself a favor and save money by just buying a subscription to all of them.  At $199, it might sound like a pricey purchase, but since you’re going to want to listen to everything he’s got to say, it works out to be a great investment in your future.

The Three B’s of Innovation

Posted on by
8

Innovation is key in our lives.  It allows us to go from adding on our fingers to having a calculator capable of solving quadratic equations.  It shows that we can graduate from playing checkers all the way up to tossing disaffecting fowl at inferior porcine construction workers.  Innovation, however, is hard work.  It requires significant time and investment to realize anything good.  Helen Hanson said, “Inspiration is the windfall from hard work and focus.”  However, in the tech arena, innovation doesn’t always seem to come from hard work.  I always like to think of the source of tech innovation as coming from three different options: building it, buying it, or b*tching about it.

Building It – This one is the classical source of inspiration.  Look at companies like Selsius developing CallManager, or Cognio’s spectrum analyzer chips.  More often than not, these companies are lead by brilliant individuals with a kind of hyper focus that allows them to shut out all distractions and build the next better widget.  Ryan Woodings of MetaGeek is a great example.  He spent lots of time and hard work analyzing a need and creating a brilliant product to address it.  Companies that go out on a limb to build better widgets help make the world better with new ideas to address our needs and desires.  The only problem with this kind of development is the amount of time it takes to come up with these ideas.  You must be willing to invest a large amount of resources to achieve your goals.  How many inventors and innovators have gone broke trying to realize their dreams?  Even Ryan had to struggle with keeping his day job until MetaGeek took off and became lucrative enough to be his new day job.  Large companies like Cisco and Juniper suffer from the same problem on a different scale.  They must sink huge budgets into research and development in order to create new products.  Sometimes, the people in charge don’t take well to R&D budgets spiraling out of control, so they cut back and risk stifling innovation.  You also run into issues with your company being brilliant, but unknown.  How many times have you heard the phrase, “The best company no one’s ever heard of.”?  Chances are the kind of company that has brilliant people with an acute ability to focus on product innovation may not have the vision to tell people about their wonderful new widget.  If that happens, they may wither and die before they get famous because no one knew who they were and how great their product was.  In many cases, this leads to…

Buying It – If you don’t have a particularly inspirational idea or an innovative team to design something new, but you happen to be sitting on a pile of cash the size of a dump truck, you always have the option of buying innovation.  This isn’t always a bad thing.  If you have a company that has a great product and no exposure, you can take your money and invest it in the technology and people and market it yourself.  It’s also faster in a lot of cases to purchase the time and effort that someone else has put in rather than reinventing the wheel.  John Chambers at Cisco has a philosophy that if Cisco can’t be the best in a market, they’ll acquire the first or second place company and make them the best.  Cisco bought Selsius and Cognio and Profigo and Protego and a whole host of other companies.  HP has purchased Colubris and 3COM and 3Par. Microsoft recently purchased Skype.  Even Juniper has purchased Trapeze.  Acquisitions happen all the time.  At one point, Cisco was even funding engineers to branch out and start their own companies to perform research and development into new product lines.  If they made good, Cisco would come back and buy the company and rehire the old engineer as the new director of that particular product’s development.  That’s not to say that buying your R&D is always the best method.  Eventually, you will run out of cash to buy companies, and then your research and innovation goes kaput.  Many of the best companies are just too big to swallow, no matter how much money you have to buy them.  It’s also very tough to integrate the development teams from purchased assets into your fold.  How many time do you see a company be purchased, then have the lead team leave six to eight months later?  People who are used to having total control over the entire creative process sometimes don’t take kindly to corporate oversight.  The have a difference of opinion and out the door they go, ostensibly to form a new company around the same idea.

B*tch About It – I’m seeing this becoming a bigger and bigger method of innovation recently.  It seems that when a company comes up with an idea, a competitor does their best to knock it down several pegs.  Whether it be to buy time to bring their own strategy to market or to pitch an idea with a totally different product, the idea is that by complaining (often loudly), you can force your people to innovate while making sure the other guys can’t sell their widgets.  For a particular example, I’m going to pick on TrILL.  Right now, TrILL isn’t a standard.  Other companies, like Cisco and Brocade, have come out with proprietary features that function much like TrILL eventually will, but don’t totally interoperate.  Others have decided to do something totally different, like Juniper or HP.  Right now, the battle of PR is being waged by Juniper in regards to their QFabric being totally different that TrILL or Cisco’s FabricPath.  HP is waging a PR war against Cisco from the standpoint that FabricPath is not standards-based, while HP is content to wait for TrILL to become ratified and rely on their own proprietary IRF solution in the interim.  To me, the innovation here isn’t so much centered around what any one particular company is capable of, but rather what there competitors are incapable of doing.  FabricPath isn’t a standard.  IRF doesn’t scale to the Nth node.  QFabric is currently a pipe dream without substance.  Every vendor is guilty of attacking their competitors rather than extolling the virtues of their particular solution.  If you spend your time telling me what your product does rather than concentrating on what your competitor’s product doesn’t do, you are much more likely to convince me to buy your widget, or so the thinking goes.

Tom’s Take

Creation and innovation take resources.  Plain and simple.  You’re either going to use brainpower to invent something, money to buy something, or hot air to talk about how much something sucks.  I love people that are creative enough to think of things that no one else has thought of before.  Late night television is littered with them. Others aren’t as creative, but have the resources to make those creative people prosper in a great environment.  These kinds of innovators are tried and true and have my utmost respect.  The last group, the whiners and complainers, not so much.  The amount of effort it takes to sell me on the idea that someone else’s hard work isn’t up to snuff could be better directed in developing new products or ideas.  That negativity can be turned into hugely positive things if only we take the time to focus.  I make an effort to stop presenters once their message becomes all about how something isn’t great and make them tell me about how great their thing is instead.  You quickly see who the true innovators are.  Real innovators are proud of their accomplishments and won’t hesitate to talk about them.  The whiners and complainers fall apart when they aren’t allowed to use negativity to sell themselves.  Think about that the next time you get ready to innovate.

Cut Me Some SLAAC, Or Why You Need RA Guard

Posted on by
4

The Internet has been buzzing for the last couple of weeks about a new vulnerability discovered in IPv6 and the way it is interpreted by networking devices.  Firstly, head over to Sam Bowne’s excellent IPv6 site and read his assessment of the attack HERE.  What is being exploited is a “feature” in IPv6.

Since IPv6 doesn’t use Address Resolution Protocol (ARP), it relies on ICMP and Neighbor Discovery messages to determine neighbors on the network.  It also uses Router Advertisements (RA) to build a picture of how to get off the local network.  When the Stateless Address AutoConfiguration (SLAAC) flag is set in the RA, the local host will choose an address from the announced address space and begin using it.  This is a great addition to the protocol, since it allows a network admin to setup an automatic addressing protocol that isn’t reliant on a server like DHCPv6.  However, from a security standpoint, it introduces some possible problems.  If a host on the network were to start sending RA packets to the LAN, that man-in-the-middle could start influencing packet routing.  Worse still, if the attacker isn’t really interested in rerouting packets, they could just take the anarchist’s approach and burn the whole network down with a specially-crafted DoS attack.  By flooding a ton of RA messages onto the local network with different network address spaces, the attacker can cause the CPUs on Windows and FreeBSD boxes to spike to 100% and stay there indefinitely.  This is because the host system continually tries to process the RAs flooding in from the network and starts trying to pick address space in every network announcement that it hears.  This consumes all its resources with updating the routing table and addressing the adapters on the system.  This could cause problems for your end users should an attacker get into a position to launch RAs into your LAN.  Right now, there are a couple of ways to fix this:

1. Disable IPv6 – Okay, this doesn’t fix the problem it just makes it go away.

2.  Disable RAs on the local network – Again, not a fix, just hiding it.  Plus, this breaks SLAAC, which I see is a real advantage to IPv6.

3.  Install a firewall or ACL on your host-facing ports to block RAs or filter out the ICMPv6 packets carrying them.

What I find even more interesting about this whole affair is the response of the three biggest players in the game in regards to the issue.  Let me sum it all up using their words:

Microsoft – This is Working As Intended (WAI).  We don’t plan on fixing this.

Juniper – We need to work with the IETF and figure out a standard solution to address this problem, and until we do we aren’t patching against it.

Cisco – We fixed this last year, and by the way have you heard of RA Guard?

Cisco has implemented a solution very similar to what they do with DHCP snooping on IPv4 switches.  They call it RA Guard.  As defined in RFC 6105, RA Guard can be enabled on all host-facing switchports to filter RAs from non-trusted sources.  In this case, the trusted source would be a switchport you know to contain a valid router, so you wouldn’t enable RA guard on it.  The RFC defines a discovery method using Secure Neighbor Discovery (SeND) that made me chuckle because the four states of the discovery are the same as 802.1w Rapid Spanning Tree.  Seems we’re never going to get rid of it.  When you enable SeND-based RA Guard discovery, it can dynamically scan the network for devices broadcasting RAs and block or allow them as necessary.  That way, you don’t have to worry about misconfiguring a switchport and killing all the advertisements coming from it. By enabling RA guard on a Cisco switch with firmware 12.2(50)SY, you can effectively mitigate the possibility of an unauthorized attacker DoSing your entire network with what amounts to a script-kiddie style attack.

Tom’s Take

Take a vulnerability that has been known about for two years but swept under the rug, add in a dash of vendor disregard, and shake until the Internet security community is frothing at the mouth to tell you that you should turn off IPv6 on your entire network.  Sounds like a recipe for overreaction to me.  I’m not denying that it is a serious vulnerability.  In fact, given the fact that IPv6 is enabled by default on the current Windows version, it could cause issues.  That is, unless you are smart and take measures to fix the issue rather than sweeping it back under the rug.  Rather than just turning off IPv6 until someone other than Microsoft releases a patch, we need to work through the issue and fix the underlying security issue.  At the same time, this needs to be agreed upon by the major networking vendors sooner rather than later.  The longer this issue exists in its present form, the more security sensationalists can point to it and decry one of the advantages of IPv6 on your network when in fact they should be focusing on the lack of security in the software that allows anyone to masquerade as an IPv6 router.  Then, maybe we can cut IPv6 a little bit of slack.

Flexing Your Muscles – HP Networking Announcements

Posted on by
2

It’s Interop week, which means a lot of new product announcements coming out on all kinds of interesting hardware and software.  HP is no different and a couple of the products that they’ve got on tap appear to have some interesting designs on how we perceive the idea of a campus network for the coming months.

HP Flex Network Architecture

HP has announced a new network design methodology they refer to as the Flex Network Architecture.  It addresses many of the deficiencies that HP is seeing in network design related to thinking about networks in old ways.  There has been a lot of innovation in networking in recent months but it has been focused primarily on the datacenter.  This isn’t all that surprising, as most of the traction around building large networks has included buzzwords like “cloud” or “fabric” and tend to focus on centralizing the exchange of data.  HP wants to take a great number of these datacenter innovations and push them out to the campus and branch network, where most of us users live.

HP’s three primary areas of focus in the Flex Network are the Flex Fabric, which is the datacenter area that includes unified computing resources and storage, Flex Campus, which is the primary user-facing network area where users connect via technologies such as wireless, and the Flex Branch, where HP is focusing on creating a user experience very similar to that of the Campus users.  To do this, HP is virtually eliminating the lines between what have been historically referred to as “SMB” or “Enterprise” type networks.  I believe this distinction is fairly unnecessary in today’s market, as many branch networks could technically qualify as “Enterprise”, and a lot of campus networks could realistically be considered “SMB”.  By marrying the technology more toward the needs of the site and less to a label, HP can better meet the needs of the environment.

According to HP, there are five key components of the Flex Network:

1. Standards – In order to build a fully resilient architecture, standards are important.  By keeping the majority of your network interoperable with key standards, it allows you to innovate in pockets to improve things like wireless performance without causing major disruptions in other critical areas.

2. Scalability – HP wants to be sure that the solutions they offer are scalable to the height of their capability.  Mike Neilsen summed this up rather well by saying, “What we do, we do well.  What we don’t do well, we don’t do at all.”

3. Security – Security should be enabled all over your network.  It should not be an afterthought, it should be planned in at the very beginning and at every step of the project.  This should be the mantra of every security professional and company out there.

4. Agility – The problem with growing your network is that you lose the ability to be agile.  Network layers quickly overwhelm your ability to quickly make changes and decrease network latency.  HP wants to be sure that Flex allows you to collapse networks down to at most one or two layers to provide the ability to have them running in top condition.

5. Consistency – If you can’t repeat your success every time, you won’t have success.  By leveraging HP’s award winning managment tools like IMC, you can monitor and verify that your network experience is the same for all users.

The focus of the Flex Campus for this briefing is the new A10500-series switch.  This is a new A-series unit designed to go into the core of the campus network and provide high-speed switching for data bound for users.  This would most closely identify with a Cisco Catalyst 6500 switch.  The A10500 is the first switch in HP’s stable to provide IRF in up to 4 chassis.  For those not familiar, Intelligent Resilient Framework (IRF) is the HP method of providing Multiple Link Aggregation (MLAG) between core switches.  By linking multiple chassis together, one logical unit can be presented to the distribution layer to provide fault tolerance and load balancing.  HP’s current A-series switches currently support only 2 chassis in an IRF instance, but the 4IRF technology is planned to be ported to them in the near future.  One of the important areas where HP has done research on the campus core is the area of multimedia.  With the world becoming more video focused and consuming more and more bandwidth dedicated to things like HD Youtube videos and rich user-focused communications, bandwidth is being consumed like alcohol and a trade show.  HP has increased the performance of the A10500 above the Cat 6500 w/ Sup720 modules by reducing latency by 75%, while increasing switching capacity by almost 250%.  There is also a focus on aggregating as many connections as possible.  The launch LPUs (or line cards in Cisco parlance) are a 16-port 10GbE module and a 48-port 1GbE module, with plans to include a 4-port 40GbE and 48-port 10GbE module at some point next year, which should provide 270% more 10GbE density that the venerable Cat 6500. The A10500 comes in 3 flavors, a 4-slot chassis that uses a single crossbar fabric to provide better than 320 Gbps of throughput, and an 8-slot chassis that can mount the LPUs either vertically or horizontally and provide no less than 640 Gbps of throughput.  These throughput numbers are courtesy of the new MPU modules, what Cisco people might call Supervisor engines.  The n+1 fabric modules that tie all the parts and pieces together are called SFMs.  This switch isn’t really designed to go into a datacenter, so there is no current plan to provide FCoE LPUs, but there is strong consideration to support TRILL and SPB in the future to ease the ability to interoperate with datacenter devices.

Another new product that HP is launching is focused on security in the datacenter.  This comes courtesy of the new TippingPoint S6100N.  This is designed to function similarly to an IDS/IPS box, inspecting traffic flying in and out of your datacenter swtiches.  It has the ability to pump up to 16 Gbps of data through the box at once, but once you start inspecting more and more of that traffic, you’ll probably see something closer to 8-10Gbps of throughput.  The S6100N also gives you the ability to have full visibility for VM-to-VM conversations, something that is currently giving many networking and security professionals grief, as much of the traffic being generated in today’s datacenter is generated and destined for virtual machines.  I think there is going to be a real opportunity in the coming months for a device that can provide this kind of visibility without impeding traffic.  HP looks to have a great future with this kind of device.

The third new piece of  the Flex Network Architecture is the addition of Intelligent Management Center (IMC) 5.0 to the Flex Management portion of the Flex Architecture.  The flagship software program for HP’s network management strategy, IMC provide Single Pane of Glass (SPoG) functionality to manage both wired and wireless networks, as well as access control and identity management for both.  It integrates with HP Network Management Center to allow total visibility into your network infrastructure, whether it consist of HP, Procurve, Cisco, 3Com, or Juniper.  There are over 2600 supported devices in IMC that can be monitored and controlled.  In addition, you can use the integrated access controls to control the software being run on the end user workstations and monitor their bandwidth utilization to determine if you’ve got a disproportionately low number of users monopolizing your bandwidth.  IMC is available for installation on a dedicated hardware appliance or in a virtual machine for those that have an invested VMware infrastructure.  You can try it out all the features for 60 days at no charge to see if it fits into your environment and helps alleviate “swivel chair” management.

Tom’s Take

The new HP Flex Architecture gives HP a great hook to provide many new services under a consistent umbrella.  The new addition of the A10500 gives HP a direct competitor to the venerable Cat 6500 platform that can provide high speed connectivity to the campus core without muddying the waters with unnecessary connectivity options, ala the A12500 or Nexus 7000 with their high-priced FCoE capabilities.  The S6100N helps HP begin to focus on the new landscape of datacenter security, where firewalls are less important the visibility into traffic flows between physical and virtual servers.  The IMC update allows all of these pieces to be managed easily from one operations center with no additional effort.  It seems that HP is gearing up to spread out from their recent success in the datacenter and take the fight to the campus and branch office.  I liked what I heard from HP on this call, as it was more of what HP could do and less of what Cisco couldn’t.  So long as HP can continue to bring new and innovative products to the networking marketplace, I think their fortunes have nowhere to go but up.

I Hate NAT! Or Do I…?

Posted on by
15

Thanks to Packet Pushers episode 43, it appears that I’m going to be known as Tom the Networking Nerd, the guy that hates NAT.  I suppose that’s as good as anything to be known for, and the trick to being a famous celebrity blogger is getting yourself a hook, like Apple Fanboy or General Grumpy Networking Guy.  I know that I made some pretty bold statements during the show, and while some of it may have been playing into a image, I’d like to take a few words to clarify my position on why I dislike NAT.

Network Address Translation goes all the way back to RFC 2663.  At its heart, it is simply a mechanism for modifying IP packets and the information in their headers when the cross a routed boundary.  As I said in my IPv6 presentation, I do really believe that basic NAT serves a purpose in a network.  As a protocol, NAT isn’t inherently bad or good.  Instead, it is the application of the protocol by people that colors it in my eyes.  Allow me to give an example that I feel is very similar to NAT: Bittorrent.  I think the Bittorrent is a very useful protocol for file downloading.  By using the number of users accessing a file to amplify the speed at which it is downloaded, it helps alleviate the effect of having thousands of people downloading a file at once.  It also has built-in error correction, as each file piece is hashed upon receipt.  It’s very useful for downloading large, error-prone files like Linux ISOs or even World of Warcraft patches.  However, Bittorrent has an evil side.  Since it is so useful for downloading large files or collections of files, it has been appropriated by the underground file sharing movement as a method for distributing movies, games, and applications less-than-legally.  The opponents of this activity say that Bittorrent encourages people to break the law, much in the same was the the motion picture industry argued that VCRs encouraged movie piracy back in the 1980’s.  The counter argument here is that Bittorrent isn’t good or bad, instead the intentions of the user must be taken into account.  I feel the same way about NAT.

NAT breaks things.  End-to-end communications between nodes are disrupted by NAT boundaries.  VoIP packets die when they hit NAT walls.  VPNs don’t particularly care for it either.  We’ve had to come up with things like STUN and ICE to workaround a workaround.  It’s getting embarrassing when you have to fix the fix that you put in place.  That’s not to say that NAT doesn’t have its good points, however.

NAT is great when two companies need to merge and have the same address space be in use during the transition.  NAT is a good idea if you need to obscure IP addresses for some reason.  Note that I didn’t say anything about securing them.  If you hear a security “expert” claim that NAT provides security, you have my explicit permission to beat them within an inch of their life.  Beyond a few cases, I think NAT has been used as a kind of “Internet duct tape” and has been stretched long past its original use case into something dastardly.  I realize that without RFC 1918 and Port Address Translation (PAT), we would have run out of IPv4 addresses long ago.  NAT has become a necessary tool in the IPv4 world, and I accept that.  However, the paradigm of NAT is started to be extended into IPv6 unecessarily.  That’s the part that I don’t agree with.

People are looking at IPv6 and wondering how to apply their IPv4 knowledge to it.  That in and of itself is nothing new.  When I was first learning how to do VoIP stuff, I used my networking background to assimilate new terms.  I used to say things like, “Oh! A calling search space is just like an access list.”  While it was great for me when I was just learning, I found out later than metaphor wasn’t quite accurate.  In much the same way, network engi…rock stars are starting to plan IPv6 deployments and they are looking for analogies for the IPv4 things they are used to.  When they don’t see things like NAT or RFC 1918, it makes IPv6 feel alien to them.  For proof of this, you need not look any further than RFC 1884 – site local addresses.  This was an idea to reserve fec0::/10 for use only within a specific site.  While link-local addressing has many uses, the idea of what amounts to RFC 1918 addresses for IPv6 is kind of silly.  Why not just use your IPv6 global addresses instead?  RFC 3879 thankfully removed site local addressing from our lexicon for the time being, until it was reintroduced again in RFC 4193 with a slighly different name.  The same thing happened (in my opinion) with NAT-PT.  People were looking for something they saw in IPv4 in IPv6 unnecessarily.

I feel that NAT in IPv6 will encourage laziness.  I know there is no such thing as infinite address space, but we’ve got enough address space right now to outlast this Internet and whatever the next revision is going to look like.  If we allow NAT64 to take hold in the networking world, we are giving our blessing to people to keep doing the same things they’ve been doing with IPv4 all over again.  It’s the networking equivalent of the famous Santayana quote, “Those who forget the past are doomed to repeat it.”  If we allow people to implement IPv6-as-IPv4, we aren’t going to do ourselves any favors.  Many of the non-address space related reasons to go to IPv6 evaporate when NAT is introduced.  We eliminate wonderful things like end-to-end communications solely for the sake of ease-of-configuration.  Bah, I say.

I’ve always been a proponent of building it right the first time.  Even if it takes a little long to learn how to do it, as long as the implementation is done correctly at the end everything will work out.  As an integrator, a large portion of my time is spent cleaning up after other people’s mistakes.  All too often in IT, we patch a problem and declare it “good enough” only to spend twice as long down the road fixing what we should have fixed the last time.  If we start using NAT unnecessarily in IPv6, it will come back to bite us down the road.  People will avoid using IPv6 internally because “it’s too much trouble”, and just use NAT64 to talk to the IPv6 Internet.  A lot of good work will have been for naught due to the laziness of some.

There are already a lot of v4-to-v6 transition mechanisms out there.  F5 is looking at using load balancers to do 4-to-6 translation.  Proxy servers can help.  Tunneling can be had in any flavor you want, even the much-hated Teredo protocol.  Incidentally, Teredo was developed because Microsoft needed a specification to get around…NAT boundaries.  If we absolutely need to do something other than the ideal of running dual stack, we can use some of the other mechanisms that have been devised that will allow us to transition to where we need to be rather than coming back to our old NAT kludge.

Tom’s Take

I once was able to work on a network that didn’t have NAT.  Globally routable IP addresses as far as the eye could see.  And yet, due to a voice implementation that didn’t have enough address space, NAT still came back to bite me in the ass over and over again.  You can see why that might make me grumpy.

In the end, I really don’t hate NAT as much as I might let on.  Yes, it’s a necessary evil in IPv4.  I don’t want people to keep using it as a crutch instead of designing proper networks with good security policies.  I want people to start thinking about IPv6 as a great new opportunity instead of thinking about it in the same old way.  If designers had kept thinking about phones in the same old way, we would never have gotten the iPhone or the Nexus S.  If computer designers had thought about portable PCs in the same old way, we wouldn’t have iPads or Galaxy Tabs or Chrome CR-48s.  I want to encourage people to solve problems with the new tools they have been given rather than falling back on broken ideas from the 90s.  I may not be as opposed to NAT64 if it had an expiration date.  Use it if you want, but be sure to have it out of your network no later than December 21, 2012 or the world will end.  I suppose I could live with that.  But it doesn’t mean I have to like NAT.

Nerd Tips – Broken Execution Association

Posted on by
2

Here’s a quick tip for those of you out there that might find yourself fighting off an offending virus or malware program that keeps coming back no matter what you try, such as Win 7 Antivirus 2011.  This particular program does have a little trick that it likes to pull in order to keep itself in memory.  When an executable file (EXE) is launched in the system, usually a set of keys in the registry are consulted to find out what to do with the file.  Most often, the file itself is run with a command string like “%1”, which calls the file.  The malware program inserts itself in front of the execution string, so that every time you try to launch a program to fight off the crapware, like Malwarebyte AntiMalware for instance, the virus just launches instead and reinfects your system.

Should you find yourself in this quandry, unable to launch the programs needed to disinfect yourself, take heart.  An old DOS trick can be used to get yourself right as rain.  In the old days, executable files came in a format other that EXE.  DOS used a file format of COM to execute simple little programs like COMMAND.COM or DOSSHELL.COM.  COM files were orginally simple, with very little code and no metadata in the header.  Likewise, when Windows 3.x was just a program executing on top of DOS, it preserved the executable format of the COM programs.  Fast forward to Windows 7, and you will see that this convention is still honored.  If you find yourself unable to launch REGEDIT.EXE or MBAM.EXE and instead keep launching the virus, do the following:

1.  Launch a command prompt (CMD.EXE or COMMAND.COM if necessary).  You might have to launch it as an administrator to make some changes to system files.

2.  Find the file you need to execute, like REGEDIT.EXE.

3.  Use the follwing command to rename the file: ren REGEDIT.EXE REGEDIT.COM


Sounds simple, eh?  You’ll find that when the file is displayed, it won’t have the neat icon it used to.  Instead, it will look like a generic DOS executable file.  That’s perfectly fine.  When you double-click the file to launch it, it will fire right up.   This is because the COM file association as an executable file format is usually not changed by the malware writers, since very few COM files are still used on modern systems.  Following these steps, you can get Malwarebytes to load and disinfect your system, bypassing the EXE file lockout.  Malwarebytes will even repair the EXE association for you, so when you reboot you’ll be back to normal.  Just remember to go back and rename the file you change to a COM file back to an EXE file.

As a disclaimer, this process doesn’t work 100% of the time, and if the malware writer was smart enough to screw up the COM file association, you’re doubly screwed.  Don’t go mucking around in your system registry changing things unless you know what your doing, since a screwed registry will really kill your system fast.  Use caution, logic, and if all else fails, find a systems rock star to help you out.

Note, I reference Malwarebytes as a removal tool not because of any consideration on their part, but instead because it just works.  I’ve installed the trial on many computers for people that tend to get infected over and over, and it really helps cut down on their infection rates.  Try it out, and don’t forget to buy it if you find it useful.  Every penny they get goes to help cut down on the amount of crap out there trying to infect your system over and over again.

Tech Movies

Posted on by
3

On occasion, a trending hashtag pops up on Twitter that I think is quite funny.  Today, it was #TechMovies.  However, I know that most people that follow me probably don’t care to listen to my inane ramblings about these kinds of things.  I’m pretty sure that if I gave it my all, I’d have about 2 followers by the end of the day.  In an effort to spare my followers the horrors that swim around in my mind, I decided instead to post my list to my blog.  Read ahead at your own peril…

Still here?  Okay, here come the #TechMovies:

A Root Bridge Too Far

A View to a TRILL

On Her Majesty’s Shared Secret Service

Event Split Horizon

SDLC Punk!

Friday the 0xd

Licesne to Kill -9

Fort httpd

The ’emerge world’ according to ARP (two for one)

BGP Neighbors

The Matrix “shutdown -r now”

DNS the Menace

The OS X 10.7 Lion King

SLAXers

Hard Route Target

Harry Potter and the Chamber of Enable Secrets

A League of Their PWN

Chitty Chitty ! !

DOS Bootdisk

Universal Image Soldier

Major Version League

ntop Gun

10 Fast 10 Furious

What About Microsoft Bob?

Visual BASIC Instinct

The EIGRP Sanction

Manos: Hands of Fate Sharing

Under Siege 2: Dark Fiber Territory

IPv6 for Enterprise Networks – Review

Posted on by
1

Unless you’ve been living in a cave with Tony Stark for the past several months, you are well aware that IPv6 is a reality that can’t be ignored by today’s networker.  Part of the issue affecting IPv6 adoption is the lack of good reading material.  Yes, there are mountains of RFCs out there that talk about IPv6 in nauseating detail.  However, these documents aren’t all that accessible for the average network rock star that is working 50 hours a week and doesn’t have time to pour over page after page of dry Internet-ese.  There have been some great posts about IPv6 for the common man from people like Jeremy Stretch and Chris Jones but there is a segment of the population that would rather read about the subject from a vendor source.  Enter Shannon McFarland and company:

IPv6 for Enterprise Networks Cover

Cisco Press graciously provided a copy of this book for my evaluation.  Clocking in at a svelte 361 pages, this tome has a great wealth of IPv6 information from a design perspective.  There are some code examples for your networking gear, but much of the discussion in this book revolves around IPv6 design and building your network right the first time.

Chapter 1 starts off the same way many network rock stars will start off pitching IPv6 to their company, with a discussion of the market drivers for IPv6 adoption.  Even though networking professionals know IPv6 is inevitable, the C-level executives will most likely need some additional convincing.  This chapter is great for them to hear about the reasons why IPv6 is necessary.  Chapter 2 is an overview of the Cisco hierarchical network design, now expanded to include IPv6 content.  If you’ve seen any network design documentation in the past decade, this should be a review for you.  Just note the IPv6 sections.

Chapter 3 starts the meat of the book.  This chapter discusses the coexistence mechanisms that you are likely to face when prepping your network, since we are going to need to run IPv4 alongside IPv6 no matter how much we might not want to.  Tunnels and NAT64 get some discussion, along with running IPv6 over MPLS.  Chapter 4 discusses the various network services that will need to be IPv6 aware to help run your network, such as OSPFv3 or BGP.  Great discussion is made about multicast, since multicast is such a crucial component in IPv6.  Chapter 5 is a short one, discussing the planning that one will need to go through for implementing an IPv6 infrastructure.  This is more of the paperwork and staging behind the scenes that might be boring, but in an enterprise is critical for painless IPv6 deployment.

Chapter 6 is the largest chapter and will most likely be where you spend most of your time.  This is a soup-to-nuts campus IPv6 deployment.  The authors analyze the deployment from three different perspectives, the dual stack (my favorite), the hybrid model that is useful for non-IPv6 applications, and the service block model, which allows you to bring IPv6 online in sections.  Every facet of your network is analyzed in this chapter, from VLANs to routing protocols to QoS and other network services.  If you are going to be deploying IPv6 in your network in the future, you’d do well to just dog ear Chapter 6 so you can turn back to it quickly.

Chapters 7 through 10 deal with specific cases of IPv6 deployment to support use-cases, such as virtualization, branch offices, datacenters, or remote access.  They exist so that you can quickly reference these scenarios as needed, since you may not need to worry about deployment of IPv6 in a datacenter in your environment for instance.  The authors do a wonderful job of explaining all the things you might need to take into account in your deployment of ancillary technologies, such as Microsoft protocols to be aware of or application requirements that may not necessarily be network dependent.

Chapter 11 is all about managing your shiny new IPv6 network through things like Netflow and SNMP.  Careful attention should be paid if you don’t want to find yourself chasing poltergeists in your network at 3 a.m. on a Sunday.  Chapter 12 gives you a great breakdown of parts and pieces that would be great to construct a lab to pilot your IPv6 implementation before unleashing it on your live network.  That way, IPv6 doesn’t call the Resume Generating Event (RGE) protcol.

Tom’s Take

I liked this book quite a bit.  There is a ton of good information to be found inside for all levels of network rock star, from those just learning about deploying IPv6 to the poor souls that find themselves mired in a remote access IPv6 deployment gone wrong.  With a big focus on proper network design, IPv6 for Enterprise Networks ensures that you don’t have to rebuild your IPv6-enabled network after a short time due to bad design decisions or compromises. Every scenario I’ve seen discussed concerning IPv6 deployment is laid out in clear language, with both pros and cons for deployment.  I highly recommend picking up this book as soon as you can so your journey down the IPv6 yellow brick road starts off smoothly and you can avoid the pitfalls before you encounter them.

As a bonus, if you are going to Cisco Live 2011 in Las Vegas, Shannon McFarland is giving a session based around this book, BRKRST-2301 Enterprise IPv6 deployment.  If you aren’t adverse to 8 a.m. sessions the morning after the Customer Appreciation Event you should sign up and check it out.  I plan on bringing my book so that Shannon can autograph it.  That way, I can claim I met him before he became a gigantic IPv6 rock star.