Author Archives: networkingnerd

About networkingnerd

Tom Hollingsworth, CCIE #29213, is a network engineer that works with Cisco, HP, Microsoft, VMware, and various other technologies. Tom has been in the IT industry since 2002, and has been a nerd since he first drew breath.

Juniper Networks Warrior – Review

Posted on by
1

Documentation is the driest form of communication there is. Whether it be router release notes or stereo instructions I never seem to be able to find a way to read more than a paragraph before tossing things aside. You’d think by now that someone would come up with a better way to educate without driving someone to drinking.

O’Reilly Media has always done a good job of creating technical content that didn’t make me pass out from boredom. They’ve figured out how to strike a balance between what needs to be said and the more effective and entertaining way to say it. Once I started reading the books with the funny animals on the covers I started learning a lot more about the things I was working on. One book in particular caught my eye – Network Warrior by Gary Donahue. Billed as “everything you need to know that wasn’t on the CCNA,” it is a great introduction to more advanced topics that are encountered in day-to-day network operations like spanning tree or the Catalyst series of switches. Network Warrior is heavily influenced by Cisco equipment. While the concepts are pretty straight forward the bias does lean toward the building on Tasman Drive. Thankfully, O’Reilly enlisted an author to bring the Warrior series to Sunnyvale as well:

Screen Shot 2013-05-13 at 2.53.13 PM

Peter Southwick was enlisted to write a Warrior book from the perspective of Juniper engineer. I picked up a copy of this book the last time I was at Juniper’s headquarters and have spent the past few weeks digesting the info inside.

What Worked

Documentation is boring. It’s a dry description of how to do everything. How-to guides are a bit better written, but they still have to cover the basics. I am a much bigger fan of the cookbook, which is a how-to that takes basic building blocks and turns them into a recipe that accomplishes something. That’s what Juniper Networks Warrior is really about. It’s a cookbook with some context. Each of the vignettes tells a story about a specific deployment or project. By providing a back story to everything you get a feel for how real implementations tend to flow back and forth between planning and execution. Also, the solutions provided really do a great job of cutting past the boring rote documentation and into things you’ll use more than once. Couple that with the vignettes being based on something other than technology-focused chapters and it becomes apparent that this is a very holistic view for technology implementation.

What Didn’t Work

There were a couple of things that didn’t work well in the narrative to me. The first was the “tribe” theme. Southwick continually refers to the teams that he worked with in his projects as “tribes.” While I understand that this does fit somewhat with the whole idea behind the Warrior books, it felt a bit out of place. Especially since Donahue didn’t use it in either Network Warrior or Arista Warrior (another entry in the series). I really did try to look past it and not imagine groups of network engineers carrying spears and slings around the data center, but it was mentioned so often in place of “team” or “group” that it became jarring after a while.

The other piece that bothered me a bit was in Chapter 3: Data Center Security Design. The author went out of the way to mention that the solution that his “tribe” came up with was in direct competition with a competing one that utilized Cisco gear. He also mentioned that the Juniper solution was going to displace the Cisco solution to a certain degree. I get that. Vendor displacement happens all the time in the VAR world. What bothered me was the few occasional mentions of a competitor’s gear with words like “forced” or casting something in a negative light simply due to the sticker on the front. I’ve covered that before in my negative marketing post. Why I bring it up here is because it wasn’t present in either Network or Arista Warrior, even though the latter is a vendor-sponsored manual like this one. In particular, an anecdote in the Arista chapter on VRRP mentions that Cisco wanted to shut down the RFC for VRRP due to similarity with HSRP. No negativity, no poking with a sharp stick. Just a statement of fact and the readers are left to draw their own conclusions.

I realize the books of this nature often require input from the technical resources of a vendor. I also realize that sometimes the regard that these books are held in sometimes looks to be a very appealing platform to launch marketing campaigns or to use a factually based volume to mention some opinion-based verbiage. I sincerely hope that future volumes tone down the rhetoric just a bit for the sake of providing a good reference volume. Engineers will keep going back to a book if it gives them a healthy dose of the information they need to do their jobs. They won’t go back nearly as often to a book that spends too much time discussing the pros and cons of a particular vendor’s solution. I’d rather see pages of facts and configs that get the job done.

Review Disclaimer

The copy of Juniper Networks Warrior that I reviewed was provided to me by Juniper Networks. I received it as part of a group of items during Network Field Day 5. At no time did Juniper ask for nor were they promised any consideration in the writing of this review. All of the analysis and conclusions contained herein are mine and mine alone.

Why Facebook’s Open Compute Switches Don’t Matter. To You.

Posted on by
5

Facebook_iconFacebook announced at Interop that they are soliciting ideas for building their own top-of-rack (ToR) switch via the Open Compute Project.  This sent the tech media into a frenzy.  People are talking about the end of the Cisco monopoly on switches.  Others claimed that the world would be a much different place now that switches are going to be built by non-vendors and open sourced to everyone.  I yawned and went back to my lunch.  Why?

BYO Networking Gear

As you browse the article that you’re reading about how Facebook is going to destroy the networking industry, do me a favor and take note of what kind of computer you’re using.  Is is a home-built desktop?  Is it something ordered from a vendor?  Is it a laptop or mobile device that you built? Or bought?

The idea that Facebook is building switches isn’t far fetched to me.  They’ve been doing their own servers for a while.  That’s because their environment looks wholly different than any other enterprise on the planet, with the exception of maybe Google (who also builds their own stuff).  Facebook has some very specialized needs when it comes to servers and to networking.  As they mention at conferences, the amount of data rushing into their network on an hourly, let alone daily, basis is mind boggling.  Shaving milliseconds off query times or reducing traffic by a few KB per flow translates into massive savings when you consider the scale they are operating at.

To that end, anything they can do to optimize their equipment to meet their needs is going to be a big deal.   They’ve got a significant motivation to ensure that the devices doing the heavy lifting for them are doing the best job they can.  That means they can invest a significant amount of capital into building their own network devices and still get a good return on the investment.  Much like the last time I built my own home desktop.  I didn’t find a single machine that met all of my needs and desires.  So I decided to cannibalize some parts out of an old machine and just build the rest myself.  Sure, it took me about a month to buy all the parts, ship them to my house, and then assemble the whole package together.  But in the end I was very happy with the design.  In fact, I still use it at home today.

That’s not to say that my design is the best for everyone, or anyone for that matter.  The decisions I made in building my own computer were one’s that suited me.  In much the same way, Facebook’s ToR switches probably serve very different needs than existing data centers.  Are your ToR switches optimized for east-west traffic flow?  I don’t see a lot of data at Facebook directed to other internal devices.  I think Facebook is really pushing their systems for north-south flow.  Data requests coming in from users and going back out to them are more in line with what they’re doing.  If that’s the case, Facebook will have a switch optimized for really fast data flows.  Only they’ll be flowing in the wrong direction for what most people are using data flows for today.  It’s like having a Bugatti Veyron and living in a city with dirt roads.

Facebook admitted that there are things about networking vendors they don’t like.  They don’t want to be locked into a proprietary OS like IOS, EOS, or Junos.  They want a whitebox solution that will run any OS on the planet efficiently.  I think that’s because they don’t want to get locked into a specific hardware supplier either.  They want to buy what’s cheapest at the time and build large portions of their network rapidly as needed to embrace new technology and data flows.  You can’t get married to a single supplier in that case.  If you do, a hiccup in the production line or a delay could cost you thousands, if not millions.  Just look at how Apple ensures diversity in the iPhone supply chain to get an idea of what Facebook is trying to do.  If Apple were to lose a single part supplier there would be chaos in the supply chain.  In order to ensure that everything works like a well-oiled machine, they have multiple companies supplying each outsourced part.  I think Facebook is driving for something simliar in their switch design.

One Throat To Choke

The other thing that gives me pause here is support.  I’ve long held that one of the reasons why people still buy computers from vendors or run Windows and OS X on machines is because they don’t want the headache of fixing things.  A warranty or support contract is a very reassuring thing.  Knowing that you can pick up the phone and call someone to get a new power supply or tell you why you’re getting a MAC flap error lets you go to sleep at night.  When you roll your own devices, the buck stops with you when you need to support something.  Can’t figure out how to get your web server running on Ubuntu?  Better head to the support forums.  Wondering why your BYOSwitch is dropping frames under load?  Hope you’re a Wireshark wizard.  Most enterprises don’t care that a support contract costs them money.  They want the assurance that things are going to get fixed when they break.  When you develop everything yourself, you are putting a tremendous amount of faith into those developers to ensure that bugs are worked out and hardware failures are taken care of.  Again, when you consider the scale of what Facebook is doing, the idea of having purpose-build devices makes sense.  It also makes sense that having people on staff that can fix those specialized devices is cost effective for you.

Face it.  The idea that Facebook is going to destroy the switching market is ludicrous.  You’re never going to buy a switch from Facebook.  Maybe you want to tinker around with Intel’s DPDK with a lab switch so you can install OpenFlow or something similar.  But when it comes time to forklift the data center or populate a new campus building with switches, I can almost guarantee that you’re going to pick up the phone and call Cisco, Arista, Juniper, Brocade, or HP.  Why?  Because they can build those switches faster than you can.  Because even though they are a big captial expenditure (capex), it’s still cheaper in the long run if you don’t have the time to dedicate to building your own stuff.  And when something blows up (and something always blows up), you’re going to want a TAC engineer on the phone sharing the heat with you when the CxOs come headhunting in the data center when everything goes down.

Facebook will go on doing their thing their way with their own servers and switches.  They’ll do amazing things with data that you never dreamed possible.  But just like buying a Sherman tank for city driving, their solution isn’t going to work for most people.  Because it’s built by them for them.  Just like Google’s server farms and search appliances.  Facebook may end up contributing a lot to the Open Compute Project and advancing the overall knowledge and independence of networking hardware.  But to think they’re starting a revolution in networking is about as far fetched as thinking that Myspace was going to be the top social network forever.

Coffee As A Service

Posted on by
6

tikigiki_misc-coffee-cup-010I have a hard time keeping all the cloud terms straight.  Everything seems to be available As A Service (aaS).  Try as I might to explain them, it just didn’t click for some people.  Since cloud terms are so nebulous some times, I decided I need to put everything in a context that people understand.  Therefore, I present…Coffee as a Service (CaaS):

 

Software as a Service (SaaS): Anytime you want coffee, it just appears in front of you.  You don’t have to make it or anything.  Sometimes it shows up immediately.  Other times it shows up an hour or so after you wanted it.  You pay a monthly fee, but if you want to have cream or sugar you have to pay a bit more.  Some SaaS coffee vendors don’t have those options, so you’re stuck with whatever you get up front.

Platform as a Service (PaaS): You’ve decided that you want to have coffee, but you want a bit more control over it.  You sign up for a new service that gives you coffee packages.  Dark roast, light roast, and Turkish coffee are all options.  You are still at the mercy of the provider for other options like latte or cappuccino.  It is still mysteriously delivered to you.  Cream and sugar are options in each of the packages for a small fee.  Coffee can still show up late, but you have an agreement with the provider that late coffee gets you a small amount off the monthly bill.

Infrastructure as a service (Iaas): You’ve now decided that you want complete control over your coffee delivery.  You’ve contacted a new provider that is willing to rent you a coffee machine with all the extra hardware needed to make any kind of coffee you want.  You’re going to have to buy your own coffee and creamer and sugar.  Once you have it at the coffee machine, they’ll make the coffee to your exact specifications and send it to you.  Might still show up late depending on how popular the service is or if some technician accidentally restarts the machine on a Friday night.  You get charged either by the cup or by how often the machine is used.  Coffee still tastes okay but you have to worry about renting more machines as it becomes more popular.  Machine rental rates fluctuate if you use the Spot Machine market.

Private Cloud: Okay, forget about the whole renting thing.  Time to go to the coffee warehouse and buy everything yourself.  You max out your credit card, but you come home with a coffee machine and a milk steamer.  You still provide everything yourself.  You find a place to hook everything up with electricity and water supply.  You grind the beans yourself.  You make your own coffee or you hire a barista to do it for you.  The coffee is excellent and on time.  Your credit card bill scares the daylights out of you.  In three years, you have to upgrade your coffee machine again to support hyper foaming milk.

Hybrid Cloud: You can make the basics with your machine.  You still can’t figure out how to make a good cappuccino.  All the easy stuff gets made locally by you or your barista.  For the really odd stuff, like double shot mocha light frappuccinos you send people to the Starbucks down the road.

Cloudbursting: Your fancy coffee machine is really popular around the office.  About once a month, there’s a line that’s 50 people long.  Rather than making them wait for their coffee, you pass out Starbucks gift cards for anyone over the 35th person.  You send them off to get their coffee.  You can justify the gift card cost because you’re only busy that one time a month.

I’m always open for suggestions in the comments below.

Cisco ASA CX 9.1 Update

Posted on by
5

Cisco LogoEvery day I seem to get three or four searches looking for my ASA CX post even though it was written over a year ago.  I think that’s due in part to the large amount of interest in next-generation firewalls and also in the lack of information that Cisco has put out there about the ASA CX in general.  Sure, there’s a lot of marketing.  When you try to dig down into the tech side of things though, you find yourself quickly running out of release notes and whitepapers to read.  I wanted to write a bit about the things that have changed in the last year that might shed some light on the positioning of the ASA CX now that it has had time in the market.

First and foremost, the classic ASA as you know it is gone.  Cisco made the End of Sale announcement back in March.  After September 16, 2013 you won’t be able to buy one any longer.  Considering the age of the platform this isn’t necessarily a bad thing.  Firstly, the software that’s been released since version 8.3 has required more RAM than the platform initially shipped with.  That makes keeping up with the latest patches difficult.  Also, there was a change in the way that NAT is handled around the 8.3/8.4 timeframe.  That lead to some heartache from people that were just getting used to the way that it worked prior to that code release.  Even though it behaves more like IOS now (i.e. the right way), it’s still confusing to a lot of people.  When you’ve got an underpowered platform that requires expensive upgrades to function at a baseline level, it’s time to start looking at replacing it.  Cisco has already had the replacement available for a while in the ASA-X line, but there hasn’t been a compelling reason to cause customers to upgrade there existing boxes.  The End of Sale/End of Life notice is the first step in migrating the existing user base to the ASA-X line.

The second reason the ASA-X line is looking more attractive to people today is the inclusion of ASA CX functionality in the entire ASA-X line.  If you recall from my previous post, the only ASA capable of running the CX module was the 5585.  It had the spare processing power needed to work the kinks out of the system during the initial trial runs.  Now that the ASA CX software is up to version 9.1, you can install it on any ASA-X appliance.  As always, there is a bit of a catch.  While the release notes tell you that the ASA CX for the mid-range (non 5585) platforms is software based, please note that you need to have a secondary solid state disk (SSD) drive installed in the chassis in order to even download the software.  If you are running ASA OS 9.1 and try to pull down the ASA CX software, you’re going to get an error about a missing storage device.  Even if you purchased the software licensing for the ASA CX, you won’t get very far without some hardware.  The part you’re looking for is ASA5500X-SSD120=, which is a spare 120GB SSD that you can install in the ASA chassis.  If you don’t already have an ASA-X and want the ASA CX functionality, you’re much better off ordering one of the bundle part numbers.  That’s because it includes the SSD in the chassis preloaded with a copy of the ASA CX software.  Save yourself some effort and just order the bundle.

Another thing that I found curious about the 9.1 release of the ASA CX software was in the release notes.  As previously mentioned, the UI for the ASA CX is a copy of Cisco Prime Security Manager (PRSM), also pronounced “prism.”  At first, I just thought this meant that Cisco had borrowed concepts from PRSM to make the ASA CX UI a bit more familiar to people.  Then I read the 9.1 release notes.  Those notes are combined for the ASA CX and PRSM 9.1.  You’d almost never know it though, outside of a couple of mentions for the ASA CX.  Almost the entire document references PRSM, which makes sense when you think about it.  That really did clear up a lot of the questions I had about the ASA CX functionality.  I wondered what kind of strange parallel development track Cisco had used to come up with their answer in the next generation firewall space.  I was also worried that they had either borrowed or licensed software from a third part and that their effort would end up as doomed as the ASA UTM module that died a painful death thanks to Trend Micro‘s strange licensing.

ASA CX isn’t really a special kit.  It’s an on-box copy of PRSM.  The ASA is configured with a rule to punt packets to PRSM for inspection before being shunted back for forwarding.  No magic.  No special sauce.  Just placing one product inside another.  When you think about how IDS/IPS has worked in the ASA for the past several years I suppose it shouldn’t come as too big of a shock.  While vendors like Palo Alto and Sonicwall have rewritten their core OS to take advantage of fast next generation processing, Cisco is still going back to their tried-and-true method of passing all that traffic to a module.  In this case, I’m not even sure what that “module” is in the midrange devices, as it just appears to be an SSD for storing the software and not actually doing any of the processing.  That means that the ASA CX is likely a separate context on the ASA-X.  All the processing for both packet forwarding and next generation inspection is done by the firewall processor.  I know that that the ASA-X has much more in the processing department than its predecessor, but I wonder how much traffic those boxes are going to be able to take before they give out?

Tom’s Take

Cisco is playing catch up in the next generation market.  Yes, I understand that the term didn’t even really exist until Palo Alto started using it to differentiate their offering.  Still, when you look at vendors like Sonicwall, Fortinet, and even Watchguard, you see that they are embracing the idea of expanding unifed threat management (UTM) into a specific focus designed to let IT people root out traffic that’s doing something it’s not supposed to be.  Cisco needs to take a long hard look at the ASA-X platform.  If it is selling well enough against units like the Juniper SRX and the various Checkpoint boxes then the next generation piece needs to be spun out into a different offering.  If the ASA-X is losing ground, what harm could there be in pushing the reset button and turning the whole box into something a bit more grand that a high speed packet filter?  The ASA CX is a great first step.  But given the lack of publicity and difficulty in finding information about it, I think Cisco is in danger of stumbling before the race is even going.

Brocade’s Pragmatically Defined Network

Posted on by
1

logo-brocadeMost of the readers of my blog would agree that there is a lot of discussion in the networking world today about software defined networking (SDN) and the various parts and pieces that make up that umbrella term.  There’s argument over what SDN really is, from programmability to orchestration to network function virtualization (NFV).  Vendors are doing their part to take advantage of some, all, or in some cases none of the above to push a particular buzzword strategy to customers.  I like to make sure that everything is as clear as possible before I start discussing the pros and cons.  That’s why I jumped at the chance to get a briefing from Brocade around their new software and hardware releases that were announced on April 30th.

I spoke with Kelly Harrell, Brocade’s new vice president and general manager of the Software Business Unit.  If that name sounds somewhat familiar, it might be because Mr. Harrell was formerly at Vyatta, the software router company that was acquired by Brocade last year.  We walked through a presentation and discussion of the direction that Brocade is taking their software defined networking portfolio.  According to Brocade, the key is to be pragmatic about the new network.  New technologies and methodologies need to be introduced while at the same time keeping in mind that those ideas must be implemented somehow.  I think that a large amount of the frustration with SDN today comes from a lot of vaporware presentations and pie-in-the-sky ideas that aren’t slated to come to fruition for months.  Instead, Brocade talked to me about real products and use cases that should be shipping very soon, if not already.

The key to Brocade is to balance SDN against network function virtualization, something I referred to a bit in my Network Field Day 5 post about Brocade.  Back then, I called NFV “Networking Done (by) Software,” which was my sad attempt to point out how NFV is just the opposite of what I see SDN becoming.  During our discussion, Harrell pointed out that NFV and SDN aren’t totally dissimilar after all.  Both are designed to increase the agility with which a company can execute on strategy and create value for shareholders.  SDN is primarily focused on programmability and orchestration.  NFV is tied more toward lowering costs by implementing existing technology in a flexible way.

NFV seeks to take existing appliances that have been doing tasks, such as load balancers or routers, and free their workloads from being tied to a specific piece of hardware.  In fact, there has been an explosion of these types of migrations from a variety of vendors.  People are virtualizing entire business lines in an effort to remove the reliance on specialized hardware or reduce the ongoing support costs.  Brocade is seeking to do this with two platforms right now.  The first is the Vyatta vRouter, which is the extension what came over in the Vyatta acquisition.  It’s a router and a firewall and even a virtual private networking (VPN) device that can run on just about anything.  It is hypervisor agnostic and cloud platform agnostic as well.  The idea is that Brocade can include a copy of the vRouter with application packages that can be downloaded from an enterprise cloud app store.  Once downloaded and installed, the vRouter can be fired up and pull a predefined configuration from the scripts included in the box.  By making it agnostic to the underlying platform, there’s no worry about support down the road.

The second NFV platform Brocade told me about is the virtual ADX application delivery switch.  It’s basically a software load balancer.  That’s not really the key point of the whole idea of applying the NFV template to an existing hardware platform.  Instead, the idea is that we’re taking something that’s been historically huge and hard to manage and moving it closer to the edge where it can be of better use.  Rather that sticking a huge load balancer at the entry point to the data center to ensure that flows are separated, the vADX allows the load balancer to be deployed very close to the server or servers that need to have the information flow metered.  Now, the agility of SDN/NFV allows these software devices to be moved and reconfigured quickly without needing to worry about how much reprogramming is going to be necessary to pull the primary load balancer out or change a ton of rules to take reroute traffic to a vMotioned cluster.  In fact, I’m sure that we’re going to see a new definition of the “network edge” being to emerge as more software-based NFV devices begin to be deployed closer and closer to the devices that need them.

On the OpenFlow front, Brocade told me about their new push toward something they are calling “Hybrid Port OpenFlow.”  OpenFlow is a great disruptive SDN technology that is gaining traction today, largely in part because of companies like Brocade and NEC that have embraced it and started pushing it out to their customer base well ahead of other manufacturers.  Right now, OpenFlow support really consists to two modes – ON and OFF.  OFF is pretty easy to imagine.  ON is a bit more complicated.  While a switch can be OpenFlow enabled and still forward normal traffic, the practice has always been to either dedicate the switch to OpenFlow forwarding, in effect turning it into a lab switch, or to enable OpenFlow selectively for a group of ports out of the whole switch, kind of like creating a lab VLAN for testing on a production box.  Brocade’s Hybrid Port OpenFlow model allows you to enable OpenFlow on a port and still allow it to do regular traffic forwarding sans OpenFlow.  That may be the best model for adopters going forward due to one overriding factor – cost.  When you take a switch or a group of ports on a switch and dedicate them for OpenFlow, you are cost the enterprise something.  Every port on the switch costs a certain amount of money.  Every minute an engineer spends working on a crazy lab project incurs a cost.  By enabling the network engineers to turn on OpenFlow at will without disrupting the existing traffic flow, Brocade can reduce the opportunity cost of enabling OpenFlow to almost zero.  If OpenFlow just becomes something that works as soon as you enable it, like IPv6 in Windows 7, you don’t have to spend as much time planning for your end node configuration.  You just build the core and let the end nodes figure out they have new capabilities.  I figure that large Brocade networks will see their OpenFlow adoption numbers skyrocket simply because Hybrid Port mode turns the configuration into Easy Mode.

The last interesting software piece that Brocade showed me is a prime example of the kinds of things that I expect SDN to deliver to us in the future.  Brocade has created an application called the Application Resource Broker (ARB).  It sits above the fray of the lower network layers and monitors indicators of a particular application’s health, such as latency and load.  When one of those indicators hits a specific threshold, ARB kicks in to request more resources from vCenter to balance things out.  If the demand on the application continues to rise beyond the available resources, ARB can dynamically move the application to a public cloud instance with a much deeper pool of resources, a process known as cloudbursting.  All of this can happen automatically without the intervention of IT.  This is one of the things that shows me what SDN can really do.  Software can take care of itself and dynamically move things around when abnormal demand happens.  Intelligent choices about the network environment can be made on solid data.  No guess what about what “might” be happening.  ARB removes doubt and lag in response time to allow for seamless network repair.  Try doing that with a telnet session.

There’s a lot more to the Brocade announcement than just software.  You can check it out at http://www.brocade.com.  You can also follow them on Twitter as @BRCDComm.

Tom’s Take

The future looks interesting at first.  Flying cars, moving sidewalks, and 3D user interfaces are all staples of futuristic science fiction.  The problem for many arises when we need to start taking steps to build those fanciful things.  A healthy dose of pragmatism helps to figure out what we need to do today to make tomorrow happen.  If we root our views of what we want to do with what we can do, then the future becomes that much more achievable.  Even the amazing gadgets we take for granted today have a basis in the real technology of the time they were first created.  By making those incremental steps, we can arrive where we want to be a whole lot sooner with a better understanding of how amazing things really are.

Blog Posts and CISSP CPE Credit

Posted on by
1

CISSPLogoAmong my more varied certifications, I’m a Certified Information Systems Security Professional (CISSP).  I got it a few years ago since it was one of the few non-vendor specific certifications available at the time.  I studied my tail off and managed to pass the multiple choice scantron-based exam.  One of the things about the CISSP that appealed to me was the idea that I didn’t need to keep taking that monster exam every three years to stay current.  Instead, I could submit evidence that I had kept up with the current state of affairs in the security world in the form of Continuing Professional Education (CPE) credits.

CPEs are nothing new to some professions.  My lawyer friends have told me in the past that they need to attend a certain number of conferences and talks each year to earn enough CPEs to keep their license to practice law.  For a CISSP, there are many things that can be done to earn CPEs.  You can listen to webcasts and podcasts, attend major security conferences like RSA Conference or the ISC2 Security Congress, or even give a security presentation to a group of people.  CPEs can be earned from a variety of research tasks like reading books or magazines.  You can even earn a mountain of CPEs from publishing a security book or article.

That last point is the one I take a bit of umbrage with.  You can earn 5 CPEs for having a security article published in a print magazine or other established publishing house.  You can write all you want but you still have to wait on an old fashioned editor to decide that your material was worth of publication before it can be counted.  Notice that “blog post” is nowhere on the list of activities that can earn credit.  I find that rather interesting considering that the majority of security related content that I read today comes in the form of a blog post.

Blog posts are topical.  With the speed that things move in the security world, the ability to react quickly to news as it happens means you’ll be able to generate much more discussion.  For instance, I wrote a piece for Aruba titled Is It Time For a Hacking Geneva Convention?  It was based on the idea that the new frontier of hacking as a warfare measure is going to need the same kinds of protections that conventional non-combat targets are offered today.  I wrote it in response to a NY Times article about the Chinese calling for Global Hacking Rules.  A week later, NATO released a set of rules for cyberwarfare that echoed my ideas that dams and nuclear plants should be off limits due to potential civilian casualties.  Those ideas developed in the span of less than two weeks. How long would it have taken to get that published in a conventional print magazine?

I spend time researching and gathering information for my blog posts.  Even those that are primarily opinion still have facts that must be verified.  I spend just as much time writing my posts as I do writing my presentations.  I have a much wider audience for my blog posts than I do for my in-person talks.  Yet those in-person talks count for CPEs while my blog posts count for nothing.  Blogs are the kind of rapid response journalism that gets people talking and debating much faster than an article in a security magazine that may be published once a quarter.

I suppose there is something to be said for the relative ease with which someone can start a blog and write posts that may be inaccurate or untrue.  As a counter to that, blog posts exist and can be referenced and verified.  If submitted as a CPE, they should need to stay up for a period of time.  They can be vetted by a committee or by volunteers.  I’d even volunteer to read over blog post CPE submissions.  There’s a lot of smart people out there writing really thought provoking stuff.  If those people happen to be CISSPs, why can’t they get credit for it?

To that end, it’s time for (ISC)^2 to start allowing blog posts to count for CPE credit.  There are things that would need to change on the backend to ensure that the content that is claimed is of high quality.  The desire to have only written material allowed for CPEs is more than likely due to the idea that an editor is reading over it and ensuring that it’s top notch.  There’s nothing to prevent the same thing from occurring for blog authors as well.  After all, I can claim CPE credits for reading a lot of posts.  Why can I get credit for writing them?

The company that oversees the CISSP, (ISC)^2, has taken their time in updating their tests to the modern age.  I’ve not only taken the pencil-and-paper version, I’ve proctored it as well.  It took until 2012 before the CISSP was finally released as a computer-based exam that could be taken in a testing center as opposed to being herded into a room with Scantrons and #2 pencils.  I don’t know whether or not they’re going to be progressive enough to embrace new media at this time.  They seem to be getting around to modernizing things on their own schedule, even with recent additions of more activist board members like Dave Lewis (@gattaca).

Perhaps the board doesn’t feel comfortable allowing people to post whatever they want without oversight or editing.  Maybe reactionary journalism from new media doesn’t meet the strict guidelines needed for people to learn something.  It’s tough to say if blogs are more popular than the print magazines that they forced into email distribution models and quarterly publication as opposed to monthly.  What I will be willing to guarantee is that the quality of security-related blog posts will continue to be high and can only get higher as those that want to start claiming those posts for CPE credit really dig in and begin to write riveting and useful articles.  The fact that they don’t have to be wasted on dead trees and overpriced ink just makes the victory that much sweeter.

Tweebot For Mac – The Only Client You Need

Posted on by
2

TweetbotBirdI live my day on Twitter.  Whether it be learning new information, sharing information, or having great conversations I love the interactions that I get.  Part of getting the most out of Twitter comes from using a client that works to present you with the best experience.  Let me just get this out of the way: the Twitter web interface sucks.  It’s clunky and expends way too much real estate to provide a very minimal amount of useful information.  I’m constantly assaulted with who I should be following, what’s trending, and who is paying for their trends to float to the top of the list.  I prefer to digest my info a little bit differently.

You may recall that when I used Windows I was a big fan of the Janetter app.  When I transitioned to using a Mac full time, I started using Janetter at first to replicate my workflow.  I still kept my eyes open for a more streamlined client that I could keep on my desktop in the background.  While I loved the way the Mac client from Twitter (nee Tweetie) displayed things, I knew that development on that client had all but ended when Loren Britcher left Twitter.  Thankfully, Mark Jardine and Paul Haddad had been busy in the mad science lab to save me.

I downloaded Tweetbot for iOS back when I used an iPhone 3GS.  I loved the interface, but the program was a bit laggy on my venerable old phone.  When I moved to an iPhone 4S, I started using Tweetbot all the time.  This was around the time that Twitter decided to start screwing around with their mobile interface through things like the Dickbar.  Tweetbot on my phone was streamlined.  It allowed me to use gestures to see conversations.  I could see pictures inline and quickly tap links to pull up websites.  I could even send those links to mobile Safari or Instapaper as needed.  It fit my workflow needs perfectly.  It met them so well that I spent most of my time checking Twitter on my phone instead of my desktop.

The wiz kids at Tapbots figure out that a client for Mac was one of their most requested features.  So the got cooking on it.  They released an alpha for us to break and test the living daylights out of.  I loved the alpha so much I immediately deleted all other clients from my Mac and started using it no matter how many undocumented features I had to live through.  I used the alpha/beta clients all the way up to the release.  The same features I loved from the mobile client were there on my desktop.  It didn’t take up tons of room on a separate desktop.  I could use gestures to see conversations.  They even managed to add new features like multi-column support to mimic one of Tweetdeck’s most popular features.  When I found that just before NFD4, I absolutely fell in love.

TweetbotMacShot

Tweetbot is beautiful.  It is optimized for retina displays on the new MacBooks, so when you scale it up to HiDPI (4x resolution) it doesn’t look like pixelated garbage.  Tweets can be streamed to the client so you don’t constantly have to pull down to refresh your timeline.  I can pin the timeline to keep up with my tweeps at my leisure instead of the client’s discretion.  I even have support within iCloud to keep my mobile Tweetbot client synced to the position of my desktop client and vice versa.  If I read tweets on my phone, my timeline position is updated when I get back to my desk.  I think that almost every feature that I need from Twitter is represented here without the fluff of promoted tweets or ads that don’t apply to me.

That’s not to say that all this awesomeness doesn’t come without a bit of bad news.  If you hop on over to the App Store, you’re going to find out that Tweetbot for Mac costs $20 US. How can a simple Twitter client cost that much?!?  The key lies in the changes to Twitter’s API in version 1.1.  Twitter has decided that third party clients are the enemy.  All users should be using the website or official clients to view things.  Not coincidentally, the website and official clients also have promoted tweets and trends injected into your timeline.  Twitter wants to monitize their user base in the worst way.  I’m sure it’s because they see Mark Zuckerberg sitting on a pile of cash at Facebook and want the same thing for themselves.  They key to that is controlling the user experience.  If they can guarantee that users will see ads they can charge a hefty fee to advertisers.  The only way to ensure that users see those ads is via official channels.  That means that third party clients like Tweetbot can’t be allowed to exist.

In order to lock the clients out without looking like they are playing favorites, a couple of changes were put in place.  First, non-official clients are limited to a maximum of 100,000 user tokens.  Once you hit your limit, you have to go back to Twitter and ask for more.  However, if Twitter determines that your client “replicates official features and offers no unique features,” you get the door slammed in your face and no more user tokens.  It’s already happened to one client.  If you don’t want to hit your limit too quickly, the only option is to make the price in the store much higher than the “casual” user is willing to pay.  As Greg Ferro (@etherealmind) likes to say, Tweetbot is “reassuringly expensive.”

Tom’s Take

I have a ton of apps on my phone and my MacBook that I’ve used once or twice. I paid the $.99 or $1.99 to test them out and found that they don’t meet my needs.  When Tweetbot was finally released, I didn’t hesitate to buy it even though it was $20.  As much as I use Twitter, I can easily justify the cost to myself.  I need a client that doesn’t get in my way. I want flexibility.  I don’t want the extra crap that Twitter is trying to force down my throat.  I want to use Twitter.  I don’t want Twitter to use me.  That’s what I get from Tweetbot.  I don’t need the metrics from Hootsuite.  I just want to read and respond to conversations and save articles for later.  Thanks to Twitter’s meddling, a lot of people have been looking for a replacement for the old Tweetdeck Air client that is getting sunsetted on May 7.  I can honestly say without reservation that Tweetbot for Mac is the replacement you’re looking for.

Review Disclaimer

I am a paying user of Tweetbot for iPhone, iPad, and Mac.  These programs were purchased by me.  This review was written without any prior contact with Tapbots.  They did not solicit any of the content or ask for any consideration in the writing of this article.  The conclusions and analysis herein are mine and mine alone.

Generation Lost

Posted on by
2

I’m not trying to cause a big sensation (talking about my generation) – The Who

GenTiltedNaming products is an art form.  When you let the technical engineering staff figure out what to call something, you end up with a model number like X440 or 8086.  When the marketing people get involved at first, you tend to get more order in the naming of things, usually in a series like the 6500 series or the MX series.  The idea that you can easily identify a product’s specs based on its name or a model number is nice for those that try to figure out which widget to use.  However, that’s all changing.

It started with mobile telephones.  Cellular technology has been around in identifiable form since the late 1970s.  The original analog signals worked on specific frequencies and didn’t have great coverage.  It wasn’t until the second generation of this technology moved entirely to digital transmission with superior encoding that the technology really started to take off.  In order to differentiate this new technology from the older analog version, many people made sure to market it as “second generation”, often shortening this to “2G” to save syllables.  When it came time to introduce a successor to the second generation personal carrier service (PCS) systems, many carriers started marketing their offerings withe moniker of “3G”, skipping straight past the idea of third generation offering in favor of the catchier marketing term.  AT&T especially loved touting the call quality and data transmission rate of 3G in advertising.  The 3G campaigns were so successful that when the successor to 3G was being decided, many companies started trying to market their incremental improvements as “4G” to get consumers to adopt them quickly.

Famously, the incremental improvement to high speed packet access (HSPA) that was being deployed en masse before the adoption of Long Term Evolution (LTE) as the official standard was known as high speed packet downlink access (HSPDA).  AT&T petitioned Apple to allow their carrier badge on the iPhone to say “4G” when HSPDA was active.  Even though speeds weren’t nearly as fast as the true 4G LTE standard, AT&T wanted a bit of marketing clout with customers over their Verizon rivals.  When the third generation iPad was released with a true LTE radio later on, Apple made sure to use the “LTE” carrier badge for it.  When the iOS 5 software release came out, Apple finally acquiesced to AT&T’s demands and rebranded the HSPDA network to be “4G” with a carrier update.  In fact, to this day my iPhone 4S still tells me I’m on 4G no matter where I am.  Only when I drop down to 2G does it say anything different.

The fact that we have started referring to carrier standards a “xG” something means the marketing is working.  And when marketing works, you naturally have to copy it in other fields.  The two most recent entries in the Generation Marketing contest come from Broadcom and Brocade.  Broadcom has started marketing their 802.11ac chipsets as “5G wireless.”  It’s somewhat accurate when you consider the original 802.11 standard through 802.11b, 802.11g, 802.11n, and now 802.11ac.  However, most wireless professionals see this more as an attempt to cash in on the current market trend of “G” naming rather than showing true differentiation.  In Brocade’s case, they recently changed the name of their 16G fibre channel solution to “Gen 5″ in an attempt to shift the marketing message away from a pure speed measurement (16 gigabit) especially when starting to put it up against the coming 40 gigabit fibre channel over Ethernet (FCoE) offerings coming from their competitor at Cisco.

In both of these cases, the shift has moved away from strict protocol references or speed ratings.  That’s not necessarily a bad thing.  However, the shift to naming it “something G” reeks quite frankly.  Are we as consumers and purchases so jaded by the idea of 3G/4G/5G that we don’t get any other marketing campaigns?  What if they’d call it Revision Five or Fifth Iteration instead?  Doesn’t that convey the same point?  Perhaps it does, but I doubt more than an handful of CxO type people know what iteration means without help from a pocket dictionary.  Those same CxOs know what 4G/5G mean because they can look down at their phone and see it.  More Gs are better, right?

Generational naming should only be used in the broadest sense of the idea.  It should only be taken seriously when more than one company uses it.  Is Aruba going to jump on the 5G wireless bandwagon?  Will EMC release a 6G FC array?  If you’re shaking your head in answer to these questions, you probably aren’t the only one.  Also of note in this discussion – What determines a generation?  IT people have trouble keeping track of what constitutes the difference between a major version change and a point release update.  Why did 3 features cause this to be software version 8.0 but the 97 new features in the last version only made it go from 7.0 to 7.5?  Also, what’s to say that a company doesn’t just skip over a generation?  Why was HSPDA not good enough to be 4G?  Because the ITU said it was just an iteration of 3G and not truly a new generation.  How many companies would have looked at the advantage of jumping straight to 5G by “counting” HSPDA as the fourth generation absent oversight from the ITU?

Tom’s Take

My mom always told me to “call a spade a spade.”  I don’t like the idea of randomly changing the name of something just to give it a competitive edge.  Fibre channel has made it this far as 2 gig, 4 gig, and 8 gig.  Why the sudden shift away from 16 gig?  Especially if you’re going to have to say it runs at 16 gig so people will know what you’re talking about?  Is it a smoke-and-mirrors play?  Why does Broadcom insist on naming their wireless 5G?  802.11a/b/g/n has worked just fine up to now.  Is this just an attempt to confuse the consumer?  We may never know.  What we need to do in the meantime is consider holding feet to the fire and ensuring that consumers and purchasers ensure that this current naming “generation” gets lost.

IOS X-Treme!

Posted on by
6

IOSXtreme

As a nerd, I’m a huge fan of science fiction. One of my favorite shows was Stargate SG-1. Inside the show, there was a joke involving an in-universe TV program called “Wormhole X-Treme” that a writer unintentionally created based on knowledge of the fictional Stargate program. Essentially, it’s a story that’s almost the same as the one we’re watching, with just enough differences to be a totally unique experience. In many ways, that’s how I feel about the new versions of Cisco’s Internetwork Operating System (IOS) that have been coming out in recent months. They may look very similar to IOS. They may behave similarly to IOS. But to mistake them for IOS isn’t right. In this post, I’m going to talk about the three most popular IOS-like variants – IOS XE, IOS XR, and NX-OS.

IOS XE

IOS XE is the most IOS-like of all the new IOS builds that have been released. That’s because the entire point of the IOS XE project was to rebuild IOS to future proof the technology. Right now, the IOS that runs on routers (which will henceforth be called IOS Classic) is a monolithic kernel that runs all of the necessary modules in the same memory space. This means that if something happens to the routing engine or the LED indicator, it can cause the whole IOS kernel to crash if it runs out of memory. That may have been okay years ago but today’s mission critical networks can’t afford to have a rogue process bringing down an entire chassis switch. Cisco’s software engineers set out on a mission to rebuild the IOS CLI on a more robust platform.

IOS XE runs as a system daemon on a “modern Linux platform.” Which one is anyone’s guess. Cisco also abstracted the system functions out of the main kernel and into separate processes. That means that if one of them goes belly up it won’t take the core kernel with it. One of the other benefits of running the kernel as a system daemon is that you can now balance the workload of the processes across multiple processor cores. This was one of the more exciting things to me when I saw IOS XE for the first time. Thanks to the many folks that pointed out to me that the ASR 1000 was the first device to run IOS XE. The Catalyst 4500 (the first switch to get IOS XE) is using a multi core processor to do very interesting things, like the ability to run inline Wireshark on a processor core while still letting IOS have all the processor power it needs. Here’s a video describing that:

Because you can abstract the whole operation of the IOS feature set, you can begin to do things like offer a true virtual router like the CSR 1000. As many people have recently discovered, the CSR 1000 is built on IOS XE and can be booted and operated in a virtualized environment (like VMware Fusion or ESXi). The RAM requirements are fairly high for a desktop virtualization platform (CSR requires 4GB of RAM to run), but the promise is there for those that don’t want to keep using GNS3/Dynamips or Cisco’s IOU to emulate IOS-like features. IOS XE is the future of IOS development. It won’t be long until the next generation of supervisor engines and devices will be using it exclusively instead of relying on IOS Classic.

IOS XR

In keeping with the sci-fi theme of this post, IOS XR is what the Mirror Universe version of IOS would look like. Much like IOS XE, IOS XR does away with the monolithic kernel and shared memory space of IOS Classic. XR uses an OS from QNX to serve as the base for the IOS functions. XR also segments the ancillary process in IOS into separate memory spaces to prevent system crashes from an errant bug. XR is aimed at the larger service provider platforms like the ASR 9000 and CRS series of routers. You can see that in the way that XR can allow multiple routing protocol processes to be executed at the same time in different memory spaces. That’s a big key to the service provider.

What makes IOS XR so different from IOS Classic? That lies in the configuration method. While the CLI may resemble the IOS that you’re used to, the change methodology is totally foreign to Cisco people. Instead of making live config changes on a live system, the running configuration is forked into a separate memory space. Once you have created all the changes that you need to make, you have to perform a sanity check on the config before it can be moved into live production. That keeps you from screwing something up accidentally. Once you have performed a sanity check, you have to explicitly apply the configuration via a commit command. In the event that the config you applied to the router does indeed contain errors that weren’t caught by the sanity checker (like the wrong IP), you can issue a command to revert to a previous working config in a process known as rollback. All of the previous configuration sets are retained in NVRAM and remain available for reversion.

If you’re keeping track at home, this sounds an awful lot like Junos. Hence my Mirror Universe analogy. IOS XR is aimed at service providers, which is a market dominated by Juniper. SPs have gotten very used to the sanity checking and rollback capabilities provided by Junos. Cisco decided to offer those features in an SP-specific IOS package. There are many that want to see IOS XR ported from the ASR/CSR lines down into more common SP platforms. Only time will tell if that will happen. Jeff Fry has an excellent series of posts on IOS XR that I highly recommend if you want to learn more about the specifics of configuration on that platform.

NX-OS

NX-OS is the odd man out from the IOS family. It originally started life as Cisco’s SAN-OS, which was responsible for running the MDS line of fibre channel switches. Once Cisco started developing the Nexus switching platform, they decided to use SAN-OS as the basis for the operating system, as it already contained much of the code that would be needed to allow networking and storage protocols to interoperate on the device, a necessity for a converged data center switch. Eventually, the new OS became known as NX-OS.

NX-OS looks similar to the IOS Classic interface that most engineers have become accustomed to. However, the underlying OS is very different from what you’re used to. First off, not every feature of classic IOS is available on demand. Yes, a lot of the more esoteric feature sets (like the DHCP server) are just plain unavailable. But even the feature sets that are listed as available in the OS may not be in the actual running code. You need to active each of these via use of the feature keyword when you want to enable them. This “opt in” methodology ensures that the running code only contains essential modules as well as the features you want. That should make the security people happy from an exploit perspective, as it lowers the available attack surface of your OS.

Another unique feature of NX-OS is the interface naming convention. In IOS Classic, each interface is named via the speed. You can have Ethernet, FastEthernet, GigabitEthernet, TenGigabit, and even FortyGigabit interfaces. In NX-OS, you have one – Ethernet. NX-OS treats all interfaces as Ethernet regardless of the underlying speed. That’s great for a modular switch because it allows you to keep the same configuration no matter which line cards are running in the device. It also allows you to easily port the configuration to a newer device, say from Nexus 5500 to Nexus 6000, without needed to do a find/replace operation on the config and risk changing a line you weren’t supposed to. Besides, most of the time the engineer doesn’t care about whether an interface is gigabit or ten gigabit. They just want to program the second port on the third line card.

Tom’s Take

No software program can survive without updates. Especially if it is an operating system. The hardware designed to run version 1.0 is never the same as the hardware that version 5.0 or even 10.0 utilizes. Everything evolves to become more efficient and useful. Think of it like seasons of sci-fi shows. Every season tells a story. There may be some similarities, but people overall want the consistency of the characters they’ve come to love coupled with new stories and opportunities to increase character development. Network operating systems like IOS are no different. Engineers want the IOS-like interface but they also want separated control planes, robust sanity checking, and modularized feature insertion. Much like the writers of sci-fi, Cisco will continue to provide new features and functionality while still retaining the things to which we’ve grown accustomed. However, if Cisco ever comes up with a hare-brained idea like the Ori, I can promise there’s no way I’ll ever run IOS-Origin.

Death of Conversation and the Nuclear Option

Posted on by
2

If you’ve spent any time online since the founding of the Internet, you know how quickly things can escalate when it comes to arguments.

"Borrowed" from Tony Bourke (@tbourke)

“Borrowed” from Tony Bourke (@tbourke)

This is no more apparent to me than the recent discourse surrounding “Donglegate.”  The short, short version:

Male Pycon attendees make inappropriate comments.  Female attendee gets upset and publicly tweets about it.  Stuff happens.  People get fired.

I’m not going to go into anything about the situation, as that’s not my place or my area of expertise to comment.  What I did find upsetting was that after the first attendee that made the inappropriate comments was fired from his job, there was a massive Distributed Denial of Service (DDoS) attack launched at the employer of Adria Richards (the female attendee).  Their website was knocked offline for a couple of days.  The news that Ms. Richards had been let go from her job had to be posted on Facebook initially, as there was no other official communication method available.  It wasn’t until the news broke of her removal that the DDoS finally died down to the point where normal operations could continue.

This is a disturbing trend that I’m starting to see in many online disagreements.  As an increasingly online society, we have started to forego polite discourse and jump straight to the “nuclear strike” option of retaliation.  Don’t like a blogger’s post?  Nuke his site with a DoS tool like LOIC.  Think a vendor employee did something wrong?  Shame them in public and release their private information (also known as “doxing”).  Even noted security researcher Brian Krebs had the local SWAT team called to his house after he wrote about a service used to knock websites offline.

How did we end up at the point where we’ve skipped past “I disagree with what you say and would like to debate this topic!” all the way to “You suck and I’m going to burn down your house and the hospital you were born in!!!“?  Rather than have a meaningful and rational discussion, it appears to be in vogue to nuke anything and everything associated with the person that has made you upset.

Look at what’s happened to Spamhaus recently.  Yes, the articles posting about a massive 300Gbps “Internet breaking” DDoS were a bit overblown.  Yet someone has decided that the best way to make Spamhaus “pay” for their crimes is to launch an attack that relies on using DNS exploits to amplify the traffic to the point where even DDoS frontends like CloudFlare are having a hard time keeping up.  Spamhaus does have a reputation for taking things to the extreme as well when it comes to blacklisting IP ranges suspected of providing havens for spammers.  What you end up with is a standoff where neither side is willing to budge from their viewpoint.  Only they fight their war with packet generators and black hole ACLs that cause problems for users and make ISP technicians pull their hair out.

I’m no stranger to the “nuclear option” myself.  I’ve made some comments on my blog that are a bit…pointed, to put it mildly.  While I do get a bit of satisfaction sometimes from verbally sparring with someone that has called me names or done other such unsavory things, that’s where it ends for me.  I have no desire to do any further harm besides jousting with clever phrases.  I’ve never considered erasing their phone or clogging their Internet connection or releasing their Social Security number online.  Ruining someone for the sake of making a point is the height of pettiness.

Here’s a thought: At the height of the arguments leading up to the American Civil War, where American representatives were calling for state secession opening in Congress, decorum never faltered.  Even when referring to a senator that was despised for their politics, the opponent always called them “the distinguished gentleman from <state>.”  Hard to believe that a conflict that saw families torn apart and Americans shooting at each other by the thousands could still have some polite discussion in a government on the verge of being ripped asunder.  Those rules served to keep a bit of decorum in a place where it was required for conversation and useful argument to take place.

Maybe the problem is anonymity.  It’s far to easy to fire off anonymous comments or be a small cog in a larger DDoS and have a huge impact while staying mostly safe behind a curtain of obscurity.  People who might never utter an ill word to another human being suddenly turn into biased uncompromising trolls.  Rather than discuss rational points, they turn to the most extreme option available to either silence their critics or prove a point in a “scorched earth” victory at any costs.  Consider this XKCD comic:

I laughed when I first read this.  Slowly, I realized that the author is right.  Sometimes reading back the comment to people proves the point better than anything.  I frequently use a commenter’s words in my replies to point out what was said and how it was construed (at least by me).

Tom’s Take

In the end, to me it comes down to a matter of manners.  I’ve always made it a rule here to never say anything about a topic or person that I wouldn’t say to them in person.  I also do my best to look at both sides of an argument with a critical eye.  I don’t call people names or threaten them.  Even when people call me biased, narcissistic, or even just plain stupid I just try and debate the facts of the discussion.  Sure, I may rant and rave and shout out loud to myself sometimes.  However, name-calling never accomplishes anything.  Moving beyond that to the nuclear option is equally appalling to me as well.  I have no desire to knock out anyone’s blog or line of business for the sake of proving a point.  If my arguments don’t suffice to change someone’s mind or get a policy I care for overturned, then that’s my fault and not the fault of others.  I just agree to disagree and move on.  Maybe therein lies the spark that will reignite polite conversation and discussion instead of leaping straight to the last resort.  After all, an attentive ear can win more battles than the sharpest spear.