Linux and the Quest for Underlays

Posted on by
1

TuxUnderlay

I’m at the OpenStack Summit this week and there’s a lot of talk around about building stacks and offering everything needed to get your organization ready for a shift toward service provider models and such. It’s a far cry from the battles over software networking and hardware dominance that I’m so used to seeing in my space. But one thing came to mind that made me think a little harder about architecture and how foundations are important.

Brick By Brick

The foundation for the modern cloud doesn’t live in fancy orchestration software or data modeling. It’s not because a retailer built a self-service system or a search engine giant decided to build a cloud lab. The real reason we have a growing market for cloud providers today is because of Linux. Linux is the underpinning of so much technology today that it’s become nothing short of ubiquitous. Servers are built on it. Mobile operating systems use it. But no one knows that’s what they are using. It’s all just something running under the surface to enable applications to be processed on top.

Linux is the vodka of operating systems. It can run in a stripped down manner on a variety of systems and leave very little trace behind. BSD is similar in this regard but doesn’t have the driver support from manufacturers or the ability to strip out pieces down to the core kernel and few modifications. Linux gives vendors and operators the flexibility to create a software environment that boots and gets basic hardware working. The rest is up to the creativity of the people writing the applications on top.

Linux is the perfect underlay. It’s a foundation that is built upon without getting in the way of things running above it. It gives you predictable performance and a familiar environment. That’s one of the reasons why Cumulus Networks and Dell have embraced Linux as a way to create switch operating systems that get out of the way of packet processing and let you build on top of them as your needs grow and change.

Break The Walls Down

The key to building a good environment is a solid underlay, whether it be be in systems or in networking. With reliable transport and operations taken care of, amazing things can be built. But that doesn’t mean that you need to build a silo around your particular area of organization.

The shift to clouds and stacks and “new” forms of IT management aren’t going to happen if someone has built up a massive blockade. They will work when you build a system that has common parts and themes and allows tools to work easily on multiple parts of the infrastructure.

That’s what’s made Linux such a lightning rod. If your monitoring tools can monitor servers, SANs, and switches with little to no modification you can concentrate your time on building on those pieces instead of writing and rewriting software to get you back to where you started in the first place. That’s how systems can be extensible and handle changes quickly and efficiently. That’s how you build a platform for other things.

Tom’s Take

I like building Lego sets. But I really like building them with the old fashioned basic bricks. Not the fancy new ones from licensed sets. Because the old bricks were only limited by your creativity. You could move them around and put them anywhere because they were all the same. You could build amazing things with the right basic pieces.

Clouds and stacks aren’t all that dissimilar. We need to focus on building underlays of networking and compute systems with the same kinds of basic blocks if we ever hope to have something that we can build upon for the future. You may not be able to influence the design of systems at the most basic level when it comes to vendors and suppliers, but you can vote with your dollars to back the solutions that give you the flexibility to get your job done. I can promise you that when the revenue from proprietary, non-open underlay technologies goes down the suppliers will start asking you the questions you need to answer for them.

Automating Change With Help From Fibonacci

Posted on by
3

FibonacciShell

A few recent conversations that I’ve seen and had with professionals about automation have been very enlightening. It all started with a post on StackExchange about an unsuspecting user that tried to automate a cleanup process with Ansible and accidentally erased the entire server farm at a service provider. The post was later determined to be a viral marketing hoax but was quite believable to the community because of the power of automation to make bad ideas spread very quickly.

Better The Devil You Know

Everyone in networking has been in a place where they’ve typed in something they shouldn’t have. Whether you removed the management network you were using to access the switch or created an access list that denied packets that locked you out of something. Or perhaps you typed an errant debug command that forced you to drive an hour to reboot a switch that was no longer responding. All of these things seem to happen to people as part of the learning process.

But how many times have we typed something in to create a change and found that it broke more than we expected? Like changing a native VLAN on a trunk and bringing down a link we didn’t intend to affect? These unforeseen accidents are the kinds of problems that can easily be magnified by scripts or automation.

I wrote a post about people blaming tools for SolarWinds a couple of months ago. In it, there was a story about a person that uploaded the wrong switch firmware to a server and used an automated tool to kick off an upgrade of his entire network. Only after the first switch failed to return to normal did he realize that he had downloaded an incorrect firmware to the server. And the command he used to kick off the upgrade was not the safe version of the command that checks for compatibility. Instead, it was the quick version of the command that copied the firmware directly into flash and rebooted the switch without confirmation.

While people are quick to blame tools for making mistakes race through the network quickly it should also be realized that those issues would be mistakes no matter what. Just because a system is capable of being automated doesn’t mean that your commands are exempt from being checked and rechecked. Too often a typo or an added word somewhere in the mix causes unintended chaos because we didn’t take the time to make sure there were no problems ahead of time.

Fibbonaci Style

I’ve always tried to do testing and regression in a controlled manner. For some places that have simulators and test networks to try out changes this method might still work. But for those of us that tend to fly by the seat of our pants on production devices, it’s best to artificially limit the damage before it becomes too great. Note that this method works with automation systems too provided you are controlling the logic behind it.

When you go out to test a network-wide change or perform an upgrade, pick one device as your guinea pig. It shouldn’t be something pushing massive production traffic like a core switch. Something isolated in the corner of the network usually works just fine. Test your change there outside production hours and with a fully documented blackout plan. Once you’ve implemented it, don’t touch anything else for at least a day. This gives the routing tables time to settle down completely and all of the aging timers a chance to expire and tables to recompile. Only then can you be sure that you’re not dealing with cached information.

Now that you’ve done it once, you’re ready to make it live to 10,000 devices, right? Abosolutely not. Now that you’ve proven that the change doesn’t cause your system to implode and take the network with it, you pick another single device and do it again. This time, you either pick a neighbor device to the first one or something on the other side of the network. The other side of the network ensures that changes don’t ripple across between devices over the 24-hour watch period. On the other hand, if the change involves direct connectivity changes between two devices you should test them to be sure that the links stay up. Much easier to recover one failed device than 40 or 400.

Once you follow the same procedure with the second upgrade and get clean results, it’s time to move to doing two devices at once. If you have a fancy automation system like Ansible or Puppet this is where you will be determining that the system is capable of handling two devices at once. If you’re still using scripts, this is where you make sure you’re pasting the right info into each window. Some networks don’t like two devices changing information at the same time. Your routing table shouldn’t be so unstable that a change like this would cause problems but you never know. You will know when you’re done with this.

Now that you’ve proven that you can make changes without cratering a switch, a link, or the entire network all at once, you can continue. Move on to three devices, then five, then eight. You’ll notice that these rollout plans are following the Fibonacci Sequence. This is no accident. Just like the appearance of these numbers in nature and math, having a Fibonacci rollout plan helps you evaluate changes and rollback problems before they grow out of hand. Just because you have the power to change the entire network at once doesn’t mean that you should.

Tom’s Take

Automation isn’t the bad guy here. We are. We are fallible creatures that make mistakes. Before, those mistakes were limited to how fast we could log into a switch. Today, computers allow us to make those mistakes much faster on a larger scale. The long-term solution to the problem is to test every change ahead of time completely and hope that your test catches all the problems. If it doesn’t then you had better hope your blackout plan works. But by introducing a rolling system similar to the Fibonacci sequence above. I think you’ll find that mistakes will be caught sooner and problems will be rectified before they are amplified. And if nothing else, you’ll have lots of time to explain to your junior admin all about the wonders of Fibonacci in nature.

Adapting Applications And Avoiding Acrobatic Adjustments

Posted on by
3

JumpingHoops

A couple of months ago, I was on a panel at TechUnplugged where we talked about scaling systems to large sizes. Here’s a link to the video of that panel:

One of the things that we discussed in that panel was applications. Toward the end of the discussion we got into a bit of a back-and-forth about applications and the systems they run on. I feel like it’s time to develop those ideas a bit more.

The Achilles’ Heel

My comments about legacy applications are pointed. If a company is spending thousands of dollars and multiples hours of time in the engineering team to reconfigure the network or the storage systems to support an old application, my response was simple: go out of business.

It does sound a bit flippant to think that a company making a profit should just close the shutters and walk away. But that’s just the problem that we’re facing in the market today. We’ve spent an inordinate amount of time creating bespoke, custom networks and systems to support applications that were written years, or even decades, ago in alien environments.

We do it every day without thinking. We have to install this specific Java version on the server to get the payroll application running. Don’t install the new security updates on this server because it breaks the HR database. We have to have the door security server running in the same VLAN as the camera system or they won’t communicate. The list seems to be endless.

The problem is not that we’re jumping through hoops to fix all of these things. The problem is really that we shouldn’t have to do it. Look at all of the work that’s being done in agile development today. Companies are creating programs that do things that we thought were impossible just a few years ago. Features are being added rapidly without compromising stability. Applications can be deployed and run in the cloud or on-site. It’s a brave new world if you’re willing to throw out the basic assumption that applications are immutable.

The Arrow of Apollo

Everyone has made concessions to an application at some point in their career. We’ve done very dumb things to make software work. We do it so often that it becomes second nature. Entire product features have been developed because applications are too stupid to work in certain scenarios.

Imagine if you will what happens when you’re told that the new HR database doesn’t like it when the database is slow to respond. You likely start thinking of all the ways that you can make the network respond faster or reroute traffic to prevent this scenario from happening. You’re willing to actively destroy and reconfigure network architecture because one software program doesn’t like latency.

How far are you willing to take it? In some cases the answers are pretty scary. Cisco has a line of industrial switches with integrated NAT because some systems don’t know how to handle not being hard-coded with IP addresses. I’ve disabled roaming between access points in a wireless network because the medical records system couldn’t handle roaming to a new AP as it made the database connection close instantly. I’ve even had to disable QoS in a network because the extra latency it was causing in the CRM application was making the sales people cranky.

All of these solutions to problems assumed that the application that was having the problem can’t be fixed. What would happen if we turned that idea back around and fought the fight differently? What if we started the discussion of application problems by working through application programming issues instead? What if the fault of the system wasn’t at the network level or the compute level but instead was at the software level because the development team did a bad job of error handling?

With all the talk of making resources more like the cloud, where the underlay systems exist in a void apart from the software running on top of them we do have the draw the line at bad application programming. Netflix can’t assume that Amazon will create a special channel link to make application recovery faster. They have to write their software in such as way that it can survive problems in the underlay and avoid them.

Application developers need to stop thinking that the underlying systems will always work perfectly and fix any problems that might crop up. Instead, those applications that are being written need to account for issues and solve their own problems. The Netflix Chaos Monkey ensures that application folks don’t get complacent and code their programs to work when other things are absent. If Netflix stopped working for anything less than AWS going down they wouldn’t have the subscriber base they enjoy today. Netflix makes money because they don’t assume that the networks and systems will always be there. They make money because they know how to react when those systems aren’t there.

Tom’s Take

The new era of networking and mobile computing have embraced software. Containers have shown us how we can create instances that can be destroyed and recreated as needed. Applications on mobile devices work like they are supposed to because we can’t get into those subsystems to make workarounds for things. It’s time for other application developers to realize that too. We need devs that are willing to write logic into their programs to take control when things go wrong. Every thing that we can do to limit custom network configurations for applications takes us one step closer to the utopia we’ve always wanted – software that can run anywhere and scale as much as needed. Because resilient software is better than trying to fix the problems in hardware any day.

Intel and the Network Arms Race

Posted on by
1

IntelLogo

Networking is undergoing a huge transformation. Software is surely a huge driver for enabling technology to grow by leaps and bounds and increase functionality. But the hardware underneath is growing just as much. We don’t seem to notice as much because the port speeds we deal with on a regular basis haven’t gotten much faster than the specs we read about years go. But the chips behind the ports are where the real action is right now.

Fueling The Engines Of Forwarding

Intel has jumped into networking with both feet and is looking to land on someone. Their work on the Data Plane Development Kit (DPDK) is helping developers write code that is highly portable across CPU architecture. We used to deal with specific microprocessors in unique configurations. A good example is Dynamips.

Most everyone is familiar with this program or the projects that spawned, Dynagen and GNS3. Dynamips worked at first because it emulated the MIPS processor found in Cisco 7200 routers. It just happened that the software used the same code for those routers all the way up to the first releases of the 15.x train. Dynamips allowed for the emulation of Cisco router software but it was very, very slow. It almost didn’t allow for packets to be processed. And most of the advanced switching features didn’t work at all thanks to ASICs.

Running networking code on generic x86 processors doesn’t provide the kinds of performance that you need in a network switching millions of packets per second. That’s why DPDK is helping developers accelerate their network packet forward to approach the levels of custom ASICs. This means that a company could write software for a switch using Intel CPUs as the base of the system and expect to get good performance out of it.

Not only can you write code that’s almost as good as the custom stuff network vendors are creating, but you can also have a relative assurance that the code will be portable. Look at the pfSense project. It can run on some very basic hardware. But the same code can also run on a Xeon if you happen to have one of those lying around. That performance boost means a lot more packet switching and processing. No modifications to the code needed. That’s a powerful way to make sure that your operating system doesn’t need radical modifications to work across a variety of platforms, from SMB and ROBO all the way to an enterprise core device.

Fighting The Good Fight

The other reason behind Intel’s drive to get DPDK to everyone is to fight off the advances of Broadcom. It used to be that the term merchant silicon meant using off-the-shelf parts instead of rolling your own chips. Now, it means “anything made by Broadcom that we bought instead of making”. Look at your favorite switching vendor and the odds are better than average that the chipset inside their most popular switches is a Broadcom Trident, Trident 2, or even a Tomahawk. Yes, even the Cisco Nexus 9000 runs on Broadcom.

Broadcom is working their way to the position of arms dealer to the networking world. It soon won’t matter what switch wins because they will all be the same. That’s part of the reason for the major differentiation in software recently. If you have the same engine powering all the switches, your performance is limited by that engine. You also have to find a way to make yourself stand out when everything on the market has the exact same packet forwarding specs.

Intel knows how powerful it is to become the arms dealer in a market. They own the desktop, laptop, and server market space. Their only real competition is AMD, and one could be forgiven for arguing that the only reason AMD hasn’t gone under yet is through a combination of video card sales and Intel making sure they won’t get in trouble for having a monopoly. But Intel also knows what it feels like to miss the boat on a chip transition. Intel missed the mobile device market, which is now ruled by ARM and custom SoC manufacturing. Intel needs to pull off a win in the networking space with DPDK to ensure that the switches running in the data center tomorrow are powered by x86, not Broadcom.

Tom’s Take

Intel’s on the right track to make some gains in networking. Their new Xeon chips with lots and lots of cores can do parallel processing of workloads. Their contributions to CoreOS will help the accelerate the adoption of containers, which are becoming a standard part of development. But the real value for Intel is helping developers create portable networking code that can be deployed on a variety of devices. That enables all kinds of new things to come, from system scaling to cloud deployment and beyond.

Video And The Death Of Dialog

Posted on by
1

video

I was reading a trivia article the other day about the excellent movie Sex, Lies, & Videotape when a comment by the director, Stephen Soderbergh, caught my eye. The quote, from this article talks about how people use video as a way to distance ourselves from events. Soderbergh used it as a metaphor in a movie made in 1989. In today’s society, I think video is having this kind of impact on our careers and our discourse in a much bigger way.

Writing It Down In Pictures

People have become huge consumers of video. YouTube gets massive amounts of traffic. Devices have video recording capabilities built in. It’s not uncommon to see a GoPro camera attached to anything and everything and see people posting videos online of things that happen.

My son is a huge fan of videos about watching other people play video games. He’ll watch hours of video of someone playing a game and narrating the experience. When I tell him that he’s capable of playing the game himself he just tells me, “It’s not as fun that way Dad.” I, too, have noticed that a lot of things that would normally have been written down are narrated as videos today.

A great example of this is the Stuck in Traffic video blog series from J Wolfgang Goerlich (@JWGoerlich). These videos are great examples of things that would have been blog posts just a few years ago but have become videos that were narrated and posted to a channel for people to consume. This is also the way that podcasts have risen to dominate the attention of people looking to consume information. But video requires a bit more attention as compared to audio-only discussions.

One of the big issues I see with videos is that they are not living, breathing documents. They exist as they are created with no way to modify the content short of destroying it and recreating it. If I write a blog post and make a factual error, it’s very easy to fix that issue. I can write a note about how I made a mistake and someone pointed it out. Or, in some cases, I can write a whole new post about the error and how I figured out what was wrong.

But video is different. I find all too often that people make factual errors in videos by either misstating something or being plain mistaken. Usually these errors aren’t corrected in the process because the subject is unaware of things. But instead of being able to correct it with a follow up or a postscript, most video producers are forced to cover the incorrect comment with a large annotation in the video window pointing out that they were wrong and force the viewer to pay attention to the comment and not the spoken word or written word in the original video.

The lack of ability to correct problems and create living documents is a huge one for video creators. Errors can’t be easily fixed. On platforms like YouTube you can’t even upload a new video in place of the old one without destroying all of your views and comments. It makes mea culpas a huge pain. There’s no process to fix things unless you catch them before posting.

On Broadcast With No Mic

The other thing that bothers me the most about video is actually very similar to  the reason why I hate keynotes. The whole process of broadcasting a message without soliciting feedback is irritating at best. With a blog post, you can have comments and discussions and even more posts about subjects that go on to create commentary. Videos are static. You can’t start watching one and them come back to it later like you can with a blog post. Of course videos can be paused, but the vast majority of video creators do their best to minimize the amount of dead air in a video.

It’s also very difficult to create discussion with videos. Instead of being able to address points one at a time and create dialog there, you are forced to address or refute points in a series with no stopping. That makes it easy to overlook things without realizing that you missed a great idea or you could have summed things up with a very easy point somewhere else.

What’s missing is the ability to let conversations develop. If you think of Slack and email as the ultimate form of conversation, video is the ultimate form of one-sided discussion. Television, movies, and other video sources are designed to deliver content with no regard for feedback going the other direction. That is the key that is missing to make video be something beyond a simple broadcast medium.

Tom’s Take

Video is a tool designed to get your views across with minimal input from viewers. It took me a while to realize this until I heard my son “closing” a video with the standard “like, share, and subscribe” type of sign off. There wasn’t any mention of leaving comments or creating a video reply. It was really at that point that a I realized that video blogs and channels are the pinnacle of insulating us from the audience. All a creator needs to do it post videos and turn off comments and you can almost guarantee that they can continue creating messages that people will hear but never be able to respond to.

Wireless As We Know It Is Dead

Posted on by
7

WirelessTombstone

Congratulations! We have managed to slay the beast that is wireless. We’ve driven a stake through it’s heart and prevented it from destroying civilization. We’ve taken a nascent technology with potential and turned it into the same faceless corporate technology as the Ethernet that it replaced. Alarmist? Not hardly. Let’s take a look at how 802.11 managed to come to an inglorious end.

Maturing Or Growing Up

Wireless used to be the wild frontier of networking. Sure, those access points bridged to the traditional network and produced packets and frames like all the other equipment. But wireless was unregulated. It didn’t conform to the plans of the networking team. People could go buy a wireless access point and put it under their desk to make that shiny new laptop with 802.11b work without needing to be plugged in.

Wireless used to be about getting connectivity. It used to be about squirreling away secret gear in the hopes of getting a leg up on the poor schmuck in the next cube that had to stay chained to his six feet of network connectivity under the desk. That was before the professionals came in. They changed wireless. They put a suit on it.

Now, wireless isn’t about making my life easier. It’s about advancing the business. It’s about planning and preparation and enabling applications. It’s about buying lots of impressively-specced access points every three years to light up new wings of the building. It’s about surveying for coverage and resource management to make sure the signal is strong everywhere. Everyone has to play nice and understand the rules.

Wireless professionals are the worst of the lot. They used to deal in black magic and secret knowledge that made them the most valuable people on the planet. They alone knew the secrets of how spectrum worked or what co-channel interference was. That was before the dark times. Before people wanted to learn more about it. Now, we can teach people these concepts. How to use tools to fix problems. Why things must be laid out in certain ways to maximize usefulness. We’ve made everyone special.

Now, the business doesn’t want wizards with strange work habits and even stranger results. They want the same predictable group that they’ve gotten for the last decade with the network team. They want people to blame when their application is slow. They want the infrastructure to work full time in every little corner of the building. And when it doesn’t, they want to know whose head must roll for this affront!

The Establishment

Another thing that destroyed wireless was everyone’s attempt to make it mainstream. Gartner’s Wired and Wireless reports didn’t help. Neither did the push to create tools that make it easy to diagnose issues with a minimum of effort. Now, companies think that wireless is something that just happens. Something that doesn’t take planning to execute. Now, wireless professionals are fired or marginalized because it shouldn’t take that much money to configure something so simple, right?

Why do wireless people need professional development? The networking team gets by with reading those old dusty books. How much can wireless really change year to year? It just gets faster and more expensive. Why should you have to learn how to put up those little access points all over again?

Now that wireless is a part of the infrastructure like switches and routers, it’s time to be forgotten. Now the business needs to focus on other technology that’s likely to be implemented incorrectly that doesn’t support the mission of the business. You know, the kinds of things that we read about in industry trade magazines that they use in sports stadiums or hospitals that sound really awesome and can’t be all that expensive, right?

Tom’s Take

We killed wireless because we used it to do the job it was designed to do. We made it boring and useful and pervasive. As soon as a technology achieves that level of use it naturally becomes something unimportant. Which you will be quick to argue about until you realize that you’re probably reading this from a smartphone that is so commonplace you forget you’re using it.

Now we talk about the apps and technology we’re building on top of wireless. Mobility, location, and other things that are more appealing to people shelling out money to buy things. Buyers don’t want boring. They want expensive gadgets they can point to and loudly proclaim that they spent a lot for this bauble.

Wireless is a victim of its own success. We fought to make it a part of the mainstream and now that it is no one cares about it any more. Now that we take it for granted we must accept that it’s not a “thing” any more. It just is.

Thoughts On Encryption

Posted on by
2

encryption

The debate on encryption has heated up significantly in the last couple of months. Most of the recent discussion has revolved around a particular device in a specific case but encryption is older than that. Modern encryption systems represent the culmination of centuries of development of making sure things aren’t seen.

Encryption As A Weapon

Did you know that twenty years ago the U.S. Government classified encryption as a munition? Data encryption was classified as a military asset and placed on the U.S. Munitions List as an auxiliary asset. The control of encryption as a military asset meant that exporting strong encryption to foreign countries was against the law. For a number of years the only thing that could be exported without fear of legal impact was regular old Data Encryption Standard (DES) methods. Even 3DES, which is theoretically much stronger but practically not much better than it’s older counterpart, was restricted for export to foreign countries.

While the rules around encryption export have been relaxed since the early 2000s, there are still some restrictions in place. Those rules are for countries that are on U.S. Government watch lists for terror states or governments deemed “rogue” states. This is why you must attest to not living in or doing business with one of those named countries when you download any software that contains encryption protocols. The problem today is that almost every piece of software includes some form of encryption thanks to ubiquitous functions like OpenSSL.

Even the father of Pretty Good Privacy (PGP) was forced to circumvent U.S. Law to get PGP into the hands of the world. Phil Zimmerman took the novel approach of printing the entirety of PGP’s source code in book form and selling it far and wide. Since books are protected speech, no one could stop them from being sold. The only barrier to creating PGP from the text was how fast one could type. Today, PGP is one of the most popular forms of encrypting written communications, such as emails.

Encryption As A Target

Today’s issues with encryption are rooted in the idea that it shouldn’t be available to people that would use it for “bad” reasons. However, instead of being able to draw a line around a place on a map and say “The people inside this line can’t get access to strong encryption”, we now live in a society where strong encryption is available on virtually any device to protect the growing amount of data we store there. Twenty years ago no one would have guessed that we could use a watch to pay for coffee, board an airplane, or communicate with loved ones.

All of that capability comes with a high information cost. Our devices need to know more and more about us in order to seem so smart. The amount of data contained in innocuous things causes no end of trouble should that data become public. Take the amount of data contained on the average boarding pass. That information is enough to know more about you than is publicly available in most places. All from one little piece of paper.

Keeping that information hidden from prying eyes is the mission of encryption. The spotlight right now is on the government and their predilection to looking at communications. Even the NSA once stated that strong encryption abroad would weaken the ability of their own technology to crack signal intelligence (SIGINT) communications. Instead, the NSA embarked on a policy of sniffing the data before it was ever encrypted by installing backdoors in ISPs and other areas to grab the data in flight. Add in the recent vulnerabilities found in the key exchange process and you can see why robust encryption is critical to protecting data.

Weakening encryption to enable it to be easily overcome by brute force is asking for a huge Pandora’s box to be opened. Perhaps in the early nineties it was unthinkable for someone to be able to command enough compute resources to overcome large number theory. Today it’s not unheard of to have control over resources vast enough to reverse engineer simple problems in a matter or hours or days instead of weeks or years. Every time a new vulnerability comes out that uses vast computing power to break theory it weakens us all.

Tom’s Take

Encryption isn’t about one device. It’s not about one person. It’s not even about a group of people that live in a place with lines drawn on a map that believe a certain way. It’s about the need for people to protect their information from exposure and harm. It’s about the need to ensure that that information can’t be stolen or modified or rerouted. It’s not about setting precedents or looking good for people in the press.

Encryption comes down to one simple question: If you dropped your phone on the floor of DefCon or BlackHat, would you feel comfortable knowing that it would take longer to break into than the average person would care to try versus picking on an easier target or a more reliable method? If the answer to that question is “yes”, then perhaps you know which side of the encryption debate you’re on without even asking the question.

Don’t Touch My Mustache, Aruba!

Posted on by
2

dont-touch-my-mustache

It’s been a year since Aruba Networks became Aruba, a Hewlett-Packard Enterprise Company. It’s  been an interesting ride for everyone involved so far. There’s been some integration between the HPE Networking division and the Aruba teams. There’s been presentations and messaging and lots of other fun stuff. But it all really comes down to the policy of non-interference.

Don’t Tread On Me

HPE has done an admirable job of keeping their hands off of Aruba. It sounds almost comical. How many companies have acquired a new piece and then done everything possible to integrate it into their existing core business? How many products have had their identity obliterated to fit in with the existing model number structure?

Aruba isn’t just a survivor. It’s come out of the other side of this acquisition healthy and happy and with a bigger piece of the pie. Dominick Orr didn’t just get to keep his company. Instead, he got all of HPE’s networking division in the deal! That works out very well for them. It allows Aruba to help integrate the HPE networking portfolio into their existing product lines.

Aruba had a switching portfolio before the acquisition. But that was just an afterthought. It was designed to meet the insane requirements of the new Gartner Wired and Wireless Magic Quadrant. It was a product pushed out to meet a marketing need. Now, with the collaboration of both HPE and Aruba, the combined business unit has succeeded in climbing to the top of the mystical polygon and assuming a leading role in the networking space.

Could you imagine how terrible it would have been if instead of taking this approach, HPE had instead insisted on integration of the product lines and renumbering of everything? What if they had insisted that Aruba engineers, who are experts in their wireless field, were to become junior to the HPE wireless teams? That’s the kind of disaster that would have led to the fall of HPE Networking sooner or later. When good people get alienated in an acquisition, they flee for the hills as fast as their feet will carry them. One look at the list of EMC and VMware departures will tell you the truth of that.

You’re Very Welcome

The other thing that makes it an interesting ride is the way that people have reacted to the results of the acquisition. I can remember seeing how folks like Eddie Forero (@HeyEddie) were livid and worried about how the whole mess was going to fall apart. Having spoken to Eddie this week about the outcome one year later, he seems to be much, much more positive than he was in the past. It’s a very refreshing change!

Goodwill is something that is very difficult to replace in the community. It takes ages to earn and seconds to destroy. Acquiring companies that don’t understand the DNA of the company they have acquired run the risk of alienating the users of that solution. It’s important to take stock of how you are addressing your user base and potential customers regularly after you bring a new business into the fold.

HPE has done a masterful job of keeping Aruba customers happy by allowing Aruba to keep their communities in place. Airheads is a perfect example. Aruba’s community is a vibrant place where people share knowledge and teach each other how to best utilize solutions. It’s the kind of place that makes people feel welcome. It would have been very easy for HPE to make Airheads conform to their corporate policies and use their platforms for different purposes, such as a renewed focus on community marketing efforts. Instead, we have been able to keep these resources available to all to keep a happy community all around.

Tom’s Take

The title above is actually holds a double meaning. You might think it refers to keeping your hands off of something. But “don’t touch my mustache” is a mnemonic phrase to help people remember the Japanese phrase do itashimashite which means “You’re Welcome”.

Aruba has continued to be a leader in the wireless community and is poised to make waves in the networking community once more because HPE has allowed it to grow through a hands-off policy. The Aruba customers and partners should be very welcome that things have turned out as they have. Given the graveyard of failed company acquisitions over they years, Aruba and HPE are a great story indeed.

Slacking Off

Posted on by
1
A Candlestick Phone (image courtesy of WIkipedia)

A Candlestick Phone (image courtesy of WIkipedia)

There’s a great piece today on how Slack is causing disruption in people’s work habits. Slack is a program that has dedicated itself to getting rid of email, yet we now find ourselves mired in Slack team after Slack team. I believe the real issue isn’t with Slack but instead with the way that our brains are wired to handle communication.

Interrupt Driven

People get interrupted all the time. It’s a fact of life if you work in business, not just IT. Even if you have your head down typing away at a keyboard and you’ve closed out all other forms of distraction, a pop up from an email or a ringing or vibrating phone will jar your concentration out of the groove and force your brain to deal with this new intruder into your solitude.

That’s evolution working against you. When we were hunters and gatherers our brain had to learn how to deal with external threats when we were focused on a task like stalking a mammoth or looking for sprouts on the forest floor. Our eyes are even developed to take advantage of this. Your peripheral vision will pick up movement first, followed by color, then finally it can discern the shape of an object. So when your email notifier slides out from the system tray or notification window it triggers your primitive need to address the situation.

In the modern world we don’t hunt mammoths or forage for shoots any longer. Instead, our survival instinct has been replaced by the need to answer communications as fast as possible. At first it was returning phone calls before the end of the day. Then it became answering emails expediently. That changed into sending an immediate email response that you had seen the email and were working on a response. Then came instant messaging for corporate environments and the idea of “presence”, which allows everyone to know what you’re doing and when you’re doing it. Which has led us to ever presence – the idea that we’re never really not available.

Think about the last time you saw someone was marked unavailable in a chat window and you sent the message anyway. Perhaps you thought they would see the message the next time they logged in or returned to their terminal. Or perhaps you guessed that they had set their status as away to avoid distraction. Either way, the first thought you had was that this person wasn’t really gone and was available.

Instant messaging programs like Slack bridge the gap because synchronous communications channels like phone calls and asynchronous channels like email. In the past, we could deal with phone calls because it required the attention of both parties involved. A single channel was opened and it was very apparent that you were holding a conversation, at least until the invention of call waiting. On the other hand, email is asynchronous by nature because we can put all of our thoughts down in a single message over the course of minutes or even hours and send it into the void. Reliable delivery ensures that it will make it to the destination but we don’t know when it will be read. We don’t know when the response will come or in what form. The receiving party may not even read your message!

The Need to Please

Think back to the last time you responded to an email. How often did you start your response with “Sorry for the delay” or some version of that phrase? In today’s society, we’ve become accustomed to instant responses to things. Amy Lewis (@CommsNinja) is famous for having an escalation process for reaching her:

  1. Text message
  2. Twitter DM
  3. Email
  4. Phone Call
  5. Anything else
  6. Voice mail

She prefers instant communication and rapid response. In a lot of cases, this is very crucial. If you need an answer to a question quickly there are ways to reach people for immediate reply. But the desire to have immediate response for all forms of communication is a bit much.

Our brains don’t help us in this matter. When we get an email or a communication from someone, we feel compelled to respond to it. It’s like a checkbox that needs to be checked. And so we will drop everything else to work on a reply even if it means we’re displeasing someone for a short time to please someone immediately.

Many of the time management systems that have been created to deal with massive email flows, such as GTD are centered on the idea of dealing with things as the come in and pigeonholing them until they can be dealt with appropriately. By treating everything the same you disappoint everyone equally until everything can evaluated. There are cutouts for high priority communications, but the methods themselves tell you to keep those exceptions small and rare so as not to disrupt the flow of things.

The key to having coherent and meaningful conversations with other people is the same online as it is in person. Rather than speaking before you think, you should take the time to consider your thoughts and respond with appropriately measured words. It’s easier to do this via email since there is built-in delay but it works just the same in instant message conversations as well. An extra minute of thought won’t make someone angry with you, but not taking that extra minute could make someone very cross with you down the road.

Tom’s Take

I agree with people that say Slack is great for small teams spread everywhere to help categorize thoughts and keep projects on track. It takes away the need for a lot of status update emails and digests of communications. It won’t entirely replace email for communications and it shouldn’t be seen that way. Instead, the important thing to realize about programs like Slack is that they will start pushing your response style more toward quick replies with little information. You will need to make a conscious decision to push back a bit to make measured responses to things with more information and less response for the sake of responding. When you do you’ll find that instant messaging tools augment your communications instead of complicating them.

Drowning in the Data of Things

Posted on by
1

DrowningSign

If you saw the news coming out of Cisco Live Berlin, you probably noticed that Internet of Things (IoT) was in every other announcement. I wrote about the impact of the new Digital Ceiling initiative already, but I think that IoT is a bit deeper than that. The other thing that seems to go hand in hand with discussion of IoT is big data. And for most of us, that big data is going to be a big problem.

Seen And Not Heard

Internet of Things is about dumb devices getting smart. Think Flowers for Algernon. Only now, instead of them just being smarter they are also going to be very talkative too. The amount of data that these devices used to hold captive will be unleashed on something. We assume that the data is going to be sent to a central collection point or polled from the device by an API call or a program that is mining the data for another party. But do you know who isn’t going to be getting that data? Us.

IoT devices are going to be talking to providers and data collection systems and, in a lot of cases, each other. But they aren’t going to be talking directly to end users or IT staff. That’s because IoT is about making devices intelligent enough to start making their own decisions about things. Remember when SDN came out and everyone started talking about networks making determinations about forwarding paths and topology changes without human inputs? Remember David Meyer talking about network fragility?

Now imagine that’s not the network any more. Imagine it’s everything. Devices talking to other devices and making decisions without human inputs. IoT gives machines the ability to make a limited amount of decisions based on data inputs. Factory floor running a bit too hot for the milling machines to keep up? Talk to the environmental controls and tell it to lower the temperature by two degrees for the next four hours. Is the shelf in the fridge where the milk is stored getting close to the empty milk jug weight? Order more milk. Did a new movie come out on Netflix that meets your viewing guidelines? Add that movie to the queue and have the TV turn on when your phone enters the house geofence.

Think about those processes for moment. All of them are fairly easy conditional statements. If this, then do that. But conditional statements aren’t cut and dried. They require knowledge of constraints and combinations. And all that knowledge comes from data.

More Data, More Problems

All of that data needs to be collected somehow. That means transport networks are going to be stressed now that there are ten times more devices chatting on them. And a good chunk of those devices, especially in the consumer space, are going to be wireless. Hope your wireless network is ready for that challenge. That data is going to be transported to some data sink somewhere. As much as we would like to hope that it’s a collector on our network, the odds are much better that it’s an off-site collector. That means your WAN is about to be stressed too.

How about storing that data? If you are lucky enough to have an onsite collection system you’d better start buying drives for it now. This is a huge amount of data. Nimble Storage has been collecting analytics data from their storage arrays for a while now. Every four hours they collect more data than there are stars in the Milky Way. Makes you wonder where they keep it? And how long are they going to keep that data? Just like the crap in your attic that you swear you’re going to get around to using one day, big data and analytics platforms will keep every shred of information you want to keep for as long you want to have it taking up drive space.

And what about security? Yeah, that’s an even scarier thought. Realize that many of the breaches we’ve read about in the past months have been hackers having access to systems for extended periods of time and only getting caught after they have exfiltrated data from the system. Think about what might happen if a huge data sink is sitting around unprotected. Sure, terabytes worth of data may be noticed if someone tries to smuggle it out of the DLP device. But all it takes is a quick SQL query against the users tables for social security numbers, a program to transpose those numbers into letters to evade the DLP scanner, and you can just email the file to yourself. Script a change from letters back to numbers and you’ve got a gold mine that someone left unlocked and lying around. We may be concentrating on securing the data in flight right now, but even the best armored car does no good if you leave the bank vault door open.

Tom’s Take

This whole thing isn’t rain clouds and doom and gloom. IoT and Big Data represent a huge challenge for modern systems planning. We have the ability to unlock insight from devices that couldn’t tell us their secrets before. But we have to know how deep that pool will be before we dive in. We have to understand what these devices represent before we connect them. We don’t want our thermostats DDoSing our home networks any more than we want the milling machines on the factory floor coming to life and trying to find Sarah Connor. But the challenges we have with transporting, storing, and securing the data from IoT devices is no different than trying to program on punch cards or figure out how to download emails from across the country. Technology will give us the key to solve those challenges. Assuming we can keep our head above water.