I was reading a great post this week from Gian Paolo Boarina (@GP_Ifconfig) about complexity in networking. He raises some great points about the overall complexity of systems and how we can never really reduce it, just move or hide it. And it made me think about complexity in general. Why are we against complex systems?
Confusion and Delay
Complexity is difficult. The more complicated we make something the more likely we are to have issues with it. Reducing complexity makes everything easier, or at least appears to do so. My favorite non-tech example of this is the carburetor of an internal combustion engine.
Carburetors are wonderful devices that are necessary for the operation of the engine. And they are very complicated indeed. A minor mistake in configuring the spray pattern of the jets or the alignment of them can cause your engine to fail to work at all. However, when you spend the time to learn how to work with one properly, you can make the engine perform even above the normal specifications.
Carburetors have been largely replaced in modern engines by computerized fuel injectors. These systems accomplish the same goal of injecting the fuel-air mixture into the engine. However, they are completely controlled by a computer system instead of being mechanically configured. It’s a great leap forward for people that aren’t mechanics or gear heads. The system either works or it doesn’t. There’s no configuration parameters. Of course, if it doesn’t work there’s also very little that you as a non-mechanic can do to rectify the situation. As Gian Paolo points out, the complexity in the system has just been moved from the carburetor to the computer system running it.
But why is that a bad thing? If the standard user is never supposed to fiddle with the system why is moving the complexity a bad thing? It could be argued that removing complications from the operation and diagnostics of the system are good, but only if you ever intend untrained people to ever work on the system. A non-mechanic might never be able to fix a fuel injector system, but a trained person should be able to fix it quickly. Here, the complexity isn’t a barrier to the people who have been trained properly to anticipate it.
Complexity is only a problem for people who don’t understand it. Whether it’s a routing protocol or a file system, complex things are going to exist no matter what we do. Understanding them doesn’t have to be the job of everyone that uses the system.
A Tangled Web
I remember briefly working with Novell’s original identity management system back when it was still called DirXML. It was horribly complicated. It required a number of XML drivers importing information into eDirectory, which itself had quirks. And that identity repository fed multiple systems via XML rules to populate those data structures. It was a complicated nightmare to end all nightmares.
Except when it worked. When the system did the job properly, it looked like magic. Information entered for a new employee in the HR system automatically created an Active Directory user account in a different system, provisioned an email account in a third different system, and even created a time card entry in a fourth totally different system. The complexity under the hood churned its way through to provide usability to the people that relied on the system. Could they have manually entered all of that information? Sure. But having it automatically happen was a huge time saver for them. And when you apply it to a school where those actions needed to be repeated dozens of times for new students you can see how it would save a significant amount of time.
Here, complexity is the reason the system exists. You didn’t have the capability to feed those individual systems at the time because of the lack of API support or various other reasons. You had to find a way to force feed the information to a system that wasn’t expecting to get it any other way. Complexity here was required. And it worked. Until it didn’t.
Troubleshooting the XML issues in the system and keeping it running with new updates and broken links consumed a huge amount of time for the people I knew that were good at using it. So much time, in fact, that a couple of them made a business out of remotely supported DirXML for customers that utilized it and either didn’t know how to use it or didn’t have the specific knowledge necessary to make it work the way they wanted. Here, the complexity wasn’t only a necessity of the system, but it was a driver to create a new support level for it.
Ultimately, DirXML went away as it was consumed by the Novell Identity Manager. And now, the idea of these systems not having an API is silly. We focus our efforts more on the programming of the API and not on the extra complexity of the layers on top of it. But even those API interactions can be complex. So, we’ve essentially traded one complexity for another. We have simplified some aspects of the complexity while introducing others. We’ve also standardized things without necessarily making them an easier to do.
Tom’s Take
Complexity is bad when we don’t understand it. Trying to explain lagrange points and orbital dynamics is a huge pain when you aren’t talking to rocket scientists. However, most people that understand the complexities of the college football playoff system are more than happy to explain it to you in depth simply because they “get it”. Complexity isn’t always the enemy. If the people working on the system understand it enough to get the reasons why it needs to be complex to fulfill a job requirement then it’s not a bad thing. Instead of trying to move or reduce complexity, we should instead to try to ensure that we don’t add any additional complexity to the system. That’s how you keep the complexity snowball from rolling you over.
The Networking Nerd 
Complexity itself is not necessarily bad, I can agree to that. I see complexity intentionally built into a process or design, simply because the project owner doesn’t have the skills necessary to deploy a simpler option. In your example above, if a mechanic knew nothing about fuel injection, they would never use one despite the clear benefits. In that case, the mechanic fears the unknown of how fuel injection works, and would never use or recommend it to others.
I see this happen frequently on projects. Many times a data or database expert is brought in, paid a high rate, and is allowed to architect a system that is way more complex than what is needed. And then others need to come in and clean up the mess they made.
As you stated, complexity isn’t the thing to fear here. Incompetence, however, is what I believe leads to the complexity taking root to begin with.
Thanks for the post.
Great Post Tom, as usual!
I personally agree with everything you said but would like to add a few points to this as well from a Business Decision Design Perspective. The biggest driver for when it is a complex solution verse a simple solution are not normally the benefits of said solution. Its that the company’s that are making these decisions may not have the staff to manage that complex solution.
In your analogy, lets assume that I have my vehicle and I hire a chief mechanic on my staff to manage my vehicle. Taking this a step further lets assume that this chief mechanic doesn’t know anything about the new technology solutions but he knows everything about the mechanical solutions, from a business perspective why would I choose to go with a solution that my chief mechanic cannot manage? There could always be a huge benefit to a solution that justifies training my chief mechanic or hiring a new mechanic under him, but these are business decisions and not technical decisions.
With all of that said, in today’s IT world we have a few “new” abilities to handle complexity. Lets use a simple networking example. Lets say a solution that we have implemented requires 10 static routes. We can handle adding those 10 static routes once, twice, and maybe even three times. But if we want this to scale to 100 times, now how complex is this solution to manage? How many user mistakes are there going to be with 1000 static routes? Complexity can come in a number of forms, here we are talking about the complexity of managing the solution but there are a number of others like the complexity of troubleshooting. Now if we take this same situation and we add some sort of automation to the 10 static routes solution, Ansible comes to mind here, have we not just reduce the overall complexity of managing the solution?
Hope that makes sense and resonates with you and others.
Respectively,
Zig