Tag Archives: The Phoenix Project

I Was A 10x Engineer. And I’m Sorry.

Posted on by
21

You probably saw the big discussion this past weekend on Twitter about 10x Engineers. It all started with a tweet about how to recognize a 10x Engineer, followed by tons of responses about how useless they were and how people that had encountered them were happy to be rid of them. All that discussion made me think back to my old days as a Senior Network Rock Star. As I reminisced I realized that I was, in fact, a 10x Engineer. And I was miserable.

Pour Some Work On Me

I wasn’t always the epitome of engineering hatred. I used to be a wide-eyed technician with a hunger to learn things. I worked on a variety of systems all over the place. In fact, I was rising through the ranks of my company as a Novell Engineer in an environment with plenty of coverage. I was just learning the ropes and getting ready to take my place in a group of interchangeable people.

Then I started getting into networking. I spent more time learning about routers and switches and even firewalls. That meant that my skill set was changing from servers to appliances. It also meant that I was spending more and more time working on devices that no one else could work on. I had special knowledge that made me much more valuable to the organization. Soon, I found myself spending less and less time working on the Novell command line and more time working on the Cisco CLI.

That was the first extra “x” on my resume. Because I had special skills it meant that I was being relied upon more to do work that no one else could do. Suddenly I wasn’t just a replaceable cog in the machine. Instead, I was a critical part of the infrastructure that needed to be on-site for certain jobs and deployments. I knew that I needed to have someone else to help me out or I was going to quickly find myself overwhelmed with work. But networking wasn’t the thing that ended up pushing me all the way to 10x territory.

A Voice In The Wilderness

In order to truly become an insufferable 10x Engineering talent, I had to pivot into voice deployments. That’s because my skill set went from “important but we’re training others” to “so complicated no one else can understand this”. Thanks to my knowledge of networking, I was asked to pick up the voice banner and run with it. And I ran really, really far.

I was the only person in the office working on Cisco voice deployments. I had my own method of doing things. My own flow for deployments. My notes were contained in a OneNote file that only I had access to. You can probably see all the issues already. But I couldn’t. To me, this was my jam! I had all the tools and talent to make this happen. I could type MAC addresses faster than filling them into a BAT spreadsheet. I could configure crazy hacks to get around limitations thanks to all the extra research I was doing. I was invincible!

I was also gumming up the works. Voice deployments had to be constantly rescheduled if I was out of town. Vacations were a distant memory at best. I wasn’t just the most important cog in the machine – I was the machine. Nothing went forward unless I was doing it. And that’s not scalable at all. Even when my boss realized that I couldn’t scale any more he also knew that getting someone up to speed on my deployment methods and knowledge was a very daunting task at best.

This is the classic setup for a 10x Engineer. All the talent in the world and all of the hubris. Large portions of the line-of-business relying on their knowledge and process. No documentation. No way to get things done other than to go through me. If you’ve ever read The Phoenix Project by Gene Kim you realize that I was Brent through and through. In fact, when I finally did get around to reading it I self-identified with him before I’d even made it 100 pages in.

Pride Goeth Before

Ultimately, it wasn’t failure that caused my 10x Engineering career to come to an end. Instead, it was success. I left my old job six years ago to come to Tech Field Day. I knew I wouldn’t be fixing routers or voicemail systems any longer. I’d be in for an entirely new and different kind of work.

But I couldn’t see how my old job would be able to work without me. I secretly confided to some friends that I thought their business would fall apart without me. How could it survive? I was responsible for so much! I was the only one that knew all of these things! Even a lengthy two-week info dump wouldn’t be enough.

The pride and hubris I displayed is still shocking to me all these years later. To think that a company that had been in business for almost 20 years before I got there would go out of business months after I left because they couldn’t do what I was doing. They did stay in business. They changed, for sure. They moved away from my specialized knowledge and found new ways to utilize their talent. They were able to keep my systems up and running with my notes and when the time came to replace them they used new systems that didn’t need my help to manage and install. They survived because they realized that what I represented was special and couldn’t be replicated.

Me? I’m happy they did. I didn’t want my 10x Engineering efforts to be an anchor around either of our necks. I couldn’t go back to fixing my old systems with my new workload. And my old company couldn’t count on me being available to fix things when I was gone. They made the right choices to put themselves into a position to keep going. And just like most positive 10x Engineer stories they found a happy ending.

Tom’s Take

I’m not proud of my engineering roots when it comes to how bad I was at times. It wasn’t always intentional. It was a product of where I was and the work I was doing. But I totally couldn’t see the forest for the trees. I realize now that I should have taken more people under my wing and helped them understand what I was doing. I should have documented my work and used repeatable methods to build processes that could be done by anyone. My institutional knowledge should have been a resource, not a crutch. And I should have had the humility to understand that companies can live and grow past a single engineer. Knowing all that today makes me realize that I may have looked like a 10x Engineer but everyone is much better off now that I’m just a simple ex-engineer.

Silo 2: On-Premise with DevOps

Posted on by
Reply

I had a great time stirring up the hornet’s nest with the last post about DevOps, so I figured that I’d write another one with some updated ideas and clarifications. And maybe kick the nest a little harder this time.

Grounding the Rules

First, we need to start out with a couple of clarifications. I stated that the mantra of DevOps was “Move Fast, Break Things.” As has been rightly pointed out, this was a quote from Mark Zuckerberg about Facebook. However, as has been pointed out by quite a few people, “The use of basic principles to enable business requirements to get to production deployments with appropriate coordination among all business players, including line of business, developers, classic operations, security, networking, storage and other functional groups involved in service delivery” is a bit more of definition than motto.

What exactly is DevOps then? Well, as I have been educated, it’s a principle. It’s an idea. A premise, if you will. An ideal to strive for. So, to say that someone is on a DevOps team is wrong. There is no such thing as a classic DevOps team. DevOps is instead something that many other teams do in addition to their other jobs.

That being said, go ask someone what their job is in an organization. I’m willing to be that a lot of people will tell you their on the “DevOps Team”. I know this because some did a report, which I wrote about here and it includes responses from the “DevOps” team. Which, according to the classic definition, is wrong. Right?

Well, almost. See, this is where this tweet of mine comes into play:

“Pure” DevOps is hard to manage. It involves organizational shifts. It pisses people off because it’s hard to track metrics. You can’t track a person that does some traditional stuff and some of that new Dev-Op stuff. Where does that part of their job end up on a report? Putting someone in a team or a silo is almost as much for the purposes of managing that person as it is for them to do their job. If I put you in a silo, I know what you do. Or, at the very least, I can assign you tasks and responsibilities that you should be doing and grade you on those. If your “silo” is a principle and not a team, it’s crazy to grade the effectiveness of how you integrated with the developers to deliver services effectively. It can be tracked, but not as easily as a checkbox.

Likewise, people fear change. So, instead of putting their people into roles that cross functional barriers and reorganize the workflows, they instead just take the young people that are talking about the “new way” of doing things and put them in a team together. They slap a DevOps on the door and it’s done. We do DevOps now. Or, worse yet, they take the old infrastructure teams, move a few people off of them into a new team, and tell them to figure out what to do while they’re repainting the team name on the door. This has rightly been called “DevOps Washing” but a lot of people.

But what happens when that team starts Devving the Ops? Do they look at the enshrined principles of The Holy Book of DevOps and start trying to change organizational culture a little bit at a time to get the happy ending from The Phoenix Project? Do they eliminate the Brents of the world and give the security teams peace of mind?

Or, do they carve out their own little fiefdoms and start behaving like an integrated team with responsibilities and politics? Do they do things like deploy new projects to the cloud with little support from other teams. With the idea that they now “own” that workflow and can control how it’s used and how their team is viewed? If you read the article above with the report from Veriflow, you’ll find that a lot of organizations are seeing this second behavior.

Just as much as people fear proper change, they also get greedy in their new roles and want to be important to the business. And taking ownership of all the new initiatives, like cloud development, is a great way to be important. And, as much as The Phoenix Project preaches that security should be integrated into the DevOps workflow, you still half the 330 respondents to the above survey saying there is an increase in security threats to their new initiatives in public cloud.

Redefining DevOps

In a way, this “definition” of DevOps is like the title of this post. I’m sure more than a few of you bristled at the use of on-premise. Because, in today’s IT landscape we’re fighting a losing battle against a premise. When you refer to something as happening in a location, you say “on-premises”. If you say “on-premise”, you should be referring to an idea or concept. And yet, so many people in Silicon Valley say “on-premise” when referring to “on site” or “on location”. It’s grammatically wrong. But it sounds hip. It’s not the classical definition of the word and yet that word is slowly be redefined to mean what people are using it to mean. It literally happened with “literally”.

For those railing against the DevOps Washing that’s going on, ask yourself this question: Why? If the pure principles of DevOps are so much better and easier, why is everyone just slapping DevOps on existing teams or reforming other people into teams and running with the DevOps idea instead of following the rules as laid down by the sacred DevOps texts?

It could be that all organizations that are doing it this way are wrong. But are their more organizations doing it the proper way? Or is the lazy way more prevalent? I don’t know the answer, but given the number of products I see aimed at “the DevOps team” or the number of people that have given me feedback about how their organization’s DevOps teams display the same behaviors I talked about in my other blog post, I’d say there are more bad apples than purists out there.

So, what does this all mean for DevOps? Are we going to go on pointing and laughing at the DevOps-In-Name-Only crowd? Are we going to silently moan about how Real DevOps doesn’t happen and that we need to stay pure to the ideals? Or are we going to step back and realize that, just like every other technology or organizational shift that has ever occurred, nothing really gets implemented in its purest form? Instead of complaining that those not doing it the “proper” way are wrong, let’s examine why things get done the way they do and figure out how to fix it.

If businesses are implementing DevOps teams to execute the things they need done, find out why it has to be a dedicated team. Maybe they’re doing it wrong, or maybe they’ve stumbled across something that wasn’t included in the strictest definitions of DevOps. If people are giving work to those teams to accomplish and excluding other functional teams at the same time, don’t just wag your finger at them and tell them that’s not the “right way”. Find out what enabled that team to violate the ideas in the first place. Maybe the DevOps Team is responsible for all cloud deployments. Maybe they want some control over things instead of just a nebulous connection to an ideal.

Tom’s Take

DevOps in theory is a great thing. DevOps as presented in The Phoenix Project is a marvelous idea. But we all know that when theory meets reality, what we get is something different than we expected. It’s not unlike von Moltke’s famous quote, “No plan survives first contact with the enemy.” In theory, DevOps is pure and works like it should. But we’re seeing practice differing greatly from reality. The results are usually the same but the paths are radically different. And for the purists out there, if you don’t want DevOps to suffer the same fate as on-premise, you need to start asking yourself the same hard questions we are supposed to ask organizations as they start to deploy these ideas.