Why putting a pane of glass on a pile of sh*t doesn’t solve your problem

There’s been this rumour going around lately, according to which the first step to successful platform transformations is visualizing what you have. “Putting a single pane of glass on top of everything”. I believe this is plain nonsense. In this short opinion piece I want to articulate a long-lived conviction of mine:

Put a pane of glass on a pile of sh*t and all your developers can see is a pile of sh*t. 

But let’s briefly rewind: you’re tasked with somehow “building a platform”. You’re on the DevOps wagon for a while now, results are stalling and your management is trying to find the next big thing because hiring is so tough.

Your setup is - as any enterprise setup is - all over the place. You’re halfway through the migration process from legacy A to legacy B. Good amount containerized, Kubernetes is starting to be rolled out. You’ve probably come across Team Topologies and as you read it you already seemed to hear the sweet gushing sound of increasing deployment frequency.

After digging for a while you understand that the best way to get there would be to go back to some of the nastier areas of your setup and rework them. As an example, configuration management. Rather than having people freestyle script YAML, you could streamline and restructure the workflow to reduce maintenance overhead. That would make a lot of sense. Letting humans write app config files is like letting Tesla engineers mold their own screws for every connection of one part of a model Y to another. That would be mind-bogglingly useless, because it drives failure and makes your setup super hard to maintain.

Another example of nastiness that’s tough to clean up is the intersection and collaboration between cloud operations and developers. Developers are blocked in their work due to dependencies on other teams. The plain Terraform files Ops set up hardly get the job done by themselves. So infrastructure orchestration has to be restructured to enable developer self-service and let engineers provision on their own what they need to run their apps and services.

These are all things that require cultural and behavioural change - they just make us a little uncomfortable because they are hard.

Because the above scenarios would be so uncomfortable, most teams are receptive to the more digestible option: visualize stuff, make it discoverable, document it. Also great to sell to management. If they can see something, you’ve done something. Dashboards are awesome. So you read about service catalogs and visual layers but you don’t actually tackle the hard stuff and you spend your time building a single pane of glass. Through which developers, if you make it to production, can observe a pile of sh*t.

There are a number of reasons why this isn’t a good idea: 

• People tend to overestimate the impact of things they see (logs from a microservice running in a staging environment) and they underestimate what they don’t see (consistency and standards in configuration management). The impact of visualization is very low, the impact of standardization and developer self-service is very high. 
• Your opportunity costs are high. The time you spend piecemealing your service catalog into something that actually works is time you could have spent on the standardization (and get the compatibility of a service catalog out of the box). 
• Visualizing stuff doesn’t change anything for the better. It’s a push and people don’t react to those at all. The better approach is to take one of your teams and spend the time to get them into a clean, structured self-service setup. This in turn affects all other teams and creates a pull-factor. Which is exactly what you want.
  

I’ve seen the above dynamic play out over and over again. The teams I’ve been part of who tackled the hard stuff head-on are those that ultimately won. I’ve never once seen the easy path of visualizing stuff lead anywhere. The impact, if any, is neglectable. If there is one thing I want to encourage you to do is take the path that seems nasty at first, tackles the thorny problems but yields results. So go out there, restructure that app config management, streamline your infrastructure orchestration and enable developer self-service.

You can do it.