What to build first: the house or the front door?

It’s great to see interest in platform engineering picking up steam in the last year or so. There is a lot of buzz around it right now, with established players like Gartner adding it to the Hype Cycle. While I am happy people are finally waking up the advantages of this discipline for their organizations, I also speak to many confused leaders in the industry. People want to get started right away building their Internal Developer Platforms (IDPs), but don’t really know where to begin. And antipatterns emerge.

Building an Internal Developer Platform (IDP) - where to start?

Especially at the enterprise scale I see a lot of tech leaders trying to get an overview of all the existing services across teams, therefore thinking a service catalog is the best place to start to gain more transparency.

The problem is this usually leads to platform teams building something in isolation (e.g. spend a year or more trying to implement a service catalog) and eventually pushing an extremely UI-heavy platform onto developers. This fails every single time.

While it might be a good idea to provide developers with their own dedicated portal to unify developer experience, a UI by itself doesn’t actually improve your delivery setup and just doesn’t deliver much value to your org, if any at all.

As Lee Ditiangkin rightly pointed out, if you “put a pane of glass on a pile of sh*t, all your developers can see is a pile of sh*t.” This is also one of the antipatterns discussed by Kaspar von Grünberg in his “Top 10 fallacies in Platform Engineering”.

So let’s take a step back. What is the actual goal of your platform initiative? When I talk to product owners at top performing engineering orgs, they often come across this compelling vision:

Developers access a developer portal. They select a certain service template and build their service via code scaffolding. Then, with a single CLI command (or some prefer a click in a user interface) everything is up and running in production, without any flaws or things developers need to be concerned about.

Developer portals are first touch point - but nor your IDP

It is true that in such a setup a service catalog like Backstage appears as the first touch point for developers. But it is only the surface, the access point, nothing more. It is the door (an interface, like a CLI, UI or API) to the house (the IDP). So what you want to be careful about is to focus on installing a door, while you are still missing the house behind it.

Because a service catalog like Backstage doesn’t solve how your infrastructure is spun up and handled. It doesn’t take care of connecting resources like databases, storage and DNS to your application. It doesn’t care about which type of environment you want to deploy to and doesn’t let you manage configuration files in a consistent way across the entire app lifecycle. And we didn’t even touch on CI/CD pipelines.

Another problem you’ll run into if you start designing your platform by looking at developer portals and service catalogs (Backstage, LeanIX, ServiceNow, OpsLevel, Cortex, etc.) is finding the right combination of what goes underneath it. How will you know which Platform Orchestrator (the centerpiece of your IDP), which kind of CI/CD setup, infrastructure orchestration approach, etc. will work best with the portal you picked, beforehand? Again, you don’t want to start by installing the door.

Different types of portals have different limitations on what they can integrate with, so taking a portal-first approach might backfire later on. When designing your IDP, you want to start with its core configuration engine, the Platform Orchestrator. Taking an orchestrator-first approach allows you to easily plug and play any portal on top of it, or even two at the same time.

In the latest Gartner report “A Software Engineering Leader’s Guide to Improving Developer Experience”, Manjunath Bhat, Research VP sheds some light into this dark corner of platform engineering with a crystal-clear differentiation:

“Internal developer portals serve as the interface through which developers can discover and access internal developer platform capabilities.”

(Full report for Gartner clients here.) 

There are some prominent examples of leading engineering companies taking this orchestrator-first approach. Go check out Brian Letham’s talk as an example to see how Netflix first built their platform and then put Backstage as an interface on top of it.

Platform orchestration first, interface second

So, what’s the right order to build your house? Start with a solid foundation (a Platform Orchestrator), then build the walls (the golden paths and configuration templates) and finally, add doors and windows (all interfaces, e.g. a CLI, UI or a service catalog).

According to Gartner, you should follow always think about these three pillars:

1. “Improve developer experience by building internal developer platforms to reduce cognitive load, developer toil and repetitive manual work.”
2. “Platforms don’t enforce a specific toolset or approach – it is about making it easy for developers to build and deliver software while not abstracting away useful and differentiated capabilities of the underlying core services”
3. “Platform engineering teams treat platforms as a product (used by developers) and design the platform to be consumed in a self-service manner.”

It’s the Internal Developer Platform (and Platform Orchestrator at its core) that reduces the cognitive load on engineers, developer toil and manual work, not the service catalog or the developer portal on top of it.

Building foundations is hard and unsexy. Pouring cement, making sure its leveled and solid is hard work, and you never see it in the end result. Putting in beautiful bay windows, a great door in a wonderful porch is fun and relatively easy. People often think they should start with these things because they are visible - and make for a great dashboard you can show your manager.Platform engineers, at the start of their platforming journey, might fall for the trap of looking for quick wins, perhaps to show off to non-technical management and essentially prove their own worth.Working on the foundation first is hard work, the results are less visible, but the sooner you start, the sooner meaningful results like this implementation will be visible.