Gartner predicts that by 2026 80% of engineering organizations will have an Internal Developer Platform (IDP) to “ultimately solve the central problem of cooperation between software developers and operators”. Platform engineering is everywhere you look at events like KubeCon and AWS re:invent. And everyone in the industry is thinking about how their company can make the most of it.
Yet, so many platform engineering initiatives are failing. They’re just not taking off. They get stuck in evaluation, unable to get all the different stakeholders on board, and as business cases they lack the strength needed to get executive buy-in. Or they simply end up in platform engineering hell.
The reality is, as platform engineering and IDPs are still relatively new concepts, the majority of teams don’t have the know-how to design, build, and roll them out successfully.
Interestingly, most people reading the above will immediately think the choice of the tooling stack or the technical design would be the main challenge. But that’s not true. Platform initiatives almost never fail because of technical reasons e.g. you picked this tool over that other one. Plus, there are established blueprints like the reference architectures shown at PlatformCon23 to help guide you in your tooling decisions.
IDP rollouts fail because of two things teams get wrong: culture and process.
We spoke about cultural failures before (see the platform engineering hell piece linked above). Reasons include:
- You can’t speak C-level lingo to get exec support.
- You don’t listen to devs who want a platform that doesn’t abstract too much.
- You don’t figure out the right separations of concerns between platform teams and existing infra ones.
But the more I speak to platform engineering teams, the more I realize these cultural aspects are not even the primary culprits. The real project killers are process and speed (or lack thereof).
Stop procrastinating: think Minimum Viable Platform (MVP) first
The problem is that most platform teams simply don’t have the right processes (and often the expertise) to:
- Get key stakeholders onboard with the IDP initiative. And…
- Prove value quickly enough to said stakeholders so their initiative doesn’t lose momentum.
If you spend 12 months designing your platform to try and accommodate every possible use case and make everyone happy at the same time, you will fail. Guaranteed.
Instead what you need to do is to identify who the key stakeholders are that will allow you to get the initiative off the ground by showing them value fast. Depending on your org structure these can be your architects, C-level, or app dev leadership. The point here is you don’t need to please every possible dev team in your org or all your security teams, plus your entire technical leadership at the same time to win.
Pick the right battles to fight, in the right order. Carefully select your internal champions and prove your platform initiative will deliver the highest ROI your company can achieve right now.
The way you do this is by shipping an MVP in weeks, instead of months or years. Find a pioneering team to build it for. Get the key stakeholders excited, get feedback, and find more pioneers and sponsors. And before you know it you’ll have built real momentum across your entire engineering org to really get this done the right way.
How you get to your Minimum Viable Product
The concept of an MVP (as in Minimum Viable Product) is of course very established and rooted in the lean methodology of launching with just the essentials to prove your concept.
Manuel Pais, co-author of Team Topologies, has been talking about the Platform as a Product approach for years now. And Jon Skarpeteig, Global Platform Lead at Signicat, recently gave a great talk on why a startup mindset is key to delivering an MVP and successful platform.
But let’s get more pragmatic. How do we define a Minimum Viable Platform? I recommend focusing on a simple HelloWorld app. This should have as much in common as possible with your real apps, but without all the custom and often unnecessarily complicated edge cases. The HelloWorld app should also use the common infrastructure your apps normally consume.
Keep in mind the M stands for minimum. This means you want to ensure the following is always true about your MVP and HelloWorld app:
- Representative: The MVP should have representative basic resources and components that are common across the technical estate. Generally, this includes clusters, containers, version control, a single database type, basic DNS, and basic CI/CD.
- Repeatable: The basic skeleton created in the MVP process should be able to be used as a “quick start” for other applications and teams within an organization. The MVP is the foundation from which the full-scale platform is built.
- Iterative: MVPs are focused on the future. The context of creating this version of the platform is with growth and iteration in mind. Remember that commitment is required to scale.
- Innovative: The teams that work on these MVPs are pioneers and drive new ideas forward. MVPs should provide inspiration for contributors to learn new things and engage with new technologies.
You should also always keep in mind what you should NOT optimize in your MVP phase:
- High compliance scenarios: You can think about security and high-compliance requirements, but an MVP shouldn’t have to comply with high compliance standards.
- Advanced architectures: Service meshes, multi-cluster deployments, failover, customized portal integrations, advanced CI/CD (Blue/Green) etc. do not need to be covered by an MVP.
- Advanced resource configuration: You should pick one common resource type to begin with and not try to cover every possible developer request.
- Advanced observability: Integration of advanced components and highly complex observability services is something you can build on top later.
.avif)
Once you follow these principles and ensure you’re building an MVP, the next thing to focus on is finding the right level of abstraction for your platform.
If you don’t abstract enough complexity away from developers, you haven’t built a platform. At the same time, if you remove too much context, devs will push back on you. Listen to them, follow a Platform as a Product approach, and iterate.
Many platform teams also confuse abstraction with just offering a developer portal UI. That’s like building the front door to your house before you have the walls in place. Developers tend to respond much better to code-first interfaces (e.g. Score) that keep them in the git-based workflows they’re familiar with.
Combine the workload spec with a Platform Orchestrator (check Thoughtworks tech radar for inspiration) to get to an MVP that’s truly enterprise ready.
Finally, set yourself an ambitious goal and aim to show value to all key stakeholders within two weeks. This might sound too ambitious, but I’ve seen it happen many times in teams that have followed all of the above.
Next steps: iterate and expand
After two weeks, you'll have a Minimum Viable Platform that:
1. Proves to you and everyone else that matters, that this is the best thing you should be investing in. And…
2. it is ready to grow to the next phase. This is the foundation upon which your full-scale IDP will be built, fostering a culture of innovation and continuous development.
Now go use this MVP to get buy-in from your selected pioneering group. The group of stakeholders that will become your lighthouse team in the organization (think the same people that first adopted Kubernetes, Terraform, etc.)
Gather their feedback and iterate. As soon as this group sees clear value in your MVP, it’ll be much easier to get funding for a larger platform engineering initiative and continue to expand to a large-scale enterprise IDP.
Platform engineering is here to stay. But it’s still far from being a guaranteed success if you don’t have the right processes in place. I hope this article can help. Join the Platform Engineering Slack to exchange best practices with other practitioners. And sign up for PlatformCon to learn from the leaders in the space.
-min.webp)
