The story of platform engineering

Platform engineering isn’t just a trend, it’s the future of software delivery.

Just a few years ago, this would have been nothing more than a wild claim. Sure, some organizations had started cobbling together their complex tech and tools into something that was easier for developers to use independently of Ops. But “platform engineering” as an established discipline didn’t exist yet.

That all changed in 2019 when we started the first platform engineering meetup in Berlin. At the time, it was a small group of engineers who were frustrated with DevOps adoption at their organizations. While “you build it, you run it” DevOps helped some software teams boost productivity and efficiency, it was extremely painful for everyone else. 

For the vast majority of engineering organizations, the growing complexity of cloud-native technologies and architectures, paired with the expectation for developers to be responsible for it all, was a crippling combination. Developers burnt out trying to ship code fast under increasingly high cognitive load. Ops were drowning in tickets from developers overwhelmed by their DevOps setups. Everyone was frustrated, and productivity suffered. Something needed to change.

The beginnings of platform engineering

In the 2010s, tech giants like Google, Facebook, Airbnb, and Netflix started building platforms to enable developers and Ops to do their jobs more efficiently and effectively. Some smaller organizations experienced similar problems and tried to follow the lead of these tech giants, but they lacked the expertise, resources, and guidance to make their platform dreams a reality. 

In 2018, Evan Bottcher gave a name to the thing a lot of these organizations were building. He defined a digital platform (what we now call an Internal Developer Platform) as “a foundation of self-service APIs, tools, services, knowledge, and support which are arranged as a compelling internal product.” The idea that platforms are and should be treated as a product would become a foundational aspect of platform engineering. 

Matthew Skelton and Manuel Pais then released Team Topologies (also known as the platform engineering bible) in 2019. The book was the first to speak about the platform team and its relationship with the rest of the engineering organization. Skelton and Pais identified DevOps antipatterns wherein team structures made ownership between developers and Ops unclear. They wrote Team Topologies to provide a clearer separation of concerns.

I can still remember how excited our CTO Chris Stephenson was after he met Pais a few weeks later for the first time, at the 2019 O’Reilly Velocity Conference in Berlin. Team Topologies was one of the first of what would become many major milestones for platform engineering.

The platform engineering community

At this point, we had a better idea of what successful platforms and platform teams could look like. But the examples we had were still those from Google and Netflix, models that weren’t attainable for most other organizations. That’s where our small meetup group came in. We wanted a space to share our learnings and get advice from practitioners who’d built platforms at organizations more similar to ours. 

This small but rapidly growing community realized that building platforms was its own discipline, complementary to but still different from DevOps. The term platform engineering was defined as “the discipline of designing and building toolchains and workflows that enable self-service capabilities in the cloud-native era.” Platform engineers build Internal Developer Platforms, which consist of “many different tech and tools, glued together in a way that lowers the cognitive load on developers without abstracting away context and underlying technologies.” Internal Developer Platforms solved the biggest challenges facing software engineering organizations by following a platform as a product approach. 

Platform engineering resonated with a lot of folks. It gave a name to and a shared understanding of something that already existed. When the Berlin meetup moved online during the pandemic, more and more platform engineers in cities around the world created their own chapters. 

New joiners were excited to have found a community of like-minded people. They were back ends, infrastructure, ops, and DevOps engineers who felt building Internal Developer Platforms was more meaningful work than what they had been doing before. Many were eager to learn more about how to follow a platform as a product approach and deliver value to their organizations. Other community members realized that platform engineering was what they had been doing all along, they just didn’t have a term for it. The community provided a space where practitioners could better access resources and share what they learned.

The Platform Engineering Slack channel was started in 2021 so that folks from different chapters and parts of the world could collaborate more directly. One year later, platformengineering.org was started to showcase the insights, discussions, and resources from members of the community.

More and more folks in the community raised the idea of creating a platform engineering conference. And so PlatformCon was born. A conference by and for platform engineers hadn’t been done before. And with our community spread out across so many different countries and continents, we had to try our hand at a new virtual format. 

One thing is for certain: the success of PlatformCon 2022 wouldn’t have been possible without the community’s enthusiasm for sharing their insights and stories. We received 78 talks from experts like then-Puppet field CTO Nigel Kersten, Cloud Strategy author Gregor Hohpe, Team Topologies co-author Manuel Pais, and OpenCredo’s Nicki Watt. PlatformCon talks covered the most foundational topics in platform engineering: Syntasso’s Paula Kennedy explained why cognitive load is a key problem for platform teams to solve. Platform teams can alleviate the cognitive load by following a platform as a product approach and finding the right level of abstraction. Speakers brought their unique perspectives and experiences to over 7k attendees. They swapped stories and resources in the community Slack channel and shared their key takeaways from the conference on social media. 

Platform engineering takes off

PlatformCon proved to the software engineering world that platform engineering was a big deal. But we had no idea just how big it would get.

In August 2022, Gartner put platform engineering on their Hype Cycle. A month later, DevOps is dead, embrace platform engineering went viral for its controversial headline. “Developers don’t want to do operations anymore, and that’s a bad sign for DevOps, at least according to this article by Scott Carey and this Twitter thread by Sid Palas” wrote the article’s author, Aeris Stewart. “When developers in teams don’t agree on the extent to which they should, or can, do operations tasks, forcing everyone to do DevOps in a one-size-fits-all way has disastrous consequences.”

Stewart explained why organizations trying to strike a better balance between developer self-service and mitigating cognitive load through abstraction increasingly chose to build Internal Developer Platforms: “Platform engineering uses a product approach to enable the right amount of developer self-service and find the right level of abstraction for individual organizations and teams.” The story resonated with developers and Ops who’d felt the brunt of DevOps burnout for years. Thousands of readers flooded into the platform engineering community Slack channel, excited to learn more about this emerging discipline and how to make it work for their organizations.

In October 2022, Gartner named platform engineering as one of the Top Strategic Technology Trends for 2023. In their research, they shared that platforms at organizations like Nike and Politiet enabled organizations to respond more quickly to change, drive faster time to market, and innovate more rapidly.

Gartner’s report was the first of many to focus specifically on platform engineering. In November 2022, Humanitec published the first-ever State of Platform Engineering Report. The report detailed the emergence of platform engineering as a popular discipline, provided a consolidated view of the platform tooling landscape, compared platform engineering and DevOps salaries, and noted community growth opportunities. 

The evolution of platform engineering

Platform engineering was both gaining hype and growing in maturity. Nowhere is this more apparent than with the return of PlatformCon this June. Not only was PlatformCon 2023 bigger (22k+ attendees and 169 talks), but it also provided the much-needed resources that were previously lacking.

One big example is the first-ever Internal Developer Platform reference architecture developed by the team at McKinsey. This is a game-changer for organizations starting their platform journey. PlatformCon 2023 also featured concrete examples of how platform teams can build scalable golden paths. Platform teams don’t have to start from scratch anymore, but instead use a reference architecture informed by hundreds of real-life platform setups — and adjust it to their organization’s specific needs.

We also saw more detailed discussions about how to understand, measure, and communicate the impact of platform engineering initiatives. After all, platform teams need to secure and maintain buy-in from executives, managers, and users to sustain their platform engineering initiatives. 

Gartner’s Manjunath Bhat demonstrated how platform teams could use Gartner’s Enterprise Value Equation to understand and communicate their Internal Developer Platform’s impact on different stakeholders’ priorities. His approach translated high-level concepts into a list of action items any platform team can follow.

Some talks built upon what was previously discussed at PlatformCon 2022. Nicki Watt revisited the platform as a product approach and discussed why many organizations still struggle with following this best practice. For example, quality user research is an established prerequisite for successful platforms. Despite this, many platform teams instead build based on the assumption that they already understand the needs of other developers.

In April 2023, Syntasso drafted a first version of the Platform Maturity Model and donated it to the Cloud Computing Foundation (CNCF) for further discussion. The model was designed to help organizations evaluate their capability for continuous improvement in platform engineering. Mature platform organizations enjoy a greater likelihood of incidents or errors resulting in improvements to the platform. According to Syntasso’s model, engineering organizations can become more mature as they align with established best practices in the discipline (such as following a platform as a product mindset) and cultivate the right organizational culture.

We asked members of the platform engineering community to respond to a short survey based on a draft of the Platform Maturity Model and our own observations from the community. Our goal was to understand the extent to which organizations follow platform engineering best practices. We were surprised to find that, while many of these best practices had been established and discussed for years, most organizations aren’t following them.

As it evolves, the Platform Maturity Model can be a helpful tool for organizations to pinpoint potential areas of improvement. Through community surveys like ours, we can also use the model to understand what resources are needed the most by practitioners, and collaborate with experts to meet that need.

Looking ahead

The popularity of platform engineering has skyrocketed in recent years, and so has the wealth of knowledge shared by community members. It’s an exciting time to be part of the platform engineering movement, and I can’t wait to see where it goes next. 

Learn more from the largest global network of platform engineers by visiting the Platform Engineering YouTube channel.

‍