The Magic of Platforms

In recent years, one thing has become clear: Cognitive (over)load is causing poor developer experience. 

There are a few clear culprits for cognitive load: the growing complexity of the cloud-native setup, shift left, and around 2,000 tools to choose and learn from, just to name a few.

In response to these challenges, many engineering organizations turned to platform engineering. They build Internal Developer Platforms (IDPs) that help developers navigate increasingly complex toolchains. 

Building an IDP is one of the most important ways a business can streamline application management and deployment while also ensuring security and scalability. An IDP enables developers to focus on quickly shipping new features to market. This is a critical competitive differentiator for many businesses. 

However, many enterprises struggle in their platform engineering initiatives. Developing and financing your own IDP is far more complex than it appears. Simply relabeling existing shared services as a "platform" won't bring the desired results. 

In his forthcoming book, Platform Strategy: Accelerating Innovation Through Harmonization and Reuse, Gregor Hohpe offers readers mental models to help them make the necessary decisions about building their IDPs. The book is part of his "Architect Elevator" series – geared toward the IT leaders and architects responsible for steering major strategic initiatives in their enterprises.

In his keynote at PlatformCon 2022, Gregor shared some groundbreaking insights on the magic of platforms and provided valuable advice for enterprises starting their platform engineering journey. He focused on the critical decisions and trade-offs one should consider when constructing an IDP.

Platform Architecture's Undifferentiated Heavy-lifting

A platform architecture is composed of two types of components: those that remain fixed over the life of the platform and those that are allowed to change or vary. The fixed component becomes a defining set of design rules or constraints governing relationships among all system parts, often simply because it outlives other system components. 

A key feature of platform architectures is that they are flexible and evolvable, meaning they can adjust to unexpected changes in the external environment. A platform architecture separates a system into stable central components and peripheral components that vary over time.

Partitioning systems into core and variable components allows companies to reuse the core while innovating. This eliminates the need to rebuild an entirely new system from scratch whenever an organization wants to release a new product or adjust its offerings. With this method, the platform architecture becomes malleable: easy to change without losing design continuity or identity.

There are many great examples where platform architectures have transformed entire industries. One of those industries is the automotive industry with its undifferentiated heavy lifting in engines, transmission, emissions, regulations, antilock brakes, and other things that make cars efficient and safe that the customer doesn't see. The customer sees the color of the interior, how nicely the doors close, and how comfortable the seats are.

The same principles apply to software and IT. It's now possible to build software platforms that different organizations can use to power their applications. 

Platforms get things moving by supplying the hefty foundational investment – freeing the engineers of the burden of repeating undifferentiated boiler-plate tasks ad infinitum. With this head start, they can focus on the differentiated elements that actually demand the greatest degree of visionary problem-solving – the key features and functionalities that make applications valuable and useful.

Platforms also help enforce standards. Standards make products more useful and often help drive innovation rather than preventing it. For example, http allows any web browser to connect to any web server. While some might argue that standards stifle creativity, Gregor disagrees. Instead, he argues, the standards platforms enforce enable developers to be creative where it really matters. He compares platforms to A4 paper, which is highly standardized. One can use A4 paper to draw or paint, can tear it apart or tape some together to change the size of the canvas, or make an origami crane. However, “nobody has to argue about how big the paper will be [or] how big the envelope.”

Platform engineers' primary responsibility is to construct a self-serve application platform that standardizes operations among tools and teams with zero impact on developers and an offering that developers will not find burdensome.

Developers only want to concern themselves with the app code and rapid deployment while being able to ignore the infrastructure altogether. They also need the ability to see their applications' architecture, dependencies, and communications, all while receiving support for apps without DevOps or platform team interference.

Mechanisms That Make Platforms Work

Gregor explains the three mechanisms that make platforms work: low friction, transparency, and shared responsibility.

1. Platforms need to have low friction - You cannot force anyone to get on your platform. They'll probably find other ways to get their work done if you try.
2. Good platforms need to be transparent - Platforms are not black boxes. People depend on the platform, so they need some insight into what's happening inside it. And if there's a problem, they need to be able to tell if the problem is coming from them or the base platform.
3. Good platforms are a shared responsibility - Platforms cannot solve all problems. Problem-solving is a shared responsibility between the people who use the platform and the people who built it. For instance, while it's possible to do great things with AWS, the platform cannot fix an unsecured, brittle, non-scaling monolithic application if you build one. It would be best if you developed applications that take advantage of the platform.

So, how do you build a platform that enables your team to be more efficient, that's low friction, transparent, and that's a shared responsibility? The best approach is to evolve the platform:

• Start with some useful pieces.
• Observe what people need.
• Implement those things into the platform over time.

Platform Architecture Decisions: Make Developers’ Lives Easier

A platform architecture provides both future evolvability and variety in the present, but only if the architect knows which components to keep stable and which can vary. The platform architecture should have stable interfaces that are versatile enough to adapt as new linkages arise. Therefore, the platform engineering team must make a series of decisions when building IDPs.

The first decision revolves around making the life of the developer easier. Developers need to be better off with your platform than without. Therefore, when building a platform, you must ask yourself what mechanisms you can use to achieve this. What are you after? Are you after productivity, better collaboration, or improved compliance? Or do you want to minimize mistakes? With answers to these questions, you can leverage the mechanisms you have in hand to create the platform.

So, how do you make the life of a developer easier when creating an IDP? First, Gregor recommends reducing the initial learning curve for using the platform.

If the initial learning curve is high, it can discourage developers from ever using the platform. Gregor says you should make it as smooth as possible so people can get started quickly and then introduce more complexity slowly over time. 

This design strategy requires foresight. At the very beginning of building a platform, you need to have a clear view of how it will make users' lives easier. You’ll also need to think about what shape its learning curve will take over its lifetime.

Building on Top of a Base Platform: Sinking Vs. Floating Platforms

Base platforms aren't static; they grow over time. So, Gregor asks: "What will you do with your platform when the base platform grows?"

He offers two options:

1. With the first option, you would keep your platform the same even as the base platform improves. However, this would mean that you would end up with duplicated capabilities since the base platform will have them, too, over time. This is called a sinking platform.
2. The second option is to build a floating platform: when the base platform gains certain capabilities, you drop them on your end. This way, instead of using your efforts to maintain those capabilities, you can use them to innovate further on top and build new things.

If you decide to build a floating platform, you must clarify this to all project stakeholders from the beginning; they need to know that you'll discontinue some capabilities on your platform when the base platform gains the same capabilities.

The beauty of a floating platform is that you can invest your resources in other parts of the platform or other products and not worry about maintaining your dropped capabilities. The danger of a floating platform is that it can feel like you're not doing your job if the platform doesn't have as many capabilities as the base platform. You'll need to be able to explain to stakeholders that this is by design and not a sign that the platform is lagging.

Sinking platforms, however, can feel like you're doing a great job because the platform always has more capabilities than the base platform. But the danger here is that you'll eventually hit a point where the duplication of capabilities is so high that it's no longer feasible to maintain the platform. The trade-off here is between short-term and long-term feasibility, and the takeaway is that you must decide what to do with your platform when the base platform grows.

Today’s companies depend on containers, infrastructure-as-code (IaC) tooling, Kubernetes, continuous delivery pipelines, and a massive range of development strategies to stay competitive. As such, platform engineering teams face a daunting task: acting as an organization's internal cloud provider to deliver entire technology ecosystems effectively.

If your IDP is subpar (or simply fails to impress development teams) you run the risk of incredible amounts of time and effort going to waste, missing your management goals, and developers rejecting the platform instead of uniformly adopting it. The tips Gregor has shared in this talk are practical steps you can take to ensure your platform engineering journey is smooth and that it achieves its goal: improving the developer experience.