Platform engineering: Best practices in “Vendor engineering”

One of the less frequently discussed topics within the domain of Platform Engineering (PE) is the concept of “vendor engineering”. Charity Majors loosely defined the term a few years back as “writing libs, modules, use cases, docs, etc to standardize the use of the third-party software across the org”. 

Many Platform Engineering teams already practice the art of proactively and pragmatically creating abstractions around specific platform components as a way to get future flexibility around replacement or extensibility decisions. "Vendor Engineering" suggests enhancing this approach by not only focusing on technical decisions but also ensuring that these decisions contribute to an optimized experience for the consumers of the Platform-as-a-Product (PaaP). This should be included as part of the success criteria for these efforts.

It's quite common for platform teams to face headcount funding limitations. During roadmap planning, these groups often discuss how to allocate their engineering team's time most effectively to achieve the best outcomes for the right user personas within their total addressable market (TAM). They also consider how to accomplish more with the same team size, or how to meet ambitious goals, targets, or requirements simultaneously or more quickly as a way to delight their user base.

These conversations typically progress to more challenging meta topics, such as: "Is this a differentiating capability for our team, business, or platform? If so, do we have sufficient executive support, funding, and prioritization to develop it, and would it be competitive in the marketplace? If not, should we instead invest in a Commercial Off-The-Shelf (COTS) solution that we need to integrate into our environment as part of our Platform-as-a-Product (PaaP)?"

Key Strategies

Since most enterprises typically allocate separate budgets for software, technology, innovation, and headcount, this is often good news for platform teams. As a result, many platform teams end up evaluating third-party (3P) vendor solutions as part of the implementation and delivery of their platform initiatives.

So, what strategies can a platform team continuously consider to address this problem space while still accounting for core platform initiative success metrics such as market adoption rate, platform onboarding time, net promoter score (NPS), and key customer metrics?

• Create, publish, and maintain a Platform Manifesto that details all the components comprising the PaaP. This "living document" should outline each component's lifecycle dimensions, including the current state, target future state, versioning, end-of-life/replacement plans, and the associated timeframes and reasons behind those decisions. The PE team, utilizing its product arm, will update the content quarterly and share it during "town hall" style meetings and product roadmap updates. A critical feedback loop will capture end-user sentiment toward the content. For instance, a different third-party (3P) tool or solution might offer better developer ergonomics, lower costs, and enhanced security when compared to the current one.
• Design and build the core internals of the platform with abstractions in mind, ensuring they are not leaky. While information about the platform components should be publicly visible and well-known internally, users should not need in-depth expertise to achieve their day-to-day outcomes or jobs-to-be-done. For example, implementing a high-level, language-specific telemetry library would enable platform users to simply send payloads to a set of API endpoints. These endpoints would then store and present the information to the user and their desired tools. This approach completely removes the associated cognitive load of design and implementation from the end user and makes it easier to get started and drive value out of the platform.
• Write and publish "How-To" style guides that either establish a clear best practice “golden path” or enhance the visibility and usability of an existing one. Utilize open source frameworks and toolkits to create documentation, templates, and compositions for the benefit of platform users.
• Engage existing platform users who have achieved successful outcomes to share their experiences with peers, highlighting how they used the PaaP to achieve business goals. This approach mirrors the effective patterns used by many public cloud providers, which can be easily adopted and implemented internally.
• Publish a collection of product use cases from the aforementioned groups to share with non-technical audiences. This will help raise awareness and educate the ultimate decision-makers about how the platform can enhance the value of engineering team time investment within the organization.

In addition, here are some 3P product dimensions that teams may consider during evaluation. The list is not in any particular order:

• Business lifecycle stage - more mature businesses typically present a lower risk profile to customers when compared to Series B or Series C startups.
• Size and composition of the R&D team - explore factors such as the ratio between engineering and product roles, geographical distribution, team average tenure, and retention versus attrition metrics.
• Product release cycle - it's beneficial to delve into detailed aspects and nuances of the product’s SDLC, such as DORA metrics, the number of tracked bugs, median bug resolution time, bug reporting and detection methods, testing procedures, governance, security measures, known product defects, and feature maturity.
• Cost profile and unit economics - analyzing the profit margins for similar products in the marketplace as a way to optimize pricing. Marketplace observers can provide valuable insights to assist with this.
• Quality of product support and engagement model - items of interest are the thoroughness and maturity of the escalation matrix, performance and availability-based SLAs, and how SLA breaches are handled by both parties, including follow-up actions.
• Size, breadth and depth of the end-user community and ecosystem.
• Technology fit - stack compatibility, ease of integration with existing platform components, alongside security considerations.
• Bi-annual "sliding window" roadmap review by the vendor, covering the next 6-12 months.
• Customer references from enterprises that fit one’s organization in terms of business vertical, headcount, size of implementation, breadth, depth and complexity of production environment workloads, and use cases.

Most popular marketplace observers offer annual guides that teams can use to fast-track getting up to speed on ~80% of the topics above. It's a good practice to leverage these guides to narrow down a list of 2-3 potential finalists, then spend more time in-depth with each to gauge and confirm the quality of a future fit and partnership.

Conclusion

By thoughtfully evaluating third-party solutions and integrating them into the Platform-as-a-Product (PaaP), teams can optimize both technical outcomes and user experiences. This requires a balanced approach, considering not only the technical fit but also business alignment, support quality, and long-term partnership potential. By maintaining clear documentation, engaging with platform users, and continuously refining strategies, platform teams can ensure they are making informed decisions that contribute to the success and scalability of their initiatives.

This robust “vendor engineering” playbook will hopefully set you, team, the PaaP, and in turn, the rest of the organization, up for success. Good luck!