How Platforms Supercharge My Software Engineering Team
How 20% of our engineers made the other 80% faster, safer and more consistent.
We rolled-out a platforms strategy across our software team around 7 years ago, dedicating 20% of our resources to solely focus on building and running platforms. Our platforms now form the bedrock of all products and processes throughout the team, ushering in a digital evolution of technical maturity and scale. In that time we have doubled the size of our team to over 170 people, yet we are operating more efficiently, consistently and securely than ever. There is absolutely no doubt in my mind that platforms have been the key to unlocking this growth!
We are at a moment in time where the stability that our platforms provide is more critical than ever. I am of course alluding to the rise of AI.
In this article I’ll uncover how we structure our teams to get the most out of platforms. We will also follow a change through the SDLC and zoom-in on where some of our most critical platforms slot-in.
Team Changes
The 2 key changes for supporting our platforms strategy were how we structured our teams and how DevOps was injected throughout them.
Team Topologies
We follow the Team Topologies approach for structuring our teams. Team Topologies promotes 4 different types of team and how they should interact with each other. The 2 main team types that we use are:
- Stream-Aligned teams: Cross-functional autonomous teams, which align to building a product or a cross-section of a product.
- Platform teams: Similar to Stream-Aligned teams, but solely focusing on building a platform or suite of platforms.
The key distinction is the focus on products, where the stakeholders are our clients, and the focus on platforms, where the stakeholders are the dev teams building those products.
No edge-of-desk platform building. Only full commitment to platforms.
DevOps
The third team type recommended by Team Topologies is Enabling teams. They are teams of specialists who cut-across our other teams and exist to empower them. This is where DevOps fits into the equation.
Do not fall into the trap of making DevOps a centralised Platform team! DevOps should be a skill which all of your Engineers have. You should have DevOps specialists to champion new processes and tooling, but their primary goal should be nurturing your other teams to learn to use them. This philosophy enables your team to scale and it also empowers them to be much more efficient and autonomous.
Platforms Throughout The SDLC
Platforms support our entire process, all the way from ideation through to run.
Inception
Before we even start planning, we need to understand if there’s already an existing asset in the team which we can use.
Asset Catalogue
Our Asset Catalogue is a web app which holds details about all of our assets. It also has an API to allow automation of information in and out. This is the first port of call for our teams to check if there’s already any assets that they can use and to register their new idea. This system has become surprisingly important as our number of assets has exploded, as well as the amount of data and governance that we need to track against them.
Admin Portal
Our Asset Catalogue is hosted within our team’s proprietary Admin Portal, along with many of our other internal services. One of the key principles in our platforms strategy is to promote self-service for our teams where possible.
All of our asset’s infrastructure environments are managed through our Admin Portal too. The first step after kicking-off a project is for the team to create fresh cloud environments to host their new asset. Making these types of platforms self-service means that teams aren’t waiting around for other teams to complete tickets – they can act with autonomy.
Bootstrapping
Now we’re ready to start scaffolding the codebase for our new asset. We aim to standardise as much as possible, so that our team can easily move around different projects and it will feel familiar. We build frameworks and libraries for cross-cutting concerns, so that our teams can spend as much time as possible on the unique challenges that their assets serve.
Template App
We use a template app to scaffold new project codebases using our team’s design principles and coding standards. The template has a bunch of different parameters which can be configured depending on what is being built e.g. which type of database, async event patterns, web SPA, authentication, namespaces.
Our team also has a developer CLI which can be used to streamline the setup process for the template app, as well as many other activities.
Design System
One of our most important internal frameworks is our frontend design system. We have some excellent developers in the team, but they don’t always have the most artistic instincts! Our design system has a suite of polished frontend components, which make it super quick and easy for our devs to knock-up great looking web apps. The design system is curated by our top frontend engineers and design team, so that anyone from the team can leverage the critical quality that they embed within it.
Build
Our platforms solve most of the common cross-cutting problems that our teams face, but when it comes to their specific app domain, that is for them to take-on themselves.
Agent Framework
Our team has built an AI Agent Framework which is used across most of the SDLC, from writing specs and prototyping, all the way through to implementation and testing. I’m not going to focus too much on the nitty-gritty implementation details, since there’s already enough opinions out there about AI harnesses.
Instead I’ll focus on our guiding principles, which we follow when designing our agents:
- Human-Led: the process must be controlled closely by a human, so they stay close to any design decisions and have a reasonable understanding of what is built. This is critical to reduce both technical debt and knowledge debt, so our teams can continue to be productive with AI over time. There is no place for completely hands-off orchestrated end-to-end processes, such as Gas Town or GSD!
- Process Alignment: agents should reflect real roles and processes followed by our team or widely across software. If the agentic process aligns with proven processes that the team already know, then adoption is much easier.
- Consistent Inputs & Outputs: each agent phase should have a consistent well-understood input and output, generally in markdown or HTML format.
- Multiple Review Layers: each phases outputs should go through a human-in-the-loop review process. We also generally aim to perform multi-model adversarial review to improve quality before any human review.
- If You Don’t Understand It, Learn It: anything that is created must be understood by the person driving the agents. Architecture, design patterns, trade-offs, added libraries etc. If you don’t understand what has been produced, you must go away and learn about it before it can be used.
We currently produce 60–70% of our code using our Agent Framework. These principles are what keep us in control of our skills and codebases when relying on AI over long periods of time. I don’t really have a desire to push this number up much further.
Platform Services
AI makes building quicker than ever. But it is also easy to quickly accumulate slop, which puts the organisation at risk and ultimately slows you down in the long-run. It’s also a huge risk that our teams waste time solving the same problem many times in slightly different ways. This is where our platform services play a huge role across our products.
Get Matt Bentley’s stories in your inbox
Join Medium for free to get updates from this writer.
We have platform services to take care of generic processes which are common across the team. Our platform services are generally managed as standalone microservices (web API and sometimes a web-component UI which can be embedded within a product’s UI), which are shared by our products. Our platform services have dedicated teams, which ensure that they are built and maintained with the same level of quality as our client facing products.
Our platform services include activities such as: securely storing documents with virus scanning, sending notifications across multiple channels, data wrangling, permission management and embedded RAG chat bots.
Test
Nowadays, this is the most important area to have reliable platforms and tooling; more and more code is being written by AI and new security threats spring-up daily. This is also an area which I generally prefer to pay for SaaS products, which are battle tested across many industries.
There are many great options for all of these areas, but these are what have worked very well for us.
SonarQube
We use SonarQube to perform quality scanning on every pull request before it can be merged. SonarQube runs against our continuous integration (CI) build process and syncs to the cloud. It gates code from being accepted until it passes security checks, meets specified test code coverage (80% for us, if you’re interested) and adheres to a suite of code quality, complexity and maintainability profiles.
Even the best developers can get tired and cut-corners – SonarQube keeps us honest and keeps standards high, whether people like it or not!
Snyk
We use Snyk for a variety of critical security testing processes.
Tests Run on Every Code Commit:
- Vulnerability Scanning: all code, libraries and container images are scanned to check for dependencies with known vulnerabilities. Snyk provides a recommended upgrade path to any vulnerable libraries where possible.
- Open-Source Software Licence Scanning: dependencies are checked for any OSS licences which are non-permissive. Non-permissive licenses risk legally forcing your proprietary source code to become open-source or exposing you to copyright litigation.
Tests Run on Every Release Candidate:
- Static Application Security Testing: SAST runs static analysis on the codebase to look for risky or unsafe code.
- Dynamic Application Security Testing: DAST runs dynamic security checks against a deployed instance of our apps, in a similar manner to a Penetration Test.
Snyk classifies findings by risk – any medium or higher issues automatically block promoting the code any further in our processes. The protection that we get from Snyk is invaluable.
Deploy
Whilst I expect all of my Engineers to have some DevOps skills, we still want to make our CI/CD processes as simple and consistent as possible.
DevOps Templates
Running tools like Snyk and SonarQube is absolutely essential, but running them consistently across the team is hard. All of our DevOps pipelines are managed via code as YAML files and we have a templates repository with pre-configured step templates for all tooling that we use, so that pipelines are consistent and correct across our teams.
We also have a repository of infrastructure-as-code modules for our teams to use when deploying their cloud infrastructure. This adds further consistency and ensures that any deployed cloud assets meet our internal policies.
Infrastructure Platform
We use a common multi-tenant infrastructure platform built on top of Kubernetes to host all of our applications. Kubernetes provides dynamic auto-scaling and is multi-tenant by design, so we can lock down security very tightly.
Much of our DevOps team’s time goes towards maintaining and improving the infrastructure platform. The DevOps team provisions locked-down app namespaces for our engineering teams to deploy their assets to and teaches them everything that they need to know to manage their own deployments. This way of working aligns with our general DevOps philosophy and keeps our teams super autonomous.
Kubernetes allows us to run a number of different system components alongside the app namespaces to extend the platform, proving functionality such as: monitoring, GitOps, caching, load balancing, service mesh etc.
We find that this approach gives the perfect balance of consistency and security, whilst also providing a flexible and powerful environment for our app developers to deploy to.
Run
By this stage many security tests have been performed, but the trouble with security is that it’s a moving target. What is considered secure today may not be secure tomorrow. New CVEs are found daily, which can leave apps exposed after they have been deployed.
Istio
We use Istio as a service mesh across our Kubernetes platform. Istio provides a tonne of features, such as Mutual TLS (mTLS) encryption, load balancing, Layer 7 network rules and complex routing patterns.
The first versions of Istio required each Kubernetes pod to run an Envoy sidecar proxy container to route all traffic in and out. Later versions use an Ambient mesh, which only requires a single proxy per node instead. We also use an optional namespace level waypoint proxy to provide additional layer 7 capabilities.
Istio is extremely powerful but it is also super complicated to manage and has taken our team a long time to master. That being said, it is one of the single most important parts of our infrastructure platform. Istio enables us to support end-to-end encryption across our platform using mTLS, without our developers needing to do anything at all. The layer 7 network rules also enable us to completely lock-down outbound traffic from app namespaces by domain.
Sysdig
Sysdig also runs within our infrastructure platform and provides real-time threat detection, vulnerability management, and runtime security. Unlike Snyk which is run once as part of development, Sysdig runs continuous monitoring, so that any new CVEs can be identified against a live app.
We get alerts if there’s any new vulnerabilities identified across our platform or if there’s any suspicious activity. Sysdig can even shut down pods and traffic if particularly dangerous activity is identified, such as a hack or a cryptominer.
Platforms have been the single biggest enabler of our engineering team’s growth and I believe they are becoming even more important as AI accelerates the pace of software development. The goal isn’t to build platforms for the sake of it, but to give our Engineers the foundations they need to move quickly without compromising on quality, security or maintainability. Get this right, and you can scale your team’s output without scaling its complexity at the same rate. For me, that’s the real power of platforms.
