Adding Quality Gates Around the Code You Change
A practical way to improve a large Python codebase without making every pull request a cleanup project
We expanded the checks in our main Python service’s GitLab pipeline. The rollout included pytest, an 80% coverage threshold on changed lines, formatting and lint checks, static typing, security checks and deeper code analysis.
That list can sound like a shopping receipt for developer tools. The useful part is how the checks relate to the change a developer is trying to ship.
Here is the approach behind the rollout, along with the questions a team should answer before calling any of it a reliable merge gate.
Give each check a specific job
Our announced setup included Black, isort and flake8 for formatting, import ordering and linting; mypy for static types; Bandit for Python security checks; and Django’s deployment and security checks.
We also included SonarQube analysis after tests passed and Gitleaks scanning for secrets in changed commits.
These tools answer different questions. A formatting result says little about authorization logic. A type check can catch an incompatible value without proving that a payment amount is correct. A security scanner can identify suspicious patterns while missing a flaw that depends on business context.
That gives the pipeline a set of complementary signals. It also gives maintainers a responsibility: explain what each signal means and what action follows a failure.
“The pipeline is red” is a status. “This changed function violates its declared return type” is feedback someone can use.
Make the changed lines visible
The coverage threshold in our rollout was 80% of changed lines. It was not a claim that the entire repository had 80% coverage.
Consider a hypothetical patch that changes twenty executable lines. If sixteen are covered, that patch reaches an 80% changed-line threshold. The example says nothing about untouched modules or whether the assertions are meaningful.
The attraction of this approach is scope. A developer adding a small feature can improve its tests without having to repair every historical coverage gap in the repository during the same review.
For teams implementing this pattern, diff-cover is one available tool. It compares coverage information with a Git diff and supports a failure threshold. Mentioning it here is an implementation option, not a claim about which package powered our historical setup.
The baseline deserves attention. A comparison against the wrong branch can produce a persuasive percentage for the wrong change. Missing coverage reports, shallow history and an empty diff also need explicit behavior.
Get Nithin Mohan’s stories in your inbox
Join Medium for free to get updates from this writer.
An unexplained green check can be surprisingly unhelpful.
Separate reports from enforcement
There are several steps between installing a checker and enforcing a policy:
- The intended job must run for the relevant change.
- A failed check must produce a failed job result.
- Pipeline configuration must preserve that failure.
- The repository’s merge rules must require the appropriate successful result.
Our rollout announcement explicitly described mypy failures as merge-blocking. It also described a changed-line coverage threshold. Those statements alone should not be stretched into a claim that every check always blocked every merge.
This is worth testing directly. In a disposable branch, introduce a known violation and walk through the result. Then check the converse: a valid change should pass without requiring someone to understand the entire CI configuration.
This verification is part of building the gate. The YAML file cannot grade its own homework.
Control the scope without creating blind spots
Several checks in our rollout focused on changed files or commits. Targeting the change makes feedback more relevant, but different tools need different context.
A formatter can inspect the changed Python files. A type error may involve an imported definition elsewhere. A security issue may span an API handler and a helper that the patch never edits.
The right boundary follows the risk being checked. Keep whole-project or scheduled analysis where a narrow diff view would miss important relationships. Document exclusions so that an exception does not become permanent invisibility.
The same care applies to timing. Our rollout had a parallel quality stage and deeper analysis after tests. Other repositories may benefit from a different arrangement. Fast feedback and understandable dependencies should guide the design.
Make failure repairable
A practical gate needs a reproducible local command, aligned tool versions, useful output and an owner who can explain exceptions.
When a rule is too noisy, tune its scope and investigate the false positives. When it identifies a real problem, keep the signal visible. Silencing the whole check is usually a much larger decision than fixing one finding.
The work continues after the first rollout. Branch rules change. Dependencies move. A new job may run successfully while being absent from the merge requirements.
For a large codebase, changed-code gates offer a manageable starting point: make the current contribution easier to review, preserve broader checks where they matter, and verify that enforcement behaves as intended.
A useful quality gate makes the next action clear. That is a better starting point than simply adding another badge.
