River Island’s Journey into Contract Testing

A New Chapter in our QA Excellence

Photo by Markus Winkler on Unsplash

At the forefront of River Island’s commitment to advancing quality assurance, we’ve introduced Pact contract tests to transform how our services interact. This shift has not only strengthened our test coverage but also streamlined our feedback loop, ensuring reliable, efficient data exchange between our core microservices.

By implementing Pact contract testing, we’ve taken a major step forward in reinforcing the reliability and integrity of our microservices architecture.

In this article, I’ll walk you through our journey, using a practical example where data flows between two microservices, each with its own CI/CD pipeline.

Contents

Before Contract Tests
Why Contract Testing?
After Contract Test
Benefits
Conclusion

Lets look at an example micro-services architecture

Scenario: data flow between two micro-services

Service A (Golang):

• Service A written in Golang produces JSON data that follows a specific schema.
• This service is managed by a separate CI/CD pipeline that validates changes to the service.

Service B (Python):

• Service B written in Python consumes the JSON data produced by the Service A.
• This service also has its own CI/CD pipeline that relies on the JSON schema produced by the producer.

for the simplicity, I haven’t put all the AWS architecture involved in this flow.

Before Contract Testing:

In our initial testing strategy, we relied on a combination of:

• Unit Tests: Validating code-level functions.
• Integration Tests: Testing interactions within one product’s architecture.
• End-to-End (E2E) Tests: Ensuring the final outcome by deploying both Services A and B to a test environment and validating the data flow end-to-end.

So our testing pyramid was similar to traditional testing pyramid.

But this traditional approach with E2E tests got below disadvantages:

Slow Feedback Loop:

• End-to-end tests require a fully deployed environment, which is time-consuming to set up.
• Waiting for E2E results can significantly slow down the development cycle.

Hard to Debug:

• When E2E tests fail, identifying the root cause can be a challenge.
• Logs can be voluminous, especially in microservices environments, making it difficult to pinpoint the exact service or change causing the failure.
• Correlation IDs are often needed to trace requests, which can be cumbersome to follow through a distributed system.

Given these limitations, we started exploring alternatives that could offer faster feedback, simpler debugging, and greater reliability in our microservices ecosystem.

Why Contract Testing?

Contract testing stood out as a promising solution, allowing us to validate interactions between services without requiring a full environment deployment. By shifting our focus from testing everything at the system level to validating each service’s expectations independently, we could achieve the following:

• Speed up feedback loops: Each service could be verified independently against a predefined contract, catching issues earlier.
• Simplify Debugging: Failures in contract tests are isolated to specific service interactions, making them easier to trace and resolve.

Evaluating Contract Testing Tools

We needed a tool that could fit seamlessly into our existing workflow, support multiple languages, and provide a reliable testing ecosystem. We conducted a Proof of Concept (PoC) with Pact, Karate, Spring Cloud Contract and Hoverfly.

The criteria for evaluating tools were based on:

• Language Flexibility: We needed a solution that could handle our tech stack diversity (primarily Java, Javascript, Go and Python).
• Support and Ecosystem: Strong community backing and a mature ecosystem were crucial to ensure long-term reliability.
• Ease of Integration: Seamless integration into our existing CI/CD workflows was a must for rapid feedback.

In the end, Pact emerged as the clear winner, offering the right balance of language flexibility, community support, and ease of integration into our CI/CD pipeline.

PACT

Pact is an open-source testing tool that enables contract testing for message-based interactions between services.

Get Srinivas Nali’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Key Features:

• Consumer-Driven Contracts: It supports the concept of consumer-driven contracts, where consumers define the expectations of interactions.
• Verification: It provides verification tools to ensure that the producer and consumer are in sync.

Key Terminology

To understand how contract testing works with Pact, let’s understand the terminology:

• Contract: An agreement or “pact” between the consumer and provider on the structure and format of data exchanged. This serves as the benchmark for testing the interactions between two components.
• Consumer: The service that depends on another service (via HTTP or event-based communication) to complete its operations. In our example, this is Service B.
• Provider: The service that responds to the consumer’s requests by providing data or performing operations. In our case, Service A is the provider.
• Pact Broker: A centralised repository that stores and manages contracts between services. It serves as the single source of truth for all interactions, allowing teams to publish and retrieve contracts, and to track contract versions and compatibility over time.

After Contract Test

With contract testing in place, we no longer rely on fully deployed environments for every change. Here’s how our testing flow improved with Pact

Consumer publishes the expectation and Provider verifies it

Consumer-Driven Contract Creation

Consumer Service B:

• In Service B, we use the pact-python library to create contract tests. These tests define the expected interactions with Service A.
• The tests generate a contract that outlines what data Service B expects from Service A.
• These contracts are published to an internal Pact Broker — a centralized repository that stores all contracts, acting as a source of truth for service interactions.

Jenkins Pipeline Integration:

• In our CI/CD pipeline for Service B, a dedicated stage generates these contracts and automatically publishes them to the Pact Broker.

Provider Verification

Service A Verification:

• In Service A, new verification tests are added using pact-go. These tests pull the expectations (contracts) from the Pact Broker.
• The producer service runs the verification to ensure that it can fulfill all the expectations defined by its consumer. If there’s a mismatch, the CI build fails, providing immediate feedback.

Jenkins Pipeline Integration:

The CI/CD pipeline for Service A includes a verification stage where contracts from the Pact Broker are validated. If any expectation is not met, developers are alerted early in the process.

The New Testing Pyramid

By introducing contract tests with Pact, our testing strategy has evolved:

Benefits

• Fast Feedback: These contract tests run quickly, like unit tests, within the CI/CD pipelines, giving rapid feedback to developers.
• Isolation: Failures are isolated to specific contract mismatches, making debugging much simpler compared to E2E failures.
• Reduced Complexity: No need for full deployments to test interactions, which streamlines the testing workflow and saves infrastructure costs.

Conclusion

With Pact-Go/Python, we introduced a new layer in our test pyramid — contract testing — to validate message interactions between our Lambda services. This new layer strengthens the foundation by catching integration mismatches early.

However, E2E tests still play an irreplaceable role in validating full workflows and overall system behavior. The combination of both approaches gives us faster feedback downstream and higher confidence upstream.

Our hope is that documenting our journey helps others facing similar contract-testing challenges in event-driven microservices.

Reference Links

Pact Docs- https://docs.pact.io/

Happy Testing !