Contract Testing

In distributed systems, interaction between services is important. Different channels and structures can be used for interaction; SOAP, REST, GraphQL, gRPC, AMQP etc. In order for the relevant service to work properly, the interaction must be verified before the applications are deployed. We try to test the correctness and consistency of this interaction with unit and integration tests. This verification can be done by mocking the interaction in unit tests or service virtualization or end-to-end tests by running all necessary services. Mocking in unit tests is based on assumption and cannot guarantee the correctness of the interaction. Service virtualization generally ensures the independence of the tests rather than the correctness of the interaction. In end-to-end testing, it is difficult to run all the necessary services and keep the structure and tests up to date. In addition, end-to-end testing can be a very late stage for error detection.

Let’s go through an example to understand the problem better. We have a service called ServiceA and it provides an API called ApiA. ServiceB and ServiceC make requests to ApiA. How can the following situations be verified?

• When ServiceA makes a change in ApiA, will it cause errors in ServiceB and ServiceC?
• ServiceB used documentation of ApiA to interaction, but was it implemented correctly?
• ServiceC made a change in accessing ApiA. Will the change cause an error?

How can we check similar scenarios before the production environment, without running all services for end-to-end testing or without the need for manual testing?

In summary, we need to verify the interactions of services independently, automatically and at the earliest possible point. We can meet this need with contract testing.

What is Contract Testing?

Contract testing is a methodology that checks the interaction and compatibility of two different services with each other. It is based on the contract created by interacting services. What is meant by the contract here is the model, not the data.

The service provider prepares its own contract. The service consumer prepares the contract that will include its own needs. The service provider or the service consumer runs their tests using these contracts created during their tests. If there is an incompatible situation, the relevant service cannot be built.

Let’s try to explain with an example;

• ServiceA provides ApiA. Let’s assume it takes id as a parameter and returns id, firstname, lastname, address and age. It prepared its own contract accordingly: ApiA(id -> {id, first name, last name, address, age})
• ServiceB consumes ApiA and uses only the first name, last name and age fields from the returned response. It prepared its contract according to this model: ApiA(id -> {id, first name, last name, age})

ServiceA and ServiceB use these contracts in their tests. ServiceB checks that there is no problem between ServiceA’s contract and its own contract. ServiceA also tests ServiceB’s contract to confirm that it does not affect its use.

• ServiceA modifies the model by adding a field called country. During its tests, it also tests ServiceB’s contract, and since there is nothing contrary to the model it uses, the tests are completed successfully. The same applies to ServiceB’s tests.
• ServiceA made a change and changed the age field in the model to date of birth. During its tests, it also tests ServiceB’s contract, and the tests fail because a situation contrary to the model it uses occurs. And built is broken.
• ServiceB added a field named province to its ApiA contract based on verbal agreement with ServiceA. Since ServiceA has not made this change yet, ServiceB also tests ServiceA’s contract during its tests, and the tests fail because a situation contrary to the model it uses occurs. And built is broken.

As can be seen from the examples, the consistency and health of the interactions of interdependent services through contracts were ensured independently and automatically during the build phase.

Solution 1: Spring Cloud Contract

https://spring.io/projects/spring-cloud-contract

Get Ahmet Tuncer’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

How it works

https://docs.spring.io/spring-cloud-contract/reference/getting-started/three-second-tour.html

• Contracts are created under the src/test/resources/contracts directory. Contracts can be created in different formats such as groovy, java, yaml. Groovy format offers a richer and more flexible structure.

import org.springframework.cloud.contract.spec.Contract
 
Contract.make {
    description "should return person by id=1"
 
    request {
        url "/person/1"
        method GET()
    }
 
    response {
        status OK()
        headers {
            contentType applicationJson()
        }
        body (
            id: 1,
            name: "foo",
            surname: "bee"
        )
    }
}

• Spring-cloud-contract-maven-plugin, added to the project as a dependency, automatically creates test classes according to the prepared contracts.

...
@SuppressWarnings("rawtypes")
public class FindPersonByIdTest extends BaseTest {
 
    @Test
    public void validate_findPersonById() throws Exception {
        // given:
            MockMvcRequestSpecification request = given();
 
 
        // when:
            ResponseOptions response = given().spec(request)
                    .get("/person/1");
 
        // then:
            assertThat(response.statusCode()).isEqualTo(200);
            assertThat(response.header("Content-Type")).matches("application/json.*");
 
        // and:
            DocumentContext parsedJson = JsonPath.parse(response.getBody().asString());
            assertThatJson(parsedJson).field("['id']").isEqualTo(1);
            assertThatJson(parsedJson).field("['name']").isEqualTo("foo");
            assertThatJson(parsedJson).field("['surname']").isEqualTo("bee");
    }
}

• Contracts are verified with the spring-cloud-starter-contract-verifier dependency added to the project.
• During the build phase, stubs.jar containing the contracts along with the application package is created.
• spring-cloud-starter-contract-stub-runner dependency is added to the project on both side (provider and consumer). This dependency (StubRunner) gets the stubs added by configuration and runs a service that exposes them as API from localhost using the contracts in the stubs.
• In tests, requests to that service are directed to the local service run by StubRunner. Thus, there is no need to go to the original service or mocked one during tests. If there is a mismatch with the models specified in the contracts, the tests will fail.

All services may be asked to create contracts for the service they provide or consume, and both parties may be asked to confirm these contracts. In this context, Spring Cloud Contract also supports the centralized stub structure. Contracts can be kept on git or in repositories such as Nexus.

It can also be made compatible with other platforms; https://spring.io/blog/2018/02/13/spring-cloud-contract-in-a-polyglot-world/

Solution 2: Pact

https://pact.io/

How it works

https://docs.pact.io/getting_started/how_pact_works

It has support for multiple platforms (Java, JS, Ruby, Golang, Android, .NET).

• Tests in applications are developed by using pact’s libraries.

...

@Pact(consumer = "ProductCatalogue")
public RequestResponsePact singleProduct(PactDslWithProvider builder) {
    return builder
        .given("product with ID 10 exists", "id", 10)
        .uponReceiving("get product with ID 10")
        .path("/products/10")
        .willRespondWith()
        .status(200)
        .body(
            new PactDslJsonBody()
                .integerType("id", 10L)
                .stringType("name", "28 Degrees")
                .stringType("type", "CREDIT_CARD")
        )
    .toPact();
}

@Test
@PactTestFor(pactMethod = "singleProduct", port="9999")
void testSingleProduct(MockServer mockServer) throws IOException {
    Product product = productServiceClient.getProductById(10L);
    assertThat(product, is(equalTo(new Product(10L, "28 Degrees", "CREDIT_CARD"))));
}

...

• With this library, contracts (pact file in json format) are generated automatically during the test phase.

{
    "consumer": {
        "name": "ProductCatalogue"
    },
    "interactions": [
        {
            "description": "get product with ID 10",
            "providerState": "product with ID 10 exists",
            "request": {
                "method": "GET",
                "path": "/products/10"
            },
            "response": {
                "body": {
                    "id": "10",
                    "name": "28 Degrees",
                    "type": "CREDIT_CARD",
                },
                "headers": {
                    "Content-Type": "application/json; charset=utf-8"
                },
                "matchingRules": {
                    "$.body": {
                        "match": "type"
                    }
                },
                "status": 200
            }
        }
    ],
    "metadata": {
        "pact-js": {
            "version": "12.1.0"
        },
        "pactRust": {
            "ffi": "0.4.9",
            "models": "1.1.11"
        },
        "pactSpecification": {
            "version": "2.0.0"
        }
    },
    "provider": {
        "name": "ProductService"
    }
}

• The generated contracts are shared between services. Contracts can be stored and shared in a central location. A public directory, git or PactFlow may be preferred.
• Pact library runs a mock server using contracts, and this mock server fulfills the requests in the tests. Thus, there is no need to go to the original service or mocked one during tests. If there is a mismatch with the models specified in the contracts, the tests will fail.

Both solutions work similar and great. Main difference is Spring Cloud Contract generates tests according to contract, Pact generates contracts according to tests. You can pick one which fits your enviroment well.

with Sevket Dokgoz