Contract testing is a type of testing based on validating the contracts between services. In this blog post, we dive into how contract testing works, which tools you can use to get started, and when contract testing could be the right choice for you.
What is Contract Testing?
Contract testing is a software testing methodology that tests the interactions between different microservices or software components based on the contracts between them. In contract testing, each service or component is given a contract, which defines how to work with the service and which responses to accept.
Contracts typically detail the expected behavior of the component, such as the inputs and outputs of its API, its performance characteristics, and its response times. Each service is then tested to ensure that it adheres to its contract requirements.
Contract tests have multiple advantages in microservices. They can help quickly identify and prevent issues and bugs that could arise when multiple components are integrated together. They speed up the development process, since they can help developers quickly understand the code and provide fast feedback loops. They also cut infrastructure costs by reducing the number of tests to run in a deployed environment.
Contract testing even supports microservices scalability by speeding up the testing process, which supports development at scale. Finally, they empower developers to take ownership of their code, by requiring them to ensure contracts are comprehensive and detailed.
How Does Contract Testing Work?
Contract testing is the process of defining and verifying (testing) a contract between two services, dubbed the “Provider” and the “Consumer”. The service owner is the “Provider” while entities that consume the service are called “Consumers”.
There are two types of contract testing: consumer-driven and provider-driven.
Consumer-Driven Contract Testing
Consumer-driven contract testing is a contract testing approach in which the service consumer, for example, a client application, defines the contracts that the service provider must adhere to.
In consumer-driven contract testing, the consumer creates a test suite that specifies the expected behavior of the service provider. These tests are called “consumer contracts”. The service provider then implements the necessary functionalities to pass these tests, i.e to meet them. The service provider’s implementation is verified against the consumer contracts to ensure that it meets the expected behavior.
Consumer-driven contract testing ensures that the services meet the functionality and performance requirements required by the consumer.
Provider-Driven Contract Testing
In provider-driven contract testing, the service provider defines the contracts that the consumer (i.e., the client) must adhere to when consuming the service. These contracts are usually defined in a format that can be shared between the service provider and the client, like OpenAPI or Swagger.
Once the contracts are defined, the service provider generates tests that verify that the client adheres to the contracts. These tests are called “provider contracts”. The service provider runs the producer contract tests against the client’s implementation to ensure that it meets the expected behavior defined in the contracts.
Provider-driven contract testing ensures that the service is designed to meet the provider’s requirements and that it is backwards compatible.
5 Contract Testing Use Cases
Contract testing is a valuable methodology for ensuring the reliability and compatibility of microservices and APIs. However, it’s not a good fit for all testing requirements, Here are some common use cases where contract testing is recommended:
- Microservices testing – Contract testing can help ensure that each service conforms to the agreed-upon API contracts. This can help prevent issues and bugs that result from architectural complexity, without having to wait for slow results from e2e tests.
- Third-party integrations – When integrating with third-party services or APIs, contract testing can help ensure that the code conforms to the requirements and behavior of the external service. This can help prevent compatibility issues and ensure that the code is reliable and secure.
- Versioning – When introducing new versions of an API or service, contract testing can help ensure that the new version is backwards-compatible with existing clients. This can help prevent issues such as downtime or data loss when upgrading to a new version.
- New project onboarding – When developers start working on a new project, contracts can help them get up to speed quickly. Contracts provide OpenAPI specifications, concise and accessible explanations of the service’s capabilities, and its expected KPIs. By using contract tests, developers can better understand the requests and responses, which makes it easier to understand the code.
- Checking backwards compatibility – Using contracts can make the backwards-compatibility checking process much easier for developers. With contracts, developers can easily search the service repository for the past versions, and test the compatibility of the new feature with each of these versions. This eliminates the need for manual testing and without compromising the overall stability of the system.
Note that contract testing is not recommended as a substitution for load or performance testing.
What is the Difference Between Contract Testing and Integration Testing?
Although the two are often confused, contract testing and integration testing are two different types of testing that serve different purposes.
Contract testing focuses on verifying that the interactions between different services or APIs are consistent with the predefined contract. It ensures that each service or API adheres to the agreed-upon contract and that changes to one service do not break the functionality of other services that rely on it.
In addition, contract testing can be performed in parallel across services, since testing a service takes place independently of other microservices or components. Finally, contract testing uses mocked data, which enables resolving issues more easily.
On the other hand, integration testing is a type of testing that focuses on verifying that the individual components of a system all work together correctly as a whole. It tests the integration points between different modules, components, or services, and the interactions between them.
Integration testing uses real data, requires a complex environment setup that mimics the production environment, and is more complex than contract testing due to the need for a larger environment setup and interactions between multiple components.
Contract Testing vs. Integration Testing: A Comparison
|
Feature/Capability |
Contract Testing |
Integration Testing |
|
Scope |
Focuses on the interactions and contracts between microservices or components. |
Tests the interaction and behavior of multiple components or systems as a whole. |
|
Timing |
Done in parallel with the development of each microservice or component. |
Usually performed after the development of individual components and before the final system testing. |
|
Purpose |
Ensures that each microservice or component behaves as expected and meets the agreed-upon contracts with other microservices or components. |
Validates the integration and interaction of multiple components or systems as a whole. |
|
Dependency |
Independent of other microservices or components. |
Dependent on the availability of other components or systems. |
|
Test Data |
Uses mocked or stubbed data to simulate interactions between microservices or components. |
Uses real data to test the interaction and behavior of multiple components or systems as a whole. |
|
Test Environment |
Can be executed in isolation without the need for a complex environment setup. |
Requires a complex environment setup that mimics the production environment. |
|
Coverage |
Covers the interfaces and contracts between microservices or components. |
Covers the integration and interaction of multiple components or systems as a whole. |
|
Complexity |
Relatively simpler than integration testing. |
More complex than contract testing due to the need for a larger environment setup and interactions between multiple components. |
|
Debugging |
Easier to debug due to the smaller scope and the use of mocked or stubbed data. |
More challenging to debug due to the larger scope and the use of real data. |
Both types of testing are important for ensuring the reliability and functionality of a software system, and may be used together as part of a comprehensive testing strategy, especially in microservices.
Contract Testing Tools
There are many open-source and commercial contract testing tools available for testing microservices and API contracts. Some of the most popular ones include:
Pact
Pact is a very known open-source contract testing tool. It allows consumers and providers to define and verify their contracts. Pact provides support for various programming languages (Ruby, Javascript, the JVM, or any other language) and frameworks. In Pact, the consumer code generates the contract and it operates based on a consumer-driven approach.
Spring Cloud Contract
Spring Cloud Contract is another open-source contract testing tool. Unlike Pact, it uses a provider-driven approach. Spring Cloud Contract provides support for Java and Spring Boot applications and allows providers to define and verify their contracts using Groovy or YAML.
Swagger/OpenAPI
Swagger/OpenAPI are tools for defining API specifications and they can also be used for contract testing. By providing a standard way to describe RESTful APIs, these specs can be used to automatically generate contract tests and documentation.
Karate DSL
Karate is an open-source framework for API testing and contract testing. Karate is based on the Cucumber / Gherkin standard, scripts are in plain text, and even non-developers can write them.
RestAssured
RestAssured is a Java-based library for testing RESTful APIs. It provides a simple syntax for writing tests and can be used to verify API contracts.
Bottom Line
Contract testing is a valuable practice and we recommend trying it to see if it’s a good fit for your needs and culture. Since one of the key concepts of contract testing is mock services, you will need a solution for setting them up.
