The process of setting up an environment for integration testing using remote Docker

Demand background

The team has the need for integration testing, which requires reliance on some middleware, such as mysql and rabbitmq. Every developer has the need to write tests and run tests on the code they develop.

In order to avoid mutual interference, you can choose to build your own dependent environment locally in the R&D center. We hope that these environments will be easy, fast, and easy to clean up.

Use Docker to build the environment

Docker can well meet the above demands.
But this is not enough, we are also facing the following problems

• The local environment is complicated to build. Every R&D team must install the Docker environment locally, which will lead to some barriers and inconveniences in use. Take our company as an example. Due to strict network control, we develop on the intranet and cannot connect to the Internet. Especially when installing Docker on Windows, you need to be connected to the Internet. Although there are ways to solve this problem, every newcomer has to go through some local environment installation and debugging process, which is cumbersome.
• Test run speed is not guaranteed. When a project relies on a lot of middleware, local Docker will also take up a lot of resources and affect the test speed.
• Multi-environment integration testing is troublesome. Since the integration test relies on local Docker, when this code is packaged and run on Jenkins, Docker needs to be installed in the corresponding environment.

In summary, using docker can help us quickly build a project dependency environment, but localized docker dependencies still make our code not pure enough during testing, and each operating environment requires local docker installation.

Improving integration testing with a centralized docker server

In fact, Docker itself provides a remote connection mode, which allows us to centrally deploy Docker, and then integrate the test code to build and test dependent middleware using Docker server via TCP connection.

Docker Server remote link configuration

Taking centos 7.6 as an example, this article explains how to configure a docker to be able to connect remotely.

Open the remote connection port in /etc/docker/daemon.json

{"hosts": ["tcp://0.0.0.0:2375", "unix:///var/run/docker.sock"]}

Add the file /etc/systemd/system/docker.service.d/override.conf . If the above path does not exist, create it manually.

 [Service]
 ExecStart=
 ExecStart=/usr/bin/dockerd

• Reload the daemon process systemctl daemon-reload
• Restart the docker container systemctl restart docker.service

Testcontainers framework

After deploying the remote docker, the problem that follows is

• How to connect and use a remote Docker environment in code
• When two developers run test cases at the same time, how can we ensure that the containers they start do not conflict on the ports?
• How to clean up the container after use

Fortunately, the Testcontainers framework helps us solve the above problems well.

• It can be integrated with junit 4 and junit 5 to help us start and stop containers
• Each time a test is run, a new container is started and different ports are exposed, so that when two developers run test cases at the same time, the environments do not interfere with each other.
• It uses testcontainers/ryuk to clean up unused containers after a specified delay.
• All of the above is transparent to the user.

Testcontainers integrated with spring boot

Furthermore, the game company Playtika provides a test framework that integrates Testcontainers with Spring Boot https://github.com/Playtika/testcontainers-spring-boot , making it easier to write integration tests for applications in the Spring Boot or Spring cloud ecosystem.

Environment variable dependencies

When using Testcontainers or playtika's testcontainers-spring-boot for remote docker links, there is no need to install the docker client locally. However, relevant environment variables need to be configured so that the code can know the address of the remote docker. There are several ways to configure this address:

• Method 1: System environment variable configuration. Configure the environment variable DOCKER_HOST=tcp://remote_docker_server_ip:2375 in the current system
• Method 2: directly in the Java test code, before the container is constructed, specify the environment variable through the code System.setProperty("DOCKER_HOST","tcp://remote_docker_server_ip:2375")
• Method 3: If the integration test uses the Maven failsafe plugin, configure the environment variables on the plugin
  

Use and manage Docker remotely via command line

The above test code does not require the installation of the Docker client. But if we need to manage docker via the command line, we can install some docker clients to communicate with the remote docker. Of course, the above Testcontainers are equivalent to a kind of client.

For client program installation methods for different operating systems, see: https://gist.github.com/kekru/4e6d49b4290a4eebc7b597c07eaf61f2

References

https://www.testcontainers.org/
https://gist.github.com/styblope/dc55e0ad2a9848f2cc3307d4819d819f
https://github.com/Playtika/testcontainers-spring-boot
https://gist.github.com/kekru/4e6d49b4290a4eebc7b597c07eaf61f2

This is the end of this article about using remote Docker for integration testing. For more information about Docker integration testing, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!