Lecture 268: Using Testcontainers for Integration Testing with Real Databases

When it comes to microservices, databases play a crucial role. Your code can look perfect, but if the SQL queries don't behave as expected, or the database schema differs from what you assumed, the results can be disastrous. Now imagine you have 10 microservices in the system. Bugs at this level can literally throw the whole system into chaos.

Integration testing with a database helps to:

• Verify that SQL queries work correctly.
• Make sure JPA annotations and entities are configured properly.
• Run database migration tests (for example, Flyway or Liquibase).
• Ensure transactions are handled correctly.

However, spinning up a real database for every test run is hard and slow. That's where Testcontainers comes in.

What is Testcontainers?

Testcontainers is a Java library for writing tests that use Docker containers. It lets you spin up and tear down temporary containers for databases, message brokers, web servers, and other services right during test execution. The beauty is that as soon as tests run, containers are started as needed, and after tests finish they're destroyed automatically.

Real-life example: imagine renting a car from a car-sharing service. You use it only when you need it, then give it back. Testcontainers works similarly with Docker containers for tests.

How does Testcontainers work?

1. You declare which container to run (for example, PostgreSQL).
2. The container is started before the test.
3. Your test code connects to the container and tests interactions.
4. After the test finishes the container is automatically removed from your machine.

Main advantage: No dependency on real databases or external services. Everything is isolated.

Installing Testcontainers

First, add dependencies to your pom.xml (if you're using Maven):

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>1.19.0</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>postgresql</artifactId>
    <version>1.19.0</version>
    <scope>test</scope>
</dependency>

If you use Gradle, add:

testImplementation 'org.testcontainers:junit-jupiter:1.19.0'
testImplementation 'org.testcontainers:postgresql:1.19.0'

Also, make sure Docker is installed and running. If you're using Docker Desktop, you're good to go.

Your first Testcontainers test

Let's start with the simplest test. We'll spin up a PostgreSQL container and check we can connect to it:

import org.junit.jupiter.api.Test;
import org.testcontainers.containers.PostgreSQLContainer;

import static org.junit.jupiter.api.Assertions.assertNotNull;

public class PostgresTest {

    @Test
    public void testPostgresContainer() {
        // Create a PostgreSQL container
        try (PostgreSQLContainer
    postgres = new PostgreSQLContainer<>("postgres:15.1")) {
            // Start the container
            postgres.start();

            // Check that the container is running and the connection URL is not null
            assertNotNull(postgres.getJdbcUrl());
            System.out.println("PostgreSQL is running on " + postgres.getJdbcUrl());
        }
    }
}

If you run this test, Docker will start a PostgreSQL container, run the test, and destroy the container. Check the console: you should see info about the started database.

Integration with Spring Data JPA

Now let's hook Testcontainers into a real application. We'll use Spring Boot and JPA.

1. Spring Boot configuration

In application.properties put some "placeholder" database settings (they'll be overridden by Testcontainers during tests):

spring.datasource.url=jdbc:tc:postgresql:15.1://localhost/testdb
spring.datasource.username=test
spring.datasource.password=test
spring.jpa.hibernate.ddl-auto=update

Note the format jdbc:tc:postgresql:15.1://localhost/testdb. Testcontainers will automatically pull a PostgreSQL container of the specified version!

2. Creating an entity

Add the User entity:

import jakarta.persistence.Entity;
import jakarta.persistence.GeneratedValue;
import jakarta.persistence.Id;

@Entity
public class User {

    @Id
    @GeneratedValue
    private Long id;

    private String name;

    // getters and setters
}

3. Creating the repository

Create a simple repository interface:

import org.springframework.data.jpa.repository.JpaRepository;

public interface UserRepository extends JpaRepository<User, Long> {
}

4. Testing the repository with Testcontainers

Add tests to verify CRUD operations. Testcontainers will create a PostgreSQL container when tests run:

import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest;

import static org.junit.jupiter.api.Assertions.assertEquals;

@DataJpaTest
public class UserRepositoryTest {

    @Autowired
    private UserRepository userRepository;

    @Test
    public void testSaveUser() {
        // Save a new user
        User user = new User();
        user.setName("John Doe");
        User savedUser = userRepository.save(user);

        // Verify the user was saved
        assertEquals("John Doe", savedUser.getName());
    }
}

When you run this test, Testcontainers will start a PostgreSQL container, create the testdb database, and then execute SQL operations. After the test the container will be destroyed automatically.

Advantages of Testcontainers

1. Environment isolation: each test database runs in its own container.
2. Minimal setup: you don't have to manually bring up databases or clean them up after tests.
3. Realism: you test your code against a real database, not a fake.
4. Support for many databases and services: e.g., MySQL, MongoDB, Redis, Kafka.

Common pitfalls and usage notes

1. "Docker not found" error. Make sure Docker is running and your dev environment can access it.
2. Slow tests. Starting containers takes time. Use shared containers for a group of tests to speed things up.
3. Resource leaks. Don't forget to close containers after tests if you manage them manually (not the auto-managed Testcontainers).

Where this is useful

1. Microservice development: spin up a real database or a message broker (Kafka, RabbitMQ) for tests.
2. Migration verification: make sure your Flyway migrations work on all target databases.
3. Job interviews: when asked "How do you test databases in your projects?" answering "Testcontainers" will spark interest and score you some extra points.

Now you know how Testcontainers lets you spin up a real test environment straight from code. Run your tests and feel the magic of Docker isolation! In the next lectures we'll keep exploring microservices and other testing tools.