Repeatable data and configuration

1. Repeatability in a regression suite

A regression test is valuable only as long as you can run it ten times in a row and get the same result. If a test is “green sometimes, red sometimes,” the team quickly starts treating it like a temperamental cat: cute enough, but better not touch it. And that is deadly for a regression suite.

From an engineering perspective, repeatability is not some abstract “purity.” It is a very pragmatic rule: a test should fail only when behavior breaks, not because “last time there was already an article with the same slug in the database” or because “today the moderation stub decided to be in a bad mood.” Any randomness turns regression into noise, and noise in tests is the most expensive kind, because it steals trust.

Let us fix a simple idea: a regression test has three “ingredients,” and all three must be under control.

Application code (obviously).
Configuration (which beans are enabled, which profiles are active, which limits and flags are set).
Seed data (what is already in the database before we start the scenario).

If even one ingredient drifts, you are not testing the system — you are testing luck. And luck, as everyone knows, works well in presentations, but poorly in CI.

Below is the scheme I suggest for thinking about a regression test in ContentHub:

flowchart TD
    A["Test profile + overrides
(configuration branch)"] --> D["Spring Boot ApplicationContext"]
    B["Flyway migrations
(schema)"] --> E["Test database"]
    C["@Sql / fixtures
(seed data)"] --> E
    D --> F["HTTP scenario steps
(MockMvc/RestTestClient)"]
    E --> F
    F --> G["Observable checks:
HTTP + rereading state"]

Notice: in this picture @SpringBootTest is not the “main character” at all. The main character is control over the environment. Annotations are just the way to pin that control down.

2. The basic test environment: the test profile

When you start getting expensive full integration tests, you suddenly discover that configuration is also part of behavior. And a very tricky part at that: it can fail in ways that keep the application from starting, or make it start “wrong,” or make tests affect one another. That is why a regression suite needs a stable base environment — like a house foundation.

In this course we already agreed that for tests we have a separate test profile with application-test.yml. Its job is to make the environment safe and predictable: a test database, test users, clear storage paths, disabled/simplified side effects, and fixed time (if it is configurable through Clock). The main point is that the test should not depend on your laptop, the mood of the operating system, or the phase of the moon.

For a daily regression, the normal baseline here is: @SpringBootTest + @AutoConfigureMockMvc + @ActiveProfiles("test"). This is the full-context MockMvc path: it brings up the real graph of the application, but does not require a live HTTP port. After that, the specific scenario changes only its own things — @Sql, fixture seed, local properties, or an additional profile. The live server remains a separate booster, not the new default.

Here is a minimal skeleton:

import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.test.context.ActiveProfiles;

@SpringBootTest
@AutoConfigureMockMvc
@ActiveProfiles("test")
class BaseRegressionMvcTest {

    // This is a normal full-context baseline for business-flow regression.
    // The scenario above only adds its own differences: @Sql, fixtures, local properties, extra profile.
}

An empty class here is normal: it simply makes the agreement about the environment explicit. We are not building a test framework inside the project; we are only fixing that a regression test lives in the test profile by default and goes through full-context MockMvc.

To make it easier to decide where to place a setting, it helps to keep a small table in mind. It is not about absolute truth, but about the discipline that saves a suite from chaos:

The main idea: the base environment should be shared and boring. Anything unusual should be local and explicit, next to the test that needs it.

3. Seed data via @Sql

In regression scenarios you almost always have a “backstory” that you do not want to replay through the API every time. For a publish flow, the backstory sounds like: “the article is already in IN_REVIEW.” For an archive flow: “the article is already published.” And that is normal: in a real product you also rarely verify publishing by starting from an empty database and a user registration process.

The most direct way to fix the backstory is @Sql. It is good because the readability is no worse than that of a well-named method. And yes, SQL in tests is not “dirty” or an “anti-pattern” if you use it as a small scenario seed, not as a second layer of migrations.

For example, for a publish test you can explicitly say: before the test, load the categories and one article in the required status.

import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.jdbc.Sql;

@SpringBootTest // Base regression baseline: full application context
@AutoConfigureMockMvc // HTTP scenario steps without a real port
@ActiveProfiles("test") // Shared test environment branch
@Sql({
        "/sql/categories.sql",
        "/sql/article-in-review.sql" // Fix the backstory: the article is already in the required status
})
class ApproveAndPublishRegressionTest {
}

What do we put into src/test/resources/sql/article-in-review.sql? Keep the file small and “story-like.” I usually choose stable IDs (for example, 100) and clear slugs — this makes the test data almost part of the lecture text.

-- src/test/resources/sql/article-in-review.sql

-- A stable ID helps tests stay deterministic and readable.
-- A clear slug makes diagnosis easier if the test fails and you inspect the database.
insert into articles (id, title, slug, status, author_username, category_code)
values (100, 'Intro to Spring', 'intro-to-spring', 'IN_REVIEW', 'editor1', 'java');

Of course, in your real schema you will have created_at, summary, body, and possibly version. That is fine: the file should match your actual DDL. The important thing is different: you fix exactly what is needed so the scenario starts, and you do not try to make a “universal dataset for all occasions.”

Now the unpleasant part, which is better to say out loud: integration tests on @SpringBootTest by default do not give you automatic rollback “like @DataJpaTest.” That means the data you inserted with @Sql will remain in the database and can interfere with the next test if you do not ensure isolation.

There are two clear paths, and both are acceptable in a regression suite.

The first is to do cleanup via @Sql after the test. This is as explicit and honest as it gets, especially for beginners.

import org.springframework.test.context.jdbc.Sql;
import static org.springframework.test.context.jdbc.Sql.ExecutionPhase.AFTER_TEST_METHOD;

@Sql(scripts = "/sql/cleanup.sql", executionPhase = AFTER_TEST_METHOD)
class ApproveAndPublishRegressionTest {
}

And cleanup.sql usually looks boring, but reliable:

-- src/test/resources/sql/cleanup.sql
delete from article_attachments;
delete from articles;

The second path is to wrap tests in a transaction at the test-method level and rely on rollback. For MockMvc scenarios this often works and makes life much easier, but there is a nuance: as soon as you move to a live server (RANDOM_PORT), the test transaction no longer “embraces” the server thread, and rollback does not save you. So within a regression suite we usually choose one clear style and do not mix “sometimes this, sometimes that.”

The important thing here is not to fall into two extremes. On one side, “let us always use @DirtiesContext; it will clean everything” is a bad idea, because you kill context caching and turn the suite into a tortoise. On the other side, “let us not clean anything, maybe it will pass” is a ticket to the world of phantom failures. A regression suite should be boringly predictable.

4. Seed data via fixtures

SQL files are a great way to describe backstory, but sometimes they start to get in the way: the schema changes, there are more columns, and you are tired of fixing 15 insert statements because you added one field. At times like that people either get angry at tests, start hating migrations, or open a third tab with memes so they will not cry. To avoid getting there, there is a second approach: prepare the initial state through code.

By “through code” I do not mean “make 5 HTTP calls in @BeforeEach,” but specifically through a repository/EntityManager and a small fixture factory. This gives you type safety and survives refactorings better: the compiler will show you that you did not fill in a required field.

A simple fixture factory can live in src/test/java/.../testfixture/ArticleFixtures.java and return a ready entity. Yes, this is test code. No, it is not a crime.

import java.time.Instant;

public final class ArticleFixtures {

    private ArticleFixtures() {
        // Prevent instance creation: this is a utility class with factory methods
    }

    public static Article inReviewArticle(long id, String slug) {
        // Important: if time affects the test, it is better to use Clock and fix it in configuration.
        return new Article(id, "Intro to Spring", slug, ArticleStatus.IN_REVIEW, "editor1", "java", Instant.now());
    }
}

I am showing this as a small example on purpose. In real life you will have a constructor/builder for your model, and possibly you will use Clock so that Instant.now() does not wander. The main point is the approach: one method = one “scenario object.”

Next, in the test you keep setup short and readable, without hiding the business precondition:

import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;

class ApproveAndPublishRegressionTest {

    @Autowired
    ArticleRepository articleRepository; // The repository is a quick way to put backstory into the database

    @Test
    void setupExample() {
        // Prepare the database so the scenario starts not "from thin air" but from an explicitly defined state.
        articleRepository.save(ArticleFixtures.inReviewArticle(100L, "intro-to-spring"));
    }
}

Why is this approach not “worse than SQL”? Because it works well when you care not about “pure DDL,” but about consistency between the Java model and persistence. If a field changed, the test fixture will remind you at compile time: “hey, buddy, this field is now required.” A SQL file stays silent and blows up later somewhere at runtime, preferably at night.

But setup-through-code has an honest cost too. First, you risk accidentally pulling in extra domain logic. For example, if you create an article through a service, you may trigger moderation or a side effect, and the test will suddenly start depending on an external adapter even though you just wanted to “put one record in the database.” Second, a code fixture can become too smart and start hiding important details: you look at the test and do not understand why the article is under review, who the author is, and why the slug is exactly that.

To keep control, I like to keep a simple rule specifically for a regression suite: for one scenario, choose one main data-preparation style, and that style should read like a story. For a submit flow, the story often starts with HTTP creation of a draft (because we want to test creation). For a publish flow, the story often starts with @Sql or a fixture seed (because creation has already been tested elsewhere). The main thing is not to make a mixed salad of @Sql, repository calls, and a couple of extra API calls “just in case.”

Here is a small table that helps you avoid arguing with yourself in the morning:

In a regression suite, the combination usually wins, but at the level of the test set, not inside a single test method. In other words, different tests use different styles, but each specific test remains transparent.

5. Configuration branches: profiles and overrides

As soon as you reach a regression suite, it turns out that “settings” are not decoration but a real source of branching. In ContentHub this is especially noticeable on submit for review: the result can depend on what moderation returns, and in a teaching project moderation usually gets stabilized through a stub mode or test flags. And we need the test to verify one branch and do so deterministically.

The term “configuration branch” in this context is very simple: it is a situation where the same application code behaves differently because of a profile or a property. For example, in one mode moderation always answers OK, and in another — BLOCK. Both modes are legitimate, but the test should not depend on which one is “accidentally active.”

And it is important to see one more step. A configuration branch is not only a different property value. If a profile or flag switches ModerationClient through @Profile or @ConditionalOnProperty, the graph of active beans itself changes. For Spring this is a different snapshot of ApplicationContext, so the rule one class = one branch is not housekeeping, but a way to avoid mixing different worlds inside one regression test.

There are two practical ways to fix a branch.

The first way is through profiles. It is good when the branch is used in several tests and you want to give it a name. For example: moderation-ok and moderation-block. Then the test class literally becomes a “representative of the mode.”

import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ActiveProfiles;

@SpringBootTest
@ActiveProfiles({"test", "moderation-ok"})
class SubmitFlowOkBranchIntegrationTest {
}

The profile can be described in a YAML file in the test resources. Yes, it is the same YAML, it just lives in src/test/resources.

# src/test/resources/application-moderation-ok.yml
# Configuration branch for tests: moderation always answers OK
contenthub:
  moderation:
    test-decision: OK

The second way is a local override directly on @SpringBootTest. It is good when the branch is needed only in one place and you do not want to create more profiles (and files).

import org.springframework.boot.test.context.SpringBootTest;

@SpringBootTest(properties = "contenthub.moderation.test-decision=BLOCK") // Targeted override on top of the shared test profile
class SubmitFlowBlockBranchIntegrationTest {
}

Both approaches work. The difference is in the “social side” of tests: profiles give a nice name and reduce copy-paste, while local properties stay as close as possible to the specific test and do not bloat the configuration directory.

But here is what really matters for a regression suite: do not mix different branches in one test class. Even if it seems like “so what, one test OK, another BLOCK.” For Spring this means different configuration conditions, so you will either get multiple contexts, or start building @DirtiesContext, or suddenly catch the wrong bean. All of this turns regression into a lottery.

There is also a purely methodological argument: when SubmitFlowOkBranchIntegrationTest fails, you immediately understand which branch broke. When you have one class called SubmitFlowIntegrationTest with different modes inside, diagnostics get noticeably worse. And we build a regression suite precisely for a fast and clear signal.

Another nuance: each unique combination of profiles/properties almost always means a separate ApplicationContext. That is normal, but we have a “regression budget.” So branches in a regression suite should not be “all possible ones,” but only those that are truly worth their cost. For ContentHub this usually means: one stable submit-flow branch (for example, OK), and, if you really want extra protection, a separate test for BLOCK as a representative case — but without trying to cover every option at once.

6. Typical mistakes in regression tests

Mistake #1: the test depends on random database state “left over from previous tests.”
This usually looks harmless: one scenario creates an article, the second “somehow” finds it by id and continues. As long as the execution order does not change, everything is fine. Then Gradle runs the tests in a different order, or you add another class, and everything starts failing “every other time.” A regression suite should start from an explicitly prepared state, not from hope.

Mistake #2: one scenario mixes @Sql, repository calls, and a couple of HTTP calls because “that is definitely more reliable.”
This is counterintuitive, but it usually gets worse: you end up with a long prologue that is hard to read, and several sources of truth about what counts as the start. Then the test breaks, and you spend time figuring out which of the three setup mechanisms created the problem. In regression, it is better to have one main way to prepare data per scenario and clear names for the backstory.

Mistake #3: different configuration branches are checked in one test class.
Sometimes people do this via different @TestPropertySource annotations on methods or through manual value switching. The result is almost always sad: either the context cache behaves unexpectedly, or you have to use @DirtiesContext, or tests affect one another through static settings. It is much cheaper and clearer to keep the rule “one class — one branch” and name the class so the branch is visible in the name.

Mistake #4: important starting assumptions are hidden in “smart” helper methods.
Helper methods are useful when they remove transport noise, but if they start creating half a system off-screen, the test stops being scenario documentation. This is especially dangerous for a regression suite: you want the test to explain what the system does, not force you to jump through files like a quest. If the backstory matters, it is better to make it explicit through @Sql with a speaking file name or through a fixture factory with a speaking method name.

Mistake #5: point settings are placed into the shared application-test.yml.
The temptation is understandable: “so we do not have to write it again in the test.” But after a couple of weeks application-test.yml turns into a museum of random flags, and nobody understands why moderation is always OK and attachments are suddenly limited to one file. The shared profile should be stable and minimal, and rare deviations should live next to the specific test — that makes the reason for the behavior transparent.