Roles of compose.yaml and compose.dev.yaml

1. The risks of one large Compose file

When you first start using Compose, it seems perfect: one compose.yaml, one docker compose up command — and everything “just” starts. Problems come quietly, like updates in production: first you add Redis, then RabbitMQ, then “just” a debug port, and then you discover that one file now contains two realities at once — the canonical startup and the local dev mode. In the end, YAML starts to look like a giant class with 2,000 lines, where business logic, UI, database access, and a little bit of “magic” all live together.

The most unpleasant part is not even the file size, but the mixing of responsibilities. In compose.yaml, settings that are only needed “right now on my laptop” gradually appear, while settings that must be stable and identical for the whole team are right next to them. In Java terms: if you mix DTOs, services, repositories, and even Thread.sleep() in one place, the code may seem to work, but living with it becomes depressing.

Compose files degrade especially quickly when you start adding debug support. A Spring Boot service gets a second port, JVM arguments for JDWP appear, sometimes you want more logs, sometimes you want to switch the Dockerfile target to the development stage. And if you put all of that into the base file, then “normal startup” will stop being normal. A new developer will launch the stack and wonder: why is the debug port open at all? why is the service built as development rather than runtime? why does our logging look like we are investigating a crime of the century?

Another effect is copy-paste. People often solve the problem like this: they make a second file, but ... copy almost the entire app service into it. Today it matches, and tomorrow you change an environment variable in one place and forget the other — and you end up with two versions of reality: “normal mode” and “dev mode,” which behave differently because YAML drifted apart.

To avoid this, we need a very simple rule: the base file should be “boring, stable, and canonical,” and the dev file should be “small and targeted.” And yes, in YAML that sounds almost like an unattainable dream, but we will try.

2. The canonical compose.yaml: boring and stable

The word “canonical” here is not for pomp, but to agree on meaning. A canonical compose.yaml is the one that describes the normal model of the stack: which services exist, how they are connected, which volumes and healthchecks are considered standard, which ports are exposed externally, and which environment variables are needed for the service to work in the “usual” mode. This is the file you want to show a colleague with the words: “this is how our local environment lives.”

And “boring” is truly a compliment here. Boring means predictable. It does not try to guess what you are debugging today. It does not enable debug “just in case.” It does not adapt itself to a specific IDE. It simply brings up the stack the way it is supposed to work in the base scenario. If in Java a good application.yml should not be “for a special case” by default, then compose.yaml is also not the place for a “special case.”

In our course, this is even more important methodologically: compose.yaml is not only a startup method, but also a document. From it, the student (and you a month later) reconstructs the picture: what dependencies the service has, what their names are, which external ports are needed, where the state lives (volumes), and how startup is organized.

It is very useful to mentally divide settings into two types: those that reflect the architecture of the environment, and those that reflect development convenience. Architectural things should go into the base file. Dev conveniences should be moved out separately.

Here is a compact table that helps keep the boundary clear:

app

postgres

redis

rabbitmq

For this boundary, a compact full-stack example with a fixed SPRING_PROFILES_ACTIVE is enough for us right now. As soon as you start something other than the full stack, this parameter is better set by the startup scenario, rather than being baked in as an eternal constant.

Note: this is not a “ban,” it is a “point of ownership.” The base file owns the canonical behavior.

3. compose.dev.yaml as an overlay

compose.dev.yaml in our model is not an alternative project and not a “second Compose stack.” It is exactly a second file that contains only the differences from the canonical startup. Its role is similar to a neat @Profile("dev") layer in Spring: it adds what is needed specifically for development, but does not rewrite the whole world from scratch.

The main idea is very simple: if you open compose.dev.yaml and see there almost a full copy of compose.yaml, then you have already lost. Not because “it is forbidden,” but because you have set a trap for yourself: the files will inevitably drift apart, and you will end up debugging YAML differences rather than code.

In a good compose.dev.yaml, you usually keep things like “change the build target,” “add a debug port,” “add one environment variable.” You are literally placing a thin film over the base configuration.

For example, the base file could look like this (a fragment of the app service, intentionally short):

# compose.yaml
services:
  app:
    build:
      # Build context: Compose will look for the Dockerfile and sources in the project root
      context: .
    ports:
      # Canonical HTTP port of the application (must be the same for the whole team)
      - "8080:8080"
    environment:
      # For a compact full-stack variant, we fix the full set of profiles:
      # here the boundary between base and dev matters, not scenario-specific switching
      SPRING_PROFILES_ACTIVE: "postgres,cache,messaging"

And the dev file could look like this:

# compose.dev.yaml
services:
  app:
    build:
      # Dev overlay: in development we build not into the runtime stage, but into development
      target: development
    ports:
      # Dev overlay: open an additional port for the debugger
      - "5005:5005"

Notice the psychological effect: the dev file does not try to redefine context, does not repeat 8080:8080, and does not duplicate profiles. It says: “in dev mode, app has two more traits: a different target and one more port.”

Now here is an example of what a “bad dev file” looks like (also a fragment):

# compose.dev.yaml (bad)
services:
  app:
    build:
      # Bad: the dev file repeats the canonical context (an unnecessary source of truth)
      context: .
      # The only real dev change here is target; the rest should stay in the base
      target: development
    ports:
      # Bad: the canonical port is duplicated here, and it may drift from compose.yaml
      - "8080:8080"
      # OK: this is a real dev addition
      - "5005:5005"
    environment:
      # Bad: we duplicate canonical profiles and risk "forgetting to update" them later
      SPRING_PROFILES_ACTIVE: "postgres,cache,messaging"

The problem is not that “it does not work.” It does work. The problem is that you created a second source of truth. In a week you change a profile or a port in compose.yaml, forget the dev file — and then you get a mystery at the level of “why is debug fine for me, but normal is not?” Surprisingly, people then blame Spring Boot.

4. The “point of ownership” rule for overrides

In any configuration system, a moment comes when it is more important not “how to launch,” but “how to understand why it launched exactly this way.” And that is where the “point of ownership” becomes almost a life raft. The rule means that every setting must have a place where it lives as the main source of truth. Otherwise, you get the eternal argument: “where is this set?”

To make the rule work, you do not need much: split the settings into “stable” and “local,” and keep that boundary intact. Stable settings are the ones that should work the same for every student and every teammate. Local settings are the ones that are enabled by circumstance and should not change the canonical startup.

In the context of our project, it is convenient to think like this: compose.yaml is the “passport” of the stack, and compose.dev.yaml is the “stickers on the passport.” Stickers are useful, but the passport should not become unreadable because of them.

If you want a more engineering-like metaphor, you can imagine the layers like this:

flowchart TD
    A["compose.yaml
canonical stack"] --> C["Final configuration
that Compose will actually launch"]
    B["compose.dev.yaml
local differences"] --> C

This diagram does not yet explain the details of merge rules (that is a separate important discussion), but it captures the main thing: compose.dev.yaml does not exist on its own. Its purpose is to change the base stack, not to create a “parallel universe.”

And yes, discipline around “fewer lines” helps a lot here. If the dev file grows, that is almost always a symptom that you started duplicating the base. In most cases, when the dev file expands, the correct reaction is not “well, let it be,” but “why can’t we keep this in the base, or alternatively, why should this even be part of dev?”

5. Honest startup commands

When configuration is split into base and dev parts, it is important that the startup command is equally honest. That is, the command should make it clear: “I am starting the canonical stack” or “I am starting the stack with the dev overlay.” It seems like a small thing, but later it saves hours of explanations.

The canonical startup should be a maximally simple command. Ideally, without -f, because the base file is called compose.yaml and Compose will pick it up automatically:

# Canonical startup (without dev overlays)
docker compose up --build

And the dev startup should be written so that you cannot forget you are in dev mode. Therefore, we explicitly specify both files:

# Startup with the dev overlay: the second file adds only the differences from the base
docker compose -f compose.yaml -f compose.dev.yaml up --build

This explicitness is part of the “developer workflow”: you read the terminal history and understand why port 5005 suddenly became open or why the build goes through the development stage.

Here it is worth briefly discussing a popular “hidden” mechanism: compose.override.yaml. Compose really can automatically pick up an override file with that name. But as a teaching path and as team discipline, this often turns into magic: a person runs docker compose up and does not even realise that some overrides were loaded.

In our course, we deliberately prefer compose.dev.yaml and an explicit -f ... -f ..., because it is better for understanding and reproducibility. When the startup command clearly shows the dev file, you get less “mysticism” and more engineering control.

6. Applying this in Container-Ready Catalog Service

Now let us ground this in our training repository. At the root, we already have a Dockerfile, we have compose.yaml, and according to the final template plan, compose.dev.yaml should appear. In a good version, the structure looks very familiar:

docker-java-catalog-service/
├── Dockerfile
├── compose.yaml
├── compose.dev.yaml
└── .env.example

The base compose.yaml stores “what our stack is.” For the lecture, we do not need the whole 200-line file; it is enough to show the key principle on app. For example, the base service can fix the familiar full-stack runtime mode and HTTP port:

# compose.yaml (fragment)
services:
  app:
    build:
      # Canonical context: this is part of the "passport" of the stack
      context: .
    ports:
      # Canonical HTTP port of the application
      - "8080:8080"
    environment:
      # For a compact full-stack example, we fix the full set of profiles:
      # here the boundary between the base and dev layer itself matters
      SPRING_PROFILES_ACTIVE: "postgres,cache,messaging"

And compose.dev.yaml neatly adds dev differences. For example, a development target and a debug port, without trying to rewrite the base:

# compose.dev.yaml (fragment)
services:
  app:
    build:
      # Dev mode: we replace only the build target
      target: development
    ports:
      # Dev mode: we add only the debug port
      - "5005:5005"

It is important that we are not discussing “how exactly to configure debug” or “which exact JVM arguments are needed” right now. That is indeed a separate topic, and it only makes sense after you have learned to cleanly separate dev mode from normal mode. But even without the JDWP line, the main thing is already visible: the dev file is a thin overlay that does not destroy the canonical model.

And one more practical point that seems funny until it hits you in the face: the base compose.yaml must remain self-sufficient. In other words, the command docker compose up --build must work without requiring the dev file. Otherwise, you turn development mode into the “default,” and the canonical mode becomes something exotic that nobody runs. It is like keeping @Profile("prod") disabled by default and then being surprised that production looks different.

7. Common mistakes

When we first split Compose into two files, the brain often tries to “make it simpler,” and the same mistakes grow out of that. They are normal, because YAML itself encourages copy-paste (it really does), but it is better to recognise them by symptoms rather than by nightmares.

Mistake #1: turning the dev file into a copy of the base.
Usually this starts with the noble idea: “I’ll just duplicate app and then add a couple of lines.” A week later, a variable changes in the base file, and the dev file falls behind. As a result, normal and dev modes start behaving differently not because that was intended, but because the configs drifted apart. The fix is a strict principle: keep only the differences in the dev file; everything else belongs in the base.

Mistake #2: putting debug settings into compose.yaml because “it is convenient for me.”
Convenient — yes, but it turns dev mode into the default. Then a teammate comes along, launches the stack, and gets an open 5005 port and a JVM listening for a debugger, even though they do not need any of that. The base file should describe the normal startup; debug is a local overlay, and it must be explicitly enabled.

Mistake #3: creating a second service app-debug instead of overriding app.
At first glance it seems logical: “let there be app and app-debug.” In practice, you double the wiring: ports, env vars, depends_on, volumes. And then the race begins: which service is the “real” one now? where do we change the config? why does one work and the other does not? In our teaching model, it is much simpler to have one app service and two launch modes through an overlay.

Mistake #4: the base file stops being usable without the dev file.
Sometimes things that are actually part of the “canonical” stack move into the dev file, such as mandatory environment variables or required ports. Then compose.yaml becomes an “undocumented draft,” and reality lives only in dev mode. This is dangerous because you break reproducibility: the “normal startup” stops being normal.

Mistake #5: pulling OS-specific absolute paths into the dev file.
This is especially common on Windows/macOS. Someone writes C:\Users\... or /Users/name/... directly into YAML, and everything “works for the author.” Then the file is committed — and other environments fail to start. Even if the dev file contains local differences, it is still part of the team workflow, so relative paths and cross-platform compatibility matter. If there is a truly local setting that should not live in the repository, it is better to think about a local file outside VCS (but as a course approach, we keep compose.dev.yaml team-oriented and reproducible).

1

Task

Docker for Spring, level 21, lesson 0

Locked

Separating normal and dev startup for a single Spring Boot service

Separating normal and dev startup for a single Spring Boot service

Open

1

Task

Docker for Spring, level 21, lesson 0

Locked

Moving dev settings out of the base file for the `app + postgres` stack

Moving dev settings out of the base file for the `app + postgres` stack

Open