Writable layer and the fate of files

1. The importance of files in Docker

If you previously wrote Java applications “on your own laptop”, files seemed like something reliable and simple: you wrote a CSV into a folder — it stayed there. In the container world, that feeling quickly lets you down, because a container has its own file reality. And it is very honest: the container keeps changes not because it is kind, but because you have not destroyed this exact container yet.

Imagine a typical scenario from our end-to-end project Container-Ready Catalog Service: the service exports a catalog to CSV. You clicked the export endpoint, see “Export completed” in the logs, and you can even enter the container and see the file. Everything looks perfect... right up until the moment you do docker rm and suddenly realize that the “perfect” file was not your result, but a temporary note in the container’s notebook.

This lecture exists so that you develop the right mental model: where the application really writes, what writable layer is, why it is tied to a specific container, and why a container is not “a small VM with a disk”, even if you really want to think so. And in the next lecture, we will learn how to make a file “real” — by moving state outside through a mount.

2. Read-only layers and writable layer

When we talked about Docker layers and build cache, we discussed image layers as part of the build-time story. But layers are also important at runtime, because a container file system is almost always built on the model “many read-only layers + one write layer”. And this very single write layer is called the writable layer.

In simplified terms, it looks like this: you have a Docker image (a set of read-only layers), and when you start a container, Docker “covers” the image with one more layer where all changes go: created files, modified files, deleted files. It is like a transparent film over printed text: the original (image) is immutable, and your edits live on the film (writable layer).

Small diagram (not about low-level storage driver details, but about intuition):

flowchart TB
    %% The image consists of read-only layers, and writes go into a separate container layer
    subgraph IMG["Docker image (read-only layers)"]
        L1["Base layer: OS + JVM runtime"]
        L2["App layer: our jar and resources"]
        L3["Config defaults: what is inside the image"]
    end

    %% This layer is created when the container starts and lives as long as the container lives
    W["Writable layer (tied to a specific container)"]

    IMG --> W

The key idea: the writable layer belongs to the container, not to the image. Two important consequences follow from this automatically.

First: if you started two containers from the same image, each one will have its own writable layer. And if one container created the file /app/exports/catalog.csv, the other container will not see that file, because it lives in the first container’s “personal notebook”.

Second: when the container disappears, its writable layer disappears too. And this is where the most common beginner trap begins: “I did docker restart, the file stayed there — so everything is reliable.” No, that only means you have not deleted the container yet. Deleting a container is the moment when Docker says: “All right, we are throwing away the notebook.”

3. Writing a file inside a container

When Java code writes a file through Files.writeString(...), it does not know and does not have to know that it is inside a container. For it, this is just a file system. But we, as engineers, must understand: if this path is not moved outside, the write will go into the container’s writable layer. And while the container is alive, the file exists. After that, the lifecycle lottery begins.

A tiny code example that is very easy to write in an export service (and roughly how it will look in a teaching project, even if with some wrapper code):

import java.nio.file.Files;
import java.nio.file.Path;

// Where we write: inside a container this will go into the writable layer if the directory is not mounted from outside
Path exportFile = Path.of("/tmp/exports", "catalog.csv");

// Create the directory if it does not exist yet (otherwise the write will fail with an error)
Files.createDirectories(exportFile.getParent());

// It is easier to read CSV through a Text block than through \n in a single string
String csv = """
        id,sku,title
        1,SKU-1,Book
        """;

// Write the content to the file
Files.writeString(exportFile, csv);

This code “works” almost always. Including inside a container. And that is exactly why it is dangerous: it creates a false sense that the file part of the application is “solved”. In reality, you have simply written the file somewhere it will survive container stop/start, but not its destruction. And in the container world, destroying a container is a normal operation. You will do it yourself (cleanup), CI will do it, orchestration will do it, and even Docker Desktop may do it indirectly if you rebuilt and recreated the container.

So we should ask ourselves in advance: is this file a temporary internal artifact of the process (for example, an intermediate piece), or is it an external result that should be available to a person, tests, or other tools? Exporting a catalog to CSV almost always belongs to the second category. That means the path to it cannot be left as “just a string in code”.

4. Lifecycle and the fate of files

A beginner has a very understandable, but tricky, logic: “the container was restarted — the file did not disappear — therefore the file is stored reliably.” And here it is worth carefully separating several similar, but fundamentally different, events: stopping, starting, restarting, and deleting a container. Docker is not trying to confuse you — it honestly behaves like a process + its writable layer.

Let us summarize the fate of files in a table. This is not “the whole truth of life”, but a practical guide for daily development:

docker stop <name>

docker start <name>

docker restart <name>

docker rm <name>

docker run ...

And this is where the “magical” confusion arises for beginners: they test only the first three rows. That really looks like “persistence”. But it is not the persistence you need for an export result.

To feel this with your hands, it is enough to do a small demo once. Suppose the container is called catalog:

# Look at the contents of the directory inside the container
docker exec -it catalog sh -lc "ls -la /tmp/exports"
# total 8
# -rw-r--r-- 1 root root 42 Mar 21 10:15 catalog.csv

Now stop and start it again:

# Stopped: the container remains the same, the writable layer remains the same
docker stop catalog

# Started again: this is the very same container
docker start catalog

# Check: the file is still there, because the writable layer was not destroyed
docker exec -it catalog sh -lc "ls -la /tmp/exports"
# catalog.csv is still there

And here the brain says: “Great, so Docker keeps files.” And now we do an action that in container thinking is not considered “extraordinary” at all:

# Delete the container: its writable layer will be deleted together with it
docker rm -f catalog

# Create a new container from the same image and check the directory again
docker run --name catalog ... your-image ...
docker exec -it catalog sh -lc "ls -la /tmp/exports"
# ls: /tmp/exports: No such file or directory

The file is gone. And that is normal behavior. Because the previous container has been destroyed, and with it — writable layer.

It is very important: here we are not saying “containers are bad because they do not store files”. We are saying something else: containers should not silently turn into state storage. State storage is a separate engineering solution. And in Docker it is expressed through mounts (bind mounts, named volumes), which will be covered in the next lecture.

5. Writable layer: temporary, not forever

So far it sounds as if writable layer is pure evil. But that is not true. Writable layer is a completely normal part of the model: without it, a container would not be able to write anything at all (neither temporary files, nor cache, nor even some runtime artifacts). The problem begins when you make the writable layer an invisible storage for a result that matters to the user or the development process.

Imagine that you are exporting a catalog to check it in Postman or through a .http request. You need to open the CSV, look at it with your own eyes, send it to a colleague, attach it to a task, compare it with a reference, or at least simply see that it physically exists on your machine. If the file lives only inside the container, you turn “look at the export” into a quest: first docker exec, then cat, then copy it out (and, of course, forget where exactly). That is not “convenient development”, that is “why is everything so hard”.

There is also a more practical side: the writable layer grows as files are written. If the service writes a lot, the container starts taking up more space, and then you wonder why free disk space disappeared. This is especially fun when you generated dozens of exports “just to try”, and they are sitting in several containers you have already forgotten about.

Therefore the rule is quite simple and very useful: you can write to the writable layer what you can afford to lose. Anything that cannot be lost (or is at least inconvenient to lose) should be moved to a separate storage solution — even if it is just a directory on the host machine mounted into the container.

6. Exporting a catalog as a runtime contract

Our end-to-end project Container-Ready Catalog Service was specifically chosen so that it has a natural file scenario: exporting a catalog to CSV and writing an ExportJob. But in a teaching project, it is important not only to “write the file”, but to make this scenario a contract, that is, something predictable and manageable.

To do this, we introduce an external parameter APP_EXPORT_DIR (it is already included in the project requirements). And the logic here comes entirely from the same principles we discussed for configuration: the same image should work differently in different environments. Today the environment is also the file system. That means the export directory path must be set from the outside, not live as a hardcoded string.

At the plain Java level, this can look like this:

import java.nio.file.Path;

class ExportLocation {
    private final Path directory;

    ExportLocation(String configuredDir) {
        // configuredDir comes from configuration (for example, env var APP_EXPORT_DIR)
        this.directory = Path.of(configuredDir);
    }

    Path resolveFile(String fileName) {
        // Centralize the logic for building the export file path
        return directory.resolve(fileName);
    }
}

And the place where you get configuredDir can already be Spring Boot configuration (env var → property → bean). Without going deep into Spring annotations yet, the meaning is important to us: the export service should not “know” what Docker is and what a mount is. It should only know the path where it is allowed to write.

Even if today you run the container “once and by hand”, tomorrow you may want to move the project to another machine or simply recreate the container for a clean experiment. And then it will turn out that “the export was inside the container”. It is better for this to become clear now, in the lecture, than in three days, when you will be debugging config, logs, and healthcheck at the same time.

7. Demo and preparation for mount

A file “inside” and a file “outside”

Now we will do a very short demonstration, without turning the lecture into a list of Docker flags. The goal is to see with our own eyes that a “file inside a container” lives by the rules of the writable layer. Let us assume that the image of our service has already been built and it knows how to export to some directory.

Let us start the container and perform an export (imagine that we have an endpoint POST /api/catalog/exports that creates a CSV). After that, we will go inside and inspect the directory:

docker exec -it catalog sh -lc "ls -la /app/exports"
# -rw-r--r-- 1 app app  128 Mar 21 10:20 catalog-2026-03-21T10-20-00.csv

Now let us repeat the key psychological trick that breaks the wrong model: instead of “restart”, do “delete”:

docker rm -f catalog

We start the container again (from the same image), and check again:

docker run --name catalog -p 8080:8080 your-image:tag
docker exec -it catalog sh -lc "ls -la /app/exports"
# total 0
# (or the directory is not there at all, depending on whether you create it at startup)

This is where container thinking grows up. The service is “the same”, the image is “the same”, but the state disappeared because the container is new. And this is not a Docker error. This is us trying to silently make the container a storage.

It is useful to pay attention to one more point: sometimes a developer makes life harder for themselves because they keep “processes” in their head instead of “containers”. In a normal JVM on a server, you started one process and “it lives for a long time”. In Docker, a developer recreates containers much more often: because they change config, change environment variables, check different options, clean up the environment. If at that moment an important file is “accidentally” lost, it means file state has not been designed properly.

Separating “where to write” from “what to write”

Now one wants to jump straight to the question “how do we make sure the file does not disappear”. And that is the right desire, but methodologically it is important to take a step back and understand: file resilience does not start with a Docker flag, but with the fact that your code can write to the place it was told to.

If the export service inside itself decides “I always write to /tmp/exports”, you can mount directories however you like — it will hurt. Much better is when the service receives the export directory from the outside and works with it as a dependency. Then the same code works equally well locally, in a container, with a mount, and without it.

The simplest sketch of a service (exactly a sketch, without a full domain and without unnecessary architectural weight):

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;

class CatalogExportService {
    private final Path exportDir;

    CatalogExportService(Path exportDir) {
        this.exportDir = exportDir;
    }

    Path export(String fileName, String csv) throws IOException {
        Files.createDirectories(exportDir);
        Path file = exportDir.resolve(fileName);
        return Files.writeString(file, csv); // returns the path to the created file
    }
}

There is nothing “Docker-specific” here. And that is good. We first make file behavior manageable, and only then (in the next lecture) connect the outside world through a mount and choose a storage strategy. Then Docker becomes a tool, not a lottery.

8. Typical mistakes when working with writable layer

For almost all beginners, the first mistake sounds the same: “I created a file in the container, it is there — so everything is done correctly.” This is a trap of observation: while you test only stop/start/restart, the writable layer behaves like a “normal disk”, and the brain calms down. The problem appears later, with docker rm or container recreation.

Mistake #1: confusing the image file system with the container file system.
Sometimes it seems that if a file appeared “in the container”, then it somehow became part of the image or “will remain in the next run”. But the image is read-only, and the file appeared in the specific container’s writable layer. If the container disappears, the file disappears with it, and that will not be a bug, but a natural consequence.

Mistake #2: treating a container as a small virtual machine with a disk.
In a VM, the “disk” is part of the entity. In a container, the writable layer is a temporary layer on top of the image. In normal engineering practice, a container is easily destroyed and created again. If you design important data as if the container must live forever, you are setting yourself up for a stumble.

Mistake #3: writing important artifacts “wherever”: /tmp, a random /app/out, the user’s home folder inside the container.
The problem is not even the specific directory, but that it was chosen as “randomly convenient”. Important files should have an explicit role (for example, export directory), and the path to them should be part of the runtime contract, meaning it should be set by configuration.

Mistake #4: not asking “who is the consumer of this file”.
If the consumer is only your process “right here and right now”, the writable layer may be sufficient. But if the file is needed by a person on the host machine, by tests, or as the result of a business operation (export), then keeping it “inside the container” means making access harder and increasing the chance of loss when the container is recreated.

Mistake #5: fixing file loss by copying and manual docker exec instead of by design.
Sometimes after the first export loss, there is a temptation: “Okay, I will just copy the file out by hand.” That is how chaos and a manual process appear, which nobody will repeat. It is much better to honestly admit once: if a file must live outside, it needs external storage through a mount. But that is already the topic of the next lecture — we have approached it correctly and without mysticism.

1

Zadanie

Docker for Spring, poziom 13, lekcja 0

Niedostępne

Plik wewnątrz kontenera żyje tylko do momentu usunięcia kontenera

Plik wewnątrz kontenera żyje tylko do momentu usunięcia kontenera

Otwórz

1

Zadanie

Docker for Spring, poziom 13, lekcja 0

Niedostępne

Jeden obraz, dwie różne warstwy zapisywalne

Jeden obraz, dwie różne warstwy zapisywalne

Otwórz