Encoding mismatches and common errors

1. Symptoms of errors

In a perfect world, developers always know which encoding a file uses and specify it correctly when reading. In reality, files travel between Windows, Linux, servers, and editors, each interpreting bytes in its own way. As a result, we encounter symptoms like these:

• “Mojibake” — instead of the expected text, you see strange characters, question marks, little squares, or a jumble of letters that looks like no language at all.
• Character loss — parts of the text disappear or are replaced with ?.
• Exceptions — for example, MalformedInputException when Java cannot interpret the bytes under the chosen encoding.
• Parsing errors — the program cannot process the file correctly because keywords or structures are corrupted due to text garbling.

Here is a classic example of “mojibake” when reading a Cyrillic file using the wrong encoding:

Expected:  Hello, world!
Received: Привет, мир

This is not a new language but the result of interpreting bytes with the wrong “dictionary”.

2. Why errors occur: the root cause

The file is saved in one encoding but read in another

Suppose someone saved the file in Windows-1251, and you open it as UTF-8. Java faithfully tries to decode the bytes according to UTF-8, but you get nonsense because the byte values don’t match what UTF‑8 expects.

Relying on the system default encoding

If you don’t specify an encoding explicitly, Java uses the system default — whatever is set on your computer. On Windows with a Russian locale this might be Windows-1251, on Linux — UTF-8, on Mac — also UTF-8. A file that opens fine for you may become unreadable for a colleague on a different OS.

Using legacy constructors

In older Java versions (and in some textbooks) you often see constructs with FileReader/FileWriter that use the system encoding and give you no control — this is a trap and a common source of “mojibake”.

FileReader reader = new FileReader("file.txt");
FileWriter writer = new FileWriter("file.txt");

Presence or absence of a BOM (Byte Order Mark)

Some encodings (for example, UTF-8 with a BOM or UTF-16) add special bytes to the beginning of a file to signal their nature. If a program does not expect a BOM or, conversely, expects one but doesn’t find it, problems can arise: either the first characters of the file are mangled, or the file isn’t recognized at all.

3. How errors show up: practical examples

Example 1: A Cyrillic file saved in Windows-1251 is read as UTF-8

import java.nio.file.*;
import java.nio.charset.*;

public class EncodingDemo {
    public static void main(String[] args) throws Exception {
        Path path = Paths.get("russian.txt");
        // The file is saved in Windows-1251, we read it as UTF-8 — mojibake ahead!
        try (BufferedReader reader = Files.newBufferedReader(path, StandardCharsets.UTF_8)) {
            System.out.println(reader.readLine());
        }
    }
}

As a result, instead of “Hello, world!” you will see a bunch of strange characters.

Example 2: The file is saved in UTF-8 but read as ISO-8859-1

try (BufferedReader reader = Files.newBufferedReader(path, StandardCharsets.ISO_8859_1)) {
    System.out.println(reader.readLine());
}

Result: All non-ASCII characters will turn into garbage or be replaced with ?.

Example 3: An exception when reading a file

If the bytes don’t follow the rules of the chosen encoding, Java may throw an exception:

Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1
    at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
    ...

This means Java encountered a byte that cannot be correctly interpreted under the chosen encoding.

4. Diagnostics: how to figure out what’s wrong with the encoding

Check the file encoding

• In editors like Notepad++, VS Code, or Sublime Text you can usually view or change a file’s encoding (often in the status bar).
• On Linux, a command can give you a hint about the encoding (though it isn’t 100% accurate):

file file_name.txt

Check Java’s system encoding

Print the value of the file.encoding system property to the console:

System.out.println(System.getProperty("file.encoding"));

Use test data

Create a small file with various characters (Cyrillic, Latin, special symbols, emoji), try reading it with different encodings, and see when the result matches your expectations.

Always specify the encoding explicitly

Whenever you see file I/O without an explicit encoding, be cautious. For example, use Files.newBufferedReader(..., StandardCharsets.UTF_8) instead of relying on “defaults”.

5. Best practices: how to stay out of trouble

Rule No. 1:
ALWAYS specify the encoding explicitly when working with files, especially if the file will be used on different computers, in different OSs, or sent over the network.

Rule No. 2:
Use modern, widely supported encodings — primarily UTF-8 (StandardCharsets.UTF_8). Only use other encodings if there are specific requirements (for example, integration with a legacy system).

Rule No. 3:
Avoid the FileReader and FileWriter classes (they don’t let you specify an encoding). Instead, use InputStreamReader, OutputStreamWriter, or Files methods with an explicit Charset.

Rule No. 4:
Verify the result! Open the files you’ve written in editors that support different encodings to make sure the text looks correct.

6. Edge cases and nuances: BOM, XML, JSON, and other “fun” cases

BOM (Byte Order Mark): sometimes a UTF-8 file starts with “invisible” bytes (EF BB BF). Most modern programs ignore them, but some may show “mojibake” at the start of the first line or reject the file (for example, older XML/JSON parsers).

XML/HTML: sometimes at the beginning of a file there’s a line like <?xml version="1.0" encoding="UTF-8"?>. It tells the program which encoding to expect. But if the actual encoding doesn’t match the declaration, you get “mojibake” again.

JSON: by standard it should be UTF-8, but if the file is created in Windows-1251, the parser will throw an error or produce garbled data.