Re-encoding files: reading in one encoding, writing in another

1. Introduction

Imagine you received a report file from an accountant that was saved in "Windows-1251" (an old encoding for Cyrillic). Your Java application expects all input data in "UTF-8"; otherwise, it starts “complaining” and shows gibberish instead of letters. Or, for example, you integrate with a system that accepts only "ISO-8859-1". What should you do? Of course, re-encode the file!

Re-encoding is the process in which you read text from a file in one encoding and save it in another encoding. It’s like rewriting a letter from one language into another so the recipient clearly understands your message.

How re-encoding works in Java: the general approach

In Java, strings (String) inside your program are always stored in Unicode (UTF‑16). This means that once you’ve read the text from a file, it’s already “universal” and can be written in any supported encoding. Therefore, re-encoding is simply:

• Read the file as text, specifying the correct source encoding.
• Write those lines to a new file, explicitly specifying the desired target encoding.

Diagram:

[File in encoding A] --(read with Charset A)--> [String in memory] --(write with Charset B)--> [File in encoding B]

2. Step-by-step re-encoding algorithm

Step 1. Determine the source and target encoding
The source encoding is the one in which the original file was saved (for example, "Windows-1251"). The target encoding is the one you want for the result (for example, "UTF-8").

Step 2. Open a reader with the required encoding
Use Files.newBufferedReader(Path, Charset) or the classic InputStreamReader.

Step 3. Open a writer with the required encoding
Use Files.newBufferedWriter(Path, Charset) or the classic OutputStreamWriter.

Step 4. Read lines and write them to a new file
Read line by line (or the whole file if it’s small) and write each line to the new file.

Step 5. Close streams (prefer try-with-resources)
This guarantees proper resource cleanup. Use the try-with-resources construct.

3. Code example: re-encoding Windows-1251 → UTF-8

Let’s implement a simple program that re-encodes a file from Windows‑1251 to UTF‑8. This example is common in real life, especially if you work with Russian-language data.

import java.io.*;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;

public class FileReencoder {
    public static void main(String[] args) throws IOException {
        // File paths
        Path inputPath = Paths.get("input-1251.txt");
        Path outputPath = Paths.get("output-utf8.txt");

        // Open a reader with the source encoding Windows-1251
        try (
            BufferedReader reader = Files.newBufferedReader(inputPath, Charset.forName("Windows-1251"));
            BufferedWriter writer = Files.newBufferedWriter(outputPath, StandardCharsets.UTF_8)
        ) {
            String line;
            while ((line = reader.readLine()) != null) {
                writer.write(line);
                writer.newLine(); // don't forget the line break!
            }
        }

        System.out.println("The file has been successfully re-encoded from Windows-1251 to UTF-8!");
    }
}

Here we explicitly specify the encoding for reading (Charset.forName("Windows-1251")) and for writing (StandardCharsets.UTF_8). Then we use try-with-resources so the streams close automatically even if an error occurs. writer.newLine() is a line break to preserve the file’s structure.

4. Important nuances and tips

How do you determine a file’s source encoding?

• A file generally doesn’t carry a “label” with its encoding (an exception is BOM, but it’s not always present).
• If you created the file, use the same encoding you used when writing it.
• If the file came from “somewhere,” try opening it in an editor that shows the encoding (for example, Notepad++, Visual Studio Code).
• On Linux you can use the command file -i filename, but it doesn’t always detect the encoding correctly.

What happens if you specify the wrong source encoding?

You’ll get gibberish or lose characters. For example, if you read a file saved in Windows-1251 as UTF-8, Cyrillic will turn into "Привет".

Exception handling

When working with files, errors are always possible: file not found, insufficient permissions, wrong encoding. Use exception handling (try-catch) or throw them up the stack (throws) so the application doesn’t crash “without explanation.”

Working with large files

If a file is huge (gigabytes!), use line-by-line reading and writing as in the examples. Don’t read the entire file into memory at once — otherwise you’ll get an OutOfMemoryError.

On-the-fly re-encoding (streaming)

If the file is very large, you can even avoid creating an intermediate file and re-encode “on the fly” — for example, when streaming data from the network.

5. Common mistakes when re-encoding files

Error #1: Wrong source encoding. If you get the source encoding wrong, the result will be unfortunate: “Hello” will turn into "Привет". Always determine which encoding the file was created with.

Error #2: Implicit use of the system default encoding. If you don’t specify the encoding explicitly, Java uses the system default (System.getProperty("file.encoding")). This can lead to different results on different machines (for example, on Windows — cp1251, on Linux — UTF‑8).

Error #3: Reading and writing the entire file in memory. For large files, this approach leads to memory exhaustion. Use line-by-line reading and writing.

Error #4: Unhandled exceptions. Files may not exist, may be locked by other processes, or may contain broken characters. Always handle exceptions or use try-with-resources.

Error #5: Re-encoding binary files. Don’t try to re-encode images, PDFs, archives, and other binary files! This will corrupt them. Re-encoding only makes sense for text files.