Core encodings: UTF-8, UTF-16, ISO-8859-1

1. UTF-8 — the king of modern encodings

In your work as a programmer you will deal with files, network protocols, databases, and web services where different encodings are used. If you don’t know how UTF-8 differs from ISO-8859-1, it’s easy to fall into a trap: your code will work “on my machine” but break for a colleague in France or in the cloud where a different system default is used.

Today we’ll cover three main encodings that appear in a Java developer’s real life:

• UTF-8 — a modern universal standard.
• UTF-16 — the internal string format in Java and often seen on Windows.
• ISO-8859-1 (Latin-1) — an old but still encountered single-byte encoding for Western European languages.

Let’s take a closer look!

What is it?

UTF-8 (Unicode Transformation Format, 8-bit) is an encoding that can represent any character from the huge Unicode table (including Cyrillic, Chinese ideographs, emoji, and even rare ancient scripts). It does this cleverly: for the most common characters (English letters, digits, punctuation) it uses just 1 byte. For others — 2, 3, or even 4 bytes.

Fact: All modern websites, most JSON and XML files, and even Java source files by default use UTF-8.

How many bytes per character?

• ASCII characters (English letters, digits, punctuation) — 1 byte.
• Cyrillic characters, European letters with diacritics — 2 bytes.
• Chinese, Japanese, Korean ideographs — 3 bytes.
• Super-exotic characters, emoji — 4 bytes.

Example:

U+0041

41

U+042F

D0 AF

U+20AC

E2 82 AC

U+1F600

F0 9F 98 80

Why is UTF-8 great?

• Backward compatible with ASCII: if a file contains only English characters, it looks exactly the same as in ASCII.
• Compactness: for short English texts, a UTF-8 file will be minimal in size.
• Internationalization: supports all languages of the world, including emoji.
• Web standard: HTML, CSS, JSON, XML, JavaScript — all of this is UTF-8 by default.

Example usage in Java

import java.nio.charset.StandardCharsets;
import java.nio.file.*;

public class Utf8Demo {
    public static void main(String[] args) throws Exception {
        // Write a string to a file in UTF-8
        String text = "Hello, world! 😀";
        Files.write(Paths.get("utf8.txt"), text.getBytes(StandardCharsets.UTF_8));

        // Read a file in UTF-8
        String read = Files.readString(Paths.get("utf8.txt"), StandardCharsets.UTF_8);
        System.out.println(read); // Will print: Hello, world! 😀
    }
}

Here we first explicitly encode the string in UTF-8 before writing, and then explicitly specify the encoding when reading. Thanks to this, the text is preserved and restored without corruption, even if it contains emoji or characters from different languages.

When to use UTF-8?

Always, unless you have a very strong reason to choose something else. It’s the universal choice for all modern applications.

2. UTF-16 — Java’s internal format

UTF-16 is another encoding in the Unicode family. Unlike UTF-8, it most often uses 2 bytes per character. This is how Java stores strings (String) and characters (char) inside the JVM. But there’s a nuance: some characters (for example, rare ideographs or emoji) require 4 bytes (two “surrogate pairs”).

Where is UTF-16 used?

• Inside Java: all strings and chars are UTF-16.
• Windows: many system files on Windows use UTF-16 (for example, Notepad saves as UTF-16 LE by default).
• Some network protocols and formats: for example, XML can be in UTF-16.

Example: how a character looks in UTF-16

U+0041

00 41

U+042F

04 2F

U+20AC

20 AC

U+1F600

D8 3D DE 00

Example usage in Java

import java.nio.charset.StandardCharsets;
import java.nio.file.*;

public class Utf16Demo {
    public static void main(String[] args) throws Exception {
        String text = "Hello, world! 😀";

        // Write a string to a file using UTF-16
        Files.write(Paths.get("utf16.txt"), text.getBytes(StandardCharsets.UTF_16));

        // Read file contents specifying UTF-16
        String read = Files.readString(Paths.get("utf16.txt"), StandardCharsets.UTF_16);
        System.out.println(read); // Correct output of the original string
    }
}

Key point: you need to specify the same encoding for both writing and reading. If you write in UTF-16 but read using the default (for example, UTF-8), you’ll get a jumble of characters.

UTF-16 specifics

• Fixed length for most characters: 2 bytes.
• Surrogate pairs: some characters (e.g., emoji) require 4 bytes.
• Byte Order Mark (BOM): a special “header” is often added at the beginning of a file indicating byte order (LE — little endian, BE — big endian). Java usually understands the BOM automatically.
• Not compatible with ASCII: if you open a UTF-16 file in an editor expecting ASCII or UTF-8, you’ll see lots of zeros and strange symbols.

When to use UTF-16?

• If you work with very specific systems or files that require UTF-16 (for example, integration with Windows programs).
• For ordinary applications and files — prefer UTF-8.

4. ISO-8859-1 (Latin-1) — a greeting from the past

ISO-8859-1, or Latin-1, is a single-byte encoding created back in the distant 1980s. It supports 256 characters: the English alphabet, Western European letters with diacritics (é, ü, ç, etc.), punctuation, and special symbols.

Fact: This encoding does not include Cyrillic, Greek, Arabic, Chinese, or other non-Latin scripts.

Where does ISO-8859-1 appear?

• In old files and programs (especially in Europe).
• In some databases and protocols as the “default”.
• In HTTP headers (by standard, textual data without an explicit charset is considered ISO-8859-1, though in practice this is rare).

Example: how a character looks in ISO-8859-1

U+0041

41

U+00E9

E9

U+00FC

FC

U+042F

Example usage in Java

import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.nio.charset.Charset;

public class Latin1Demo {
    public static void main(String[] args) throws Exception {
        String text = "Bonjour, ça va? Café!"; // French text
        Files.write(Paths.get("latin1.txt"), text.getBytes(StandardCharsets.ISO_8859_1));

        // Read a file in ISO-8859-1
        String read = Files.readString(Paths.get("latin1.txt"), StandardCharsets.ISO_8859_1);
        System.out.println(read); // Everything reads correctly!

        // Try to write Cyrillic
        String cyrillic = "Hello, world!";
        try {
            Files.write(Paths.get("badlatin1.txt"), cyrillic.getBytes(StandardCharsets.ISO_8859_1));
        } catch (Exception e) {
            System.out.println("Error: unable to write Cyrillic in ISO-8859-1!");
        }
    }
}

The ISO-8859-1 (Latin-1) encoding supports only Western European characters — for example, accented letters from French, German, or Spanish. Therefore, French text is written and read without issues.

However, Cyrillic is absent in this encoding, and attempts to write it result in an error or character loss. That’s why for multilingual texts it’s better to use UTF-8 or UTF-16, which cover a much larger character set.

ISO-8859-1 specifics

• Language limitation: Western European languages only.
• 1 byte per character: compact but very limited.
• Writing “foreign” characters causes corruption: if you try to save Cyrillic or ideographs, they turn into question marks or garbage.

When to use ISO-8859-1?

• Only if you integrate with a very old system or database that specifically requires this encoding.
• Not recommended for modern tasks.

5. Useful nuances

Encoding comparison table

UTF-8

UTF-16

ISO-8859-1

How to choose an encoding?

• UTF-8 — your choice for 99% of tasks. Compatible with ASCII, supports all languages, minimal size for English text, the standard for the web and Java.
• UTF-16 — use only if required by a specification or integration with Windows programs.
• ISO-8859-1 — for backward compatibility with old systems only. Never use it to store Cyrillic, Greek, Arabic, etc.

How to determine a file’s encoding?

• Open the file in an editor that shows the encoding (for example, Notepad++, VS Code).
• If you see garbled characters (mojibake), try opening the file with a different encoding.
• In a Linux terminal, you can use the file command with a filename — it sometimes suggests the encoding.

How to set the encoding in Java?

Use classes from the java.nio.charset package:

• StandardCharsets.UTF_8
• StandardCharsets.UTF_16
• StandardCharsets.ISO_8859_1

Or by charset name:

Charset windows1251 = Charset.forName("Windows-1251");
Charset utf8 = Charset.forName("UTF-8");

6. Practice: what happens if you mix up the encoding?

Let’s try a small experiment. We’ll write a string in UTF-8, then try to read it as ISO-8859-1:

import java.nio.file.*;
import java.nio.charset.*;

public class EncodingMismatchDemo {
    public static void main(String[] args) throws Exception {
        String text = "Hello, world!"; // Cyrillic

        // Write as UTF-8
        Files.write(Paths.get("utf8demo.txt"), text.getBytes(StandardCharsets.UTF_8));

        // Read as ISO-8859-1 (INCORRECT!)
        String wrong = Files.readString(Paths.get("utf8demo.txt"), StandardCharsets.ISO_8859_1);
        System.out.println("Incorrect reading: " + wrong);

        // Now read correctly
        String correct = Files.readString(Paths.get("utf8demo.txt"), StandardCharsets.UTF_8);
        System.out.println("Correct reading: " + correct);
    }
}

Result:

• In the first case you will see garbled characters — a set of strange symbols.
• In the second case — normal Russian text.

7. Common mistakes when working with encodings

Error #1: Reading a file with the wrong encoding. If a file is written in UTF-8 and you read it as ISO-8859-1 or Windows-1251, all non-ASCII characters will turn into garbage. This is a classic: “it works on my machine, but not on a colleague’s”.

Error #2: Attempting to write “foreign” characters in a single-byte encoding. If you try to save Cyrillic or ideographs in ISO-8859-1, Java will replace them with question marks or throw an error. The data will be lost irreversibly.

Error #3: Using the system “default” encoding. In some Java methods (for example, new FileReader("file.txt")) the encoding isn’t specified explicitly — the system default is used. On one computer this might be UTF-8, on another — Windows-1251, on a third — something exotic. Therefore, always use methods where you can explicitly specify the encoding.

Error #4: Opening a UTF-16 file as UTF-8. A file written in UTF-16 will show lots of zeros and strange characters if opened as UTF-8, because the bytes will be interpreted incorrectly.

1

Task

JAVA 25 SELF, level 37, lesson 1

Locked

Attempt to send Cyrillic through an antiquated communication system 📟

Attempt to send Cyrillic through an antiquated communication system 📟

Open

1

Task

JAVA 25 SELF, level 37, lesson 1

Locked

Linguistic experiment: The weight of words in different encodings ⚖️

Linguistic experiment: The weight of words in different encodings ⚖️

Open