Deeper into the binary format and its problems

1. Introduction

In short: a binary serialization format turns an object into a sequence of bytes that encode its structure and values as compactly as possible. Imagine you're not describing an object in words (like JSON or XML), but writing every bit exactly as it's stored in memory.

In a text format the data is like a letter to a friend in Russian (each character is understandable to a person). In a binary format it's more like Morse code, where every dot and dash is written as compactly as possible, and you can't really read it "by eye".

Scheme: comparing formats

How does binary serialization work in .NET?

In the .NET ecosystem the historical go-to tool for binary serialization was the BinaryFormatter class. But as the platform evolved it was deemed unsafe and removed from .NET 9. Today the standard approaches are different: BinaryWriter/BinaryReader, and for complex objects — third-party libraries (for example, protobuf-net).

Brief history (historical excursus)

BinaryFormatter could take any class marked with the [Serializable] attribute and turn it into bytes, and upon deserialization reconstruct the object graph. Sounds magical, but that magic hid a ton of problems (more on that below).

Modern tools

For primitive types and simple structs it's convenient to use BinaryWriter and BinaryReader. For complex objects — third-party libraries (for example, protobuf-net, MessagePack-CSharp, etc.).

2. Serializing primitives with BinaryWriter

Let's continue improving our educational app. For example, we want to write user settings (user name, score, login time) to a binary file.

public class UserProfile
{
    public string Name { get; set; }
    public int Score { get; set; }
    public DateTime LoginTime { get; set; }
}

public static Task SaveUserProfile(UserProfile profile, string filePath)
{
    // Open the file for writing
    using var stream = new FileStream(filePath, FileMode.Create, FileAccess.Write, FileShare.None);
    using var writer = new BinaryWriter(stream);

    // Write data piece by piece. First the string, then the number, then the date
    writer.Write(profile.Name ?? string.Empty); // string
    writer.Write(profile.Score);                // integer
    writer.Write(profile.LoginTime.ToBinary()); // date converted to "long"
}

Now the reading example:

public static Task<UserProfile> LoadUserProfile(string filePath)
{
    using var stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read);
    using var reader = new BinaryReader(stream);

    string name = reader.ReadString();
    int score = reader.ReadInt32();
    long dateData = reader.ReadInt64();
    DateTime loginTime = DateTime.FromBinary(dateData);

    return new UserProfile { Name = name, Score = score, LoginTime = loginTime };
}

Features of primitive binary serialization

With BinaryWriter we serialize each property separately. This is a reliable and predictable approach: if the data structure changes, you see it in the code.

3. Problems of classic binary serialization

Now let's look at the flip side. Why did Microsoft so aggressively discourage BinaryFormatter and even ban its use?

Format brittleness (Schema Evolution Hell)

Binary data is tightly coupled to the class layout. Change the class (rename a field, add a new one, remove one) — old binary files become unreadable. Change the order of fields — also a problem.

Illustration:

// Yesterday
public class Profile
{
    public string Name;
    public int Score;
}

// Today
public class Profile
{
    public string Name;
    public double Rating; // A new field was added
    public int Score;
}

Reading an old file will either throw an error or read fields with "mixed-up" data. Unlike JSON or XML, where missing elements can be skipped, binary format doesn't adapt to changes — it's like a solid concrete road: step off it and you immediately "fall off the bike".

Deserialization vulnerabilities

The biggest problem of BinaryFormatter is potential vulnerability. If your software deserializes binary data received from an untrusted source (for example, from a user over the internet), an attacker can plant a malicious "object". Historically this even led to remote execution of arbitrary code on the victim's machine.

Cross-platform and compatibility

A binary serializer is tightly bound to the internal representation of data in .NET and the current runtime version, compiler, and architecture (for example, x64/ARM). If you serialize on Windows and try to deserialize on Linux — surprises are guaranteed! Even between .NET versions incompatibilities can appear.

Diagnostic inconvenience

When text formats have problems you can open the file, look at the contents, and guess what went wrong. A binary file is a sealed mystery. All you see is a meaningless stream of bytes. "Analyzing" such a file is for enthusiasts.

4. Binary serialization of complex objects

References

BinaryFormatter could remember object references (for example, if two properties reference the same object), but BinaryWriter and most third-party libraries don't have that magic. Usually serialization is done by "embedding one object into another and writing them sequentially".

Cyclic references

Serializing objects with cyclic references (for example, a "mother" has a Child property, and the "child" has a Parent property pointing back to the parent) either throws an error or leads to an infinite loop.

Example:

public class Node
{
    public Node? Next { get; set; }
    public Node? Prev { get; set; }
}

Attempting to serialize this object "naively" will cause recursion.

5. Binary serialization and portability

Any binary format (especially a homemade one) is a "for insiders only" format. If you plan to exchange data with other programs or keep it "for ages" — choose open standards: JSON, XML, or ProtoBuf.

When is binary serialization justified?

• If the data lives within a single application and is stored "for a short time".
• If speed and compactness matter (for example, for large logs or exchange between services inside one ecosystem).
• If you strictly control both sides: serialization and deserialization.

Alternatives: protobuf, MessagePack and others

• protobuf-net: a port of Google Protocol Buffers for .NET, suitable for cross-platform exchange and compatibility.
• MessagePack-CSharp: a fast implementation of MessagePack for .NET.

Unlike "raw" BinaryWriter, these libraries implement schemas, support format evolution, cross-platform compatibility, and safety. Use them if you plan any compatibility with other systems.

6. "Manual" binary serialization

If you still need to write binary data (for apps where performance matters), use BinaryWriter/BinaryReader — and always explicitly encode the order and data types.

Tips:

• Always write data in the same order you plan to read it.
• When changing file structure include a version number or write a "magic header" (Magic Header).
• Write lengths of strings/arrays before the actual data.
• Document the file structure: otherwise in a year you won't understand your own format.

Example: versioning

// Write the format version number first
writer.Write((byte)1); // Version 1

writer.Write(profile.Name ?? "");
writer.Write(profile.Score);
writer.Write(profile.LoginTime.ToBinary());

/*
Allows you to add conditional reading logic when the format changes later
*/

7. Common mistakes when working with binary serialization

You encoded fields in one order, but read them in a different order. As a result values "shift": a string is read as an int, an int as a date, etc.

You wrote 10 objects but read 11. The stream is broken: you'll get an end-of-file exception.

You changed the class structure and old binary files can't be read — you lose all historical data.

You forgot to handle exceptions when reading an important file — the app crashes at the first disk hiccup (for example, EndOfStreamException).

Trying to exchange binary files between different programming languages without an explicit format — in 99% of cases this is guaranteed pain.

Deserializing data received over the network from unknown users — hello, vulnerabilities! Never use BinaryFormatter; validate input and use safe formats.