Busy-wait vs blocking

1. Introduction

When you start writing multithreaded code, you quickly discover that threads not only "run in parallel" but also interfere with each other. Often one thread cannot proceed until another has done its part: prepared data, pushed a task into a queue, or flipped a "can start" flag. And here the question arises: how should a thread wait properly?

In real programs waiting happens all the time. A worker thread waits for new tasks. A logger thread waits for messages to write. A network service thread waits for incoming requests. Even if you are not writing a server yet and are making a study program, the model is the same: "while there is no work — you must wait, but don't break the computer (and your nerves)".

For this lecture we need to distinguish two approaches:

• Busy-wait / polling: "I loop and check whether it's good yet".
• Blocking wait: "I sleep and wake up only when there is a chance it became good".

Today we will aim at motivation: why the "spinner" is almost always a bad idea, even if it looks simple.

Busy-wait: a simple "spinner" that hurts

Busy-wait is when a thread literally does: "checked condition → not ready → checked again → …". At first glance this looks like a perfect solution: no libraries, no complex objects, everything is transparent. But this transparency has a side effect: your CPU starts feeling like a fan.

Let's look at a minimal example:

#include <iostream>

bool ready = false;

void spin_wait_bad() {
    while (!ready) {
        // busy-wait: do nothing, just spin
    }
    std::cout << "Ready!\n"; // Ready!
}

In a single-threaded program this is "just strange". In multithreading it's worse: the variable ready becomes a shared state, and if one thread writes it while another reads it without synchronization, you enter the "you must not do this" zone (data race). Even if "it works on my machine", that does not make the code correct.

But even if we momentarily forget about correct data access, busy-wait has a purely practical problem: the thread does not sleep. It consumes CPU, performs millions of checks per second and wins the grand prize: "Award for the most useless diligence".

To feel this, you can make a small demo with an "empty turns" counter:

#include <iostream>

bool ready = false;

void spin_wait_with_counter() {
    long long spins = 0;
    while (!ready) {
        ++spins;
    }
    std::cout << "Spins = " << spins << '\n'; // e.g.: Spins = 154238910
}

If ready becomes true after 1–2 seconds, spins may grow to huge values. Those are "wasted CPU cycles".

2. Why busy-wait is bad (even if "it works")

It's tempting to think: "so what, it spun a bit". The problem is that "a bit" in multithreading quickly becomes "constantly". Then a cascade of unpleasant effects starts: from extra load to unpredictable behavior on weak machines and laptops.

First, busy-wait eats CPU. If one thread spins in an empty loop, it can occupy a whole core. On a quad-core machine that might be acceptable, but on a dual-core machine it's already sad: one thread "spins", the other "tries to work", the OS starts switching them, and you get less benefit than expected.

Second, it's bad for power consumption. On a laptop and especially on mobile devices a "spinner" means "I don't let the CPU sleep", so the battery dies faster. The user sees no result — they just hear the fan.

Third, busy-wait usually leads to poor synchronization design. A beginner often starts with a shared flag without a mutex, then adds a second flag, then a third, and then you get "if this flag is true but that hasn't been updated yet…". The code becomes a fragile system of "magic states" where one missed transition breaks everything.

And finally, the most important (and most unpleasant): in multithreading we must not "just read a shared flag as a regular variable". Shared data requires disciplined access. If you are already using std::mutex, you implicitly agree: "this is shared state; touch it only under the lock". Busy-wait in the style of while(!ready){} usually ignores that contract.

4. Polling with sleep_for: fixing the spinner, but it's a compromise

After the first encounter with busy-wait the idea usually arises: "Okay, I'll add sleep_for and the thread will rest". That's better than a pure spinner, but here you get the classic trade-off: either you still wake up often (load the system), or you wake up rarely (lose reactivity).

First, make polling at least correct in access to the shared flag: read ready under a mutex.

#include <chrono>
#include <mutex>
#include <thread>

std::mutex m;
bool ready = false;

void polling_with_sleep_still_not_great() {
    while (true) {
        {
            std::lock_guard<std::mutex> lock(m);
            if (ready) break;
        }
        std::this_thread::sleep_for(std::chrono::milliseconds{10});
    }
}

Now reading ready is protected. But design problems remain:

• If you set sleep_for(1ms), you will wake up often and still create extra load, especially if several waiting threads exist.
• If you set sleep_for(100ms), the load is lower — but the reaction to an event may be delayed up to 100 ms. For a UI this can be noticeable, and for a task processing system it turns into "weird pauses" for no reason.

And that's not counting that sleep_for is not a promise of "exactly 10ms", but a request to the OS "let me sleep at least roughly that long". On a busy system it can be longer.

So sleep_for-polling is a band-aid: sometimes fine for demos, but dubious as the main waiting mechanism.

5. The correct model: wait for state, notify is a reason to recheck

Here it's important to change the mental picture. In multithreading we almost never should wait for a "signal as an event". We wait for a state that can be checked right now: "queue is not empty", "shutdown flag is set", "there is at least one token", "data is ready".

A notification (signal) in good synchronization primitives is not a "remembered event" but merely a phrase: "hey, check again, something might have changed". Therefore a correct protocol is usually: a thread sleeps when the state is not suitable, and wakes up to recheck the state.

The C++ standard library provides a mechanism for precisely this model: std::condition_variable, and it has a waiting form "wait until a predicate becomes true" — wait(lock, predicate). We'll study it in detail in the next lecture; for now it's enough to grasp the principle: waiting should be tied to a checkable state, not to a "magical signal".

For clarity you can imagine it like this:

flowchart TD
    A[Thread wants to continue working] --> B{Is the state "ready"?}
    B -- yes --> C[Continue working]
    B -- no --> D[Go to wait]
    D --> E[Woke up: "check again"]
    E --> B

Note there is no idea of "a signal being stored" in this diagram. There is only "state" and the recheck loop.

6. Practical example: task handler with polling and why it's inconvenient

So our examples are not random fragments, let's set up a simple study story. Imagine a "mini-service": the main thread pushes tasks (numbers), and a worker thread "processes" these numbers (for simplicity — prints and sleeps). The task queue is a shared resource.

Without a condition_variable (that's the next lecture), let's make a polling version to feel the drawbacks.

Shared state skeleton:

#include <mutex>
#include <queue>

std::mutex m;
std::queue<int> tasks;
bool stop = false;

Function to "push a task":

#include <mutex>
#include <queue>

extern std::mutex m;
extern std::queue<int> tasks;

void push_task(int x) {
    std::lock_guard<std::mutex> lock(m);
    tasks.push(x);
}

And now the worker that periodically checks the queue:

#include <chrono>
#include <iostream>
#include <mutex>
#include <queue>
#include <thread>

extern std::mutex m;
extern std::queue<int> tasks;
extern bool stop;

void worker_polling() {
    while (true) {
        int item = 0;
        bool has_item = false;

        {
            std::lock_guard<std::mutex> lock(m);
            if (stop) return;
            if (!tasks.empty()) {
                item = tasks.front();
                tasks.pop();
                has_item = true;
            }
        }

        if (has_item) {
            std::cout << "Processed " << item << '\n'; // Processed 10
        } else {
            std::this_thread::sleep_for(std::chrono::milliseconds{10});
        }
    }
}

This code is already better than busy-wait: it doesn't burn a core at 100%. But you see the main downside: when there are no tasks the worker becomes a "periodically waking guard": "checked — empty — slept — checked — empty …". And you inevitably choose: either a small sleep (less latency, more extra wakeups), or a large sleep (more latency, fewer wakeups).

Now imagine there are many tasks and you want the worker to react instantly. You will have to reduce the sleep. But then you return to high load. This is why polling is a compromise, while blocking wait is the normal engineering answer.

7. Comparison of approaches and where busy-wait is encountered

Comparison table

When a beginner reads about busy-wait, polling and blocking, everything can blur. It's helpful to see a direct comparison once: not by "API beauty" but by program behavior.

In this table the point is not that busy-wait is "bad because it's bad". The point is that it is bad by default: in typical application tasks it hurts the system and provides no real advantage.

When busy-wait is still used

Sometimes someone will say: "But I've heard spin-wait is used in high-performance systems!" Yes, it is used. Usually there are other conditions: waits are guaranteed to be very short, threads are pinned to cores, the architecture is carefully measured, and the code author understands contention and why it happens.

In teaching and most application programs this is almost never your case. Your case is a task queue, waiting for I/O, waiting for data from another thread, waiting "until something appears". That can last milliseconds, seconds, or minutes. Spinning the CPU for a minute is not "a few extra instructions", it's a philosophy of "let the computer suffer".

Therefore we need a mechanism that allows a thread to sleep until the state changes. That is exactly what the next lecture about std::condition_variable will cover.

To fix the idea, here's how the "blocking wait idea" looks (without details — we'll cover them later):

#include <condition_variable>
#include <mutex>

std::mutex m;
std::condition_variable cv;
bool ready = false;

void blocking_wait_idea() {
    std::unique_lock<std::mutex> lock(m);
    cv.wait(lock, [] { return ready; }); // wait for state ready==true
}

Here the thread does not spin and does not "tick" every 10ms. It waits until it is woken up, and only then continues.

8. Common mistakes when waiting in threads

Mistake #1: treating sleep_for as "synchronization".
A very common logic: "if I sleep, the other thread will finish". In multithreading this is not a contract but divination. sleep_for only pauses the current thread, but does not guarantee that the other thread did the needed work, and certainly does not guarantee correct access to shared data.

Mistake #2: making a shared flag without any discipline of access.
Beginners often write a bool ready that one thread changes and another reads. Without a mutex (or another coordination mechanism) this becomes a data race. Even if you "just read a bool", it's still shared state and the rules are as strict as for a queue or vector.

Mistake #3: holding a mutex while "waiting via sleep".
Sometimes code takes a lock_guard, checks a condition, and then inside the same scope does sleep_for. The result is that the thread "fell asleep but didn't release the lock". Other threads cannot change the state, and the wait becomes a self-block (mild) or a deadlock (severe).

Mistake #4: confusing "signal" and "state".
Even when you start using condition_variable, beginners want to "catch notify". But the correct logic is "I check the state; the notification only says: recheck". If you build a protocol around an event that can "fly by", sooner or later you'll get a hang that reproduces only on Friday afternoons (the most mystical category of bugs).

Mistake #5: using too small sleep_for, turning polling back into busy-wait.
It may seem that sleep_for(1ms) is "almost free". In practice it often means very frequent wakeups, high load and degraded performance on weak systems. Polling with micro-sleeps is sometimes worse than honest busy-wait because it looks "polite" but still consumes resources, just less obviously.