Multithreading

1.1 The Origins

Humanity often comes up with the idea to create a new grand project, one that will outshine all previous monumental projects. The Pyramid of Khufu in Giza — the largest, Burj Khalifa in Dubai — the tallest, and the Great Wall of China — the longest.

However, organizing work on such projects is very challenging. If you were to build a new pyramid twice as tall, it would require eight times more stone. That means either increasing the productivity of quarries, or opening more of them.

You'd also need to find eight times more engineers, possibly open engineering schools, standardize drawings, geometry, writing. Basically, a real challenge...

Thousands of years since the pyramids were built, nothing has really changed — people still think about how to get more work done in less time. And when computers were invented, the best minds of humanity started tackling this problem.

How do you run a program ten times faster? Seems like a weird question — if the processor is already running at full speed, then maybe you can't. But I mentioned the best minds of humanity for a reason. They analyzed the operation of all programs and concluded that there are three major areas for growth.

Eliminating Downtime

It turns out that most of the time, a program is idle. It's constantly waiting for something: data needs to be copied from one place in memory to another, loaded from the hard drive into memory, waiting for a server response to a request, user input, etc.

All these tasks are carried out not by the central processor, but by memory controllers, disk, etc. And during this time, the central processor could be busy doing something useful. Thus, the idea emerged to run not one thread of instruction execution (thread), but several, on a single processor.

So while, say, one thread waits for user input, a second downloads something from the network, a third processes data, and a fourth renders images on the screen. This then evolves into asynchronous tasks and coroutines, but more on that later.

More Programs

If programs are idle 80% of the time, that's obviously not good. On the other hand, you can't just rewrite every program to fit our new approaches. Perhaps this problem can be solved differently — by simply running several programs on the computer simultaneously.

In this case, the operating system monitors the programs, and if a program is idle, it transfers its execution time to another program. This switching happens dozens of times per second, and the user doesn't notice the switches — programs appear to be running simultaneously from their point of view.

More Processors

Running programs simultaneously is cool, but what if there are many programs, and they're hardly idle? No downtime means no efficient use. For example, we have ten programs doing calculations or a resource-intensive game and something else along with it.

The solution to this problem was to add multiple processors to one processor. To avoid confusion, they started calling them cores. Now we have processors with multiple cores (sub-processors), with dozens of programs running in parallel.

Fun Fact! Number of Cores in Processors

Nowadays, server processors can have 64 to 256 cores, and in some specialized cases even more. For example, AMD's 4th gen EPYC processors offer up to 96 cores, while IBM's POWER10 can have up to 240 cores per chip. Consumer processors have also evolved significantly: high-performance desktop CPUs like AMD Threadripper can have up to 64 cores, while more common models usually have between 6 and 16 cores.

What's next? There's still room to grow! First of all, a single server motherboard can have several processors, like two or even four. Secondly, servers can be combined into server racks of 10-20 pieces. And server racks — into data centers, where there are thousands of such racks.

Distributed systems and microservices architectures have become common practice in modern software development. Many large companies, like Google, Amazon, Netflix, and even smaller businesses, use distributed systems to handle large amounts of data, ensure high availability, and scale their services. These systems allow for efficient use of numerous servers working together as a single unit, significantly boosting performance and fault tolerance of applications.

1.2 Advantages

Multithreading, or the concurrent execution of tasks within a program, offers several key advantages that can significantly improve the performance and efficiency of software. Let's explore the main benefits of multithreading.

1. Increased Performance and Speed

Parallel task execution: Multithreading allows for simultaneous execution of multiple tasks, which is particularly useful for programs that require the execution of many independent operations. This accelerates program execution, as tasks are conducted concurrently, utilizing all available CPU resources.

Utilizing multi-core processors: Modern processors have multiple cores, and multithreading allows for complete utilization of their power, distributing threads across various cores for parallel task execution.

2. Enhanced Responsiveness and User Interaction

Background tasks: Multithreading allows for long-lasting or resource-intensive operations in the background, keeping the main thread responsible for the user interface responsive. This enhances user experience, as the interface isn't blocked during heavy tasks.

Asynchronous operations: User interaction and event handling can occur concurrently with main task execution, making applications more responsive and efficient.

3. Efficient System Resource Utilization

Resource optimization: Multithreading allows for more efficient use of system resources, like CPU time and memory. This is especially crucial for servers and high-performance computing systems, where multithreading allows handling numerous requests and tasks simultaneously.

I/O management: Multithreaded applications can manage I/O tasks (e.g., network operations, file reading and writing) more effectively, as threads can handle other tasks while one waits for I/O operations to complete.

4. Support for Multitasking

Multitasking: Multithreading allows multiple tasks to be executed concurrently within a single process, simplifying development of applications requiring multitasking, like web servers, databases, and real-time applications.

Parallel data processing: In tasks involving large data processing, multithreading allows splitting data into chunks and processing them in parallel, significantly speeding up task execution.

Examples of Multithreading Usage

Web servers: Multithreading allows web servers to handle numerous client requests simultaneously, enhancing server performance and scalability.

Graphical user interfaces (GUIs): In apps with graphical interfaces, multithreading allows executing lengthy computations in the background, keeping the interface responsive.

Scientific computations: Multithreading is used for parallel data processing in scientific calculations, significantly speeding up complex computational tasks.

Games and simulations: In games, multithreading enables concurrent processing of physics, graphics, sound, and user actions, enhancing performance and realism.

1.3 Proper Name

There are some issues with the term "multithreading." It consists of two words: "multi" and "thread," hinting that inside the program there are many "threads of command execution" doing something.

A nice analogy, but in English (the original) literature for designating several concurrently executing actions, the term "thread" (thread) is used. And accordingly, multithreading is referred to as multi-threading.

This might be considered a minor misunderstanding — who cares how different terms translate into another language, if programming hasn't started actively using such a thing as stream, which can only be translated as "stream" in Russian.

Therefore, there is some confusion in Russian terminology, resolved in two ways:

• Thread (thread) is translated as "execution [command] thread."
• Stream (stream) is translated as "data stream."

On the other hand, many programmers have just started using English terms without translation:

• Thread (thread) is pronounced as "thred," and multi-threading as "multi-threding."
• Stream (stream) is pronounced as "streem."

Thread is often called a thread, but the term "multi-threading" never really caught on. Thus, "thread" and "multithreading" are often used in conversation simultaneously.

Using lots of borrowed terms makes the language richer, adds new meaning to words, and simplifies communication with colleagues from other countries. I'm all for this approach.