Beginner's Guide to Concurrent Programming in Rust

Concurrency and Parallelism

Many people cannot distinguish between the concepts of concurrency and parallelism, so before diving into asynchronous programming in Rust, it’s essential to first understand the difference between concurrency and parallelism.

We often come across the following definitions in operating systems textbooks:

Concurrency refers to two or more events happening within the same time interval.
Parallelism refers to a system’s ability to perform computations or operations simultaneously.
Explanation 1: Concurrency means two or more events occur within the same time interval, while parallelism means two or more events occur at the exact same moment.
Explanation 2: Concurrency refers to multiple events on the same entity, while parallelism refers to multiple events on different entities.
Explanation 3: Concurrency is the handling of multiple tasks "simultaneously" on a single processor, while parallelism is the simultaneous handling of tasks on multiple processors, such as in a distributed cluster.

Rob Pike, one of the creators of Golang, offered a very insightful and intuitive explanation:

Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.

Concurrency is the ability to handle many tasks at once, while parallelism is the technique of executing many tasks at once.

When we place tasks into multiple threads or asynchronous tasks for processing, we are leveraging concurrency. When these threads or async tasks run simultaneously on a multi-core or multi-CPU machine, we are leveraging parallelism. In a sense, concurrency enables parallelism. Once we have the ability to handle concurrent tasks, parallelism comes naturally.

Concurrency: Refers to only one instruction being executed at any given moment, but with multiple process instructions being rapidly switched in and out, giving the macro illusion of simultaneous execution. However, on the micro level, they are not truly simultaneous; time is sliced into segments to allow rapid alternating execution among processes.
Parallelism: Refers to multiple instructions being executed at the same time on different processors. So both on the micro and macro levels, tasks are executed and processed simultaneously.

When multiple threads are operating, and if the system has only one CPU, it is impossible for more than one thread to truly execute at the same time. The system divides CPU time into segments and assigns them to threads. When one thread is running, others are suspended. This approach is called concurrency.

When the system has more than one CPU, thread operations may be non-concurrent. While one CPU executes one thread, another CPU can execute another thread. The threads do not compete for the same CPU resources and can run at the same time. This is called parallelism.

Here's a conclusion: Both concurrency and parallelism describe multi-tasking. Concurrency is about alternating execution and focuses on handling capacity, such as concurrency levels; parallelism is about simultaneous execution and focuses on the execution strategy, such as task parallelism.

Concurrency Programming Models

We know that different programming languages have different implementations, which leads to varying concurrency models across languages. When we write and compile a program in a specific language, that program will occupy a process when it runs. Within this process, threads can be created, and these are at the operating system level. Within the language itself, threads created by the programmer using language-level features are considered language-level threads. Whether these two types of threads are one-to-one depends on the language’s internal implementation:

OS-native threads: For example, the Rust language directly invokes APIs provided by the operating system, so the number of threads in the program matches the number of OS threads used.
Coroutines: Similar to how Go language operates—internally, M threads in the program are mapped in some way to N operating system threads.
Event-driven: This model is often used in combination with callbacks. It performs very well in terms of performance, but its biggest problem is the risk of callback hell.
Actor model: Based on message passing, this model performs concurrent computation on decomposed small units. It’s a killer feature of the Erlang language.
Async/await model: This model has high performance, supports low-level programming, and behaves similarly to threads or coroutines without requiring significant changes to the programming model. However, the trade-off is that its internal implementation mechanism is quite complex.

In short, after weighing the trade-offs, Rust ultimately chose to provide both multi-threading and async/await as two concurrency programming models:

Multi-threading is implemented in the standard library by directly invoking OS APIs. It is simple to implement and use, making it suitable for scenarios with a small number of concurrent tasks.
Async/await is more complex to implement, but Rust uses a combination of language features, standard libraries, and third-party libraries to abstract and encapsulate it. This allows developers to use async/await without worrying about the underlying implementation logic. It's suitable for large-scale concurrency and asynchronous I/O.

Asynchronous Programming in Rust

Asynchronous programming is a concurrency programming model. It allows us to run a large number of tasks concurrently while requiring only a few—or even a single—OS thread or CPU core. In terms of usage experience, modern asynchronous programming is almost indistinguishable from synchronous programming.

Many languages today support asynchronous programming via async, but Rust’s implementation is different in several important ways:

Futures are lazy in Rust: They only begin to run when they are polled. Discarding a future prevents it from ever being executed. You can think of a Future as a task scheduled to be executed at some point in the future.
Zero-cost abstraction: Using async in Rust incurs zero runtime cost. This means only the code you write (your visible code) incurs performance overhead; the internal implementation of async introduces no performance penalty. For example, you don’t need to allocate heap memory or perform dynamic dispatch to use async. This greatly benefits performance, especially in hot paths, and is one of the reasons why Rust’s async performance is so high.
Rust does not include a built-in runtime required for async calls**, but that’s not a concern—Rust’s ecosystem offers excellent runtime implementations, such as the well-known **Tokio**.
The runtime supports both single-threaded and multi-threaded modes, each with its own advantages and trade-offs, which will be discussed later.

Choosing Between Async and Multithreading

Although both async and multithreading can be used to achieve concurrent programming—and the latter can even enhance concurrency through thread pools—these two approaches are not interchangeable. Switching from one to the other often requires significant code refactoring. Therefore, understanding their differences and applicable scenarios, and making the right choice in advance, is extremely important.

For CPU-intensive tasks, such as parallel computation, multithreading is more advantageous. This is because such tasks tend to keep the threads running at full capacity for long periods. The number of threads you create should equal the number of CPU cores to make full use of parallel processing capabilities. In this case, there’s no need to frequently create or switch threads, because any thread context switch causes performance overhead. You can bind threads to specific CPU cores to reduce this overhead.
For IO-intensive tasks, such as web servers, database connections, and other network services, asynchronous programming is more advantageous. This is because these tasks spend most of their time waiting. If you use multithreading, most threads will remain idle much of the time. Combined with the high cost of thread context switching, this results in significant performance loss. With async, you can effectively reduce CPU and memory usage while still running a large number of tasks concurrently. Once a task enters an IO or other blocking state, it will immediately yield, allowing another task to run. The cost of switching tasks in async is far lower than thread context switching in multithreading.

It’s important to note: Async is also based on threads under the hood. However, it wraps around threads via a runtime that maps multiple tasks to a small number of threads. Essentially, it throws a large number of IO-bound concurrent events into a small number of threads and communicates efficiently through events.

The cost of this approach is that it increases the runtime of Rust programs (the runtime being the Rust code that gets bundled into every executable). This causes the compiled binary size to increase significantly.

Let’s illustrate the difference between the two with a simple example: Suppose we want to download two files. We could download them one after another (serial execution), but that’s obviously not the fastest approach. Naturally, we think of using multithreading for parallel downloading:

Multithreaded Programming:

fn download_two_files() {
    // Create two new threads to perform the tasks
    let thread_one = thread::spawn(|| download("URL1"));
    let thread_two = thread::spawn(|| download("URL2"));
    // Wait for both threads to complete
    thread_one.join().expect("thread one panic");
    thread_two.join().expect("thread two panic");
}

If you’re only downloading one or two files each time, this approach works fine. But the problem arises when you need to download hundreds or thousands of files simultaneously—each download task consumes a thread, and the resource cost of threads scales up quickly (threads are still too heavy). In this case, you might consider using async:

Async Programming:

async fn get_two_sites_async() {
    // Create two separate futures
    // You can think of a future as a scheduled task to be executed at some future point—similar to a Promise in JS
    // Once both futures are run, they will download the target pages concurrently
    let future_one = download_async("URL1");
    let future_two = download_async("URL2");
    // Run both futures concurrently until they complete
    join!(future_one, future_two);
}

Compared to the multithreaded model, async shows its advantage here: for the same level of concurrency, it reduces the cost of creating and switching threads.

Summary

Concurrency and parallelism are both descriptions of multi-task processing. Concurrency refers to tasks being handled in turns, while parallelism refers to tasks being handled simultaneously. Concurrent programming means different parts of a program execute independently, while parallel programming means different parts of a program execute at the same time.

In terms of concurrency programming models, due to Rust’s language design philosophy—emphasizing safety, performance, and control—Rust did not adopt the "radical simplicity" approach like Go. Instead, it chose to combine multi-threading with async/await. The advantage of this is stronger control and higher performance. The disadvantage is higher complexity. But of course, this is an expected trade-off for a systems programming language: using complexity in exchange for control and performance.

In fact, async and multithreading are not mutually exclusive—in many applications, both are used together. Although both async and multithreading can achieve concurrent programming—and multithreading can even leverage thread pools to enhance concurrency—these two models are not interoperable. Switching from one to the other requires large-scale code refactoring. Therefore, choosing the right concurrency model early on becomes critically important for your project.

In conclusion, async programming is suited for IO-bound tasks, while multithreading is suited for CPU-bound tasks. Here's a brief summary of the selection rules:

When you have a large number of IO tasks that need to run concurrently, choose the async model.
When you have a few IO tasks that need to run concurrently, choose multithreading. If you want to reduce the overhead of thread creation and destruction, you can use a thread pool.
When you have a large number of CPU-intensive tasks to run in parallel (e.g., heavy computations), choose the multithreading model, and try to match or slightly exceed the number of threads with the number of CPU cores.
If the choice doesn’t really matter, default to using multithreading.