Are you still writing multithreading in C++ with synchronization errors?
Hello colleagues! In this article, I'll show my approach to writing multi-threaded code that helps avoid common pitfalls associated with using basic synchronization primitives.
The idea will be demonstrated using live code examples in modern C++. I first applied most of the solutions described on my own projects, and now some of these approaches are already used in ours. Kaspersky Lab's own microkernel operating system (KasperskyOS).
I would like to make a reservation right away that the topic of multithreading is a very large and serious one. And this article is not a full analysis of multithreading problems, but only specific (but quite common) cases when we are forced to use mutexes.
Well, I’ll also add that the concept that I will talk about has already been described in general terms here in this article of mine. The time has come to talk about it in more detail from a purely practical side.
Common errors in multi-threaded code
Let's start by looking at typical multi-threaded code errors that you often encounter in code reviews or somewhere in legacy code. This way it will be clear in what niche my idea can be applied in practice.
Shared data that is shared between different threads is usually protected by mutexes, which are created independently of the data being protected. Therefore, situations are quite common when:
- The mutex was not locked while accessing shared data.
- The wrong mutex (intended to protect other data) was locked.
- The mutex is locked without accessing shared data. This is probably the least critical situation, but it can have a negative impact on performance.
I think many people have encountered the consequences of such mistakes:
- Unexpected falls begin in various strange places. The backtraces show that the program crashed, but why exactly is not always clear.
- Crashes occur randomly, but may depend on the load on the application and the system environment.
- It is difficult to identify and reproduce the failure scenario.
- Data integrity violation occurs.
- Productivity drops.
All this is a lot of pain, red eyes and debugging sessions. More often than not, problems arise because they forgot to lock the mutex. As a result, the data is violated by another thread and ends up in an inconsistent state. Somewhere later it goes off and the program crashes. In a bad scenario, you can get a database corruption.
The code below is an illustration of what I mean. This is not a real example, just a demonstration.
There is a class with data fields, not all of which need protection. For example, _id is not needed. There is a first mutex (_mutex1) that protects _name and _data1, and also a _mutex2 that protects _data2. Perhaps it was added later by another person who did not understand all the details. It's hard to get confused in the code above, but the classes can be quite long. When there is no way to rewrite them, you have to understand someone else's logic, and in this case it is very easy to get confused with mutexes.
The first method is dataSize. Here we used lock_guard, locked the first mutex and accessed _data1. But in the name method they forgot about _mutex1, although it was assumed that the first mutex would also protect _name.
And in the next method, _mutex1 was locked to access _id (even though _id was not supposed to be protected). We experienced a performance hit because another thread was waiting at that moment.
In the next block of code, we could use copy-paste for quick writing, but forget that in the second line we need to lock _mutex2 (and mistakenly locked _mutex1).
Well, the last example – we first lock one mutex, and then access the data that is protected by another mutex.
Existing methods for solving the problem
Let's talk about how this problem can be solved.
— Improve the quality of code inspection. This is the first thing that comes to mind. But no one has canceled the human factor – you can always miss some points, especially in complex code with non-trivial logic. Therefore, the method does not provide a 100% guarantee.
— Use ready-made libraries for parallelization. This is an attempt to avoid direct use of synchronization primitives. For example, Streams in Java 8 is made conveniently – just add one call, and the entire collection is already processed in parallel, and the developer does not need to think about these low-level entities.
— Use static analyzers/code sanitizers that can catch such situations. True, they can be quite expensive and it is not always possible to use them. And, again, they do not provide a 100% guarantee.
— Use only immutable shared data. This is a specific approach in which risks simply disappear, since several threads in parallel cannot change data. But there are certain limitations in using this approach. We simply cannot implement some things with it.
— Completely prohibit the use of shared data, for example, apply the actor model in Scala when only asynchronous message exchange is allowed between threads. By the way, this approach is built into the language and you can use it freely. But there are not analogues in every language. In C++, we will have to look for some kind of reliable framework, and questions will remain about how well everything is implemented under the hood – are there any errors, will it survive under heavy load. That is, when introducing the methodology, quite serious research will have to be carried out. Plus, not every multi-threaded task can be applied to such an abstraction.
— Use a model in which direct calls to class methods are possible only from the thread that generated it; the rest occur indirectly, for example through a message queue. As an example, there is a rather interesting multithreading model in Qt. Within this model, you can write as in single-threaded mode, because a call will never come from another thread. But this is also a convention that must be observed (and yet this is a Qt feature, not a language standard).
— Use specific languages. In some languages, the compiler is able to fully control any access to shared data and prohibit incorrect attempts. For example, this is available in the programming languages D or Rust and partly C#. In the same D there is a keyword shared, which declares shared data. Next, the compiler tracks all incorrect calls to them and suggests errors at the compilation stage.
I propose an alternative approach – an attempt to correct the situation with existing language capabilities without introducing new keywords or using third-party frameworks.
The essence of the idea
In my opinion, the main problem is that the mutex lives separately from the protected data and the connection with it is very ephemeral. The author who wrote the code assumed that the mutex would protect this particular data, but over time, as the class was modified, this implicit agreement could be lost. And because of this, errors occur. You need to associate mutexes and the data they protect so that they have a common life cycle. Moreover, I would like to completely get away from manual management of mutexes and clearly regulate access to shared data. And this needs to be done using the programming language itself, without using third-party frameworks or changes in the compiler.
The idea came up to introduce an abstraction – a template class SharedState, which will encapsulate both common data that is planned to be accessed from several threads, and means for protecting it.
Thus, all shared data is placed in a separate class or structure, which, in fact, specifies the SharedState template class. The shared data object is created in the SharedState constructor, that is, all the parameters needed to create it are passed to the SharedState class. And the shared data object is not accessible from the outside, since it exists as a private field.
SharedState guarantees strictly regulated, secure access to shared data. You can request access to read or modify data, or wait for a certain state.
Lambda expressions, which are available in C++, are well suited for accessing such a model.
SharedState interface
In practice, we pass all the parameters needed to create a shared data object to the SharedState interface.
The template method for viewing data is called view and takes as a parameter a std::function, which will be called with a constant reference to shared data (that is, in this call the data is protected – roughly speaking, the mutex is locked internally). The view method can be specified for any return value. Perhaps in production code it is better to use a template instead of std::function (when accessing dynamic memory, std::function can cause performance degradation), but for clarity and transparency of signatures, here is an example using std::function.
A simple modify method for changing shared data takes a std::function that will be called with a non-const reference to the shared data. Inside we somehow modify the data and exit. A more complex case is the modify method, which returns the Action class. This way you can implement more complex things.
The Action interface has methods for easily modifying shared data without notifying about its change, access and extract, as well as methods for modifying shared data while notifying about its change. notifyOne and notifyAll – they were introduced precisely to provide the opportunity for one or, accordingly, all waiting threads to receive notifications about changes. There is also a when method that accepts a std::function, a predicate for determining the appropriate state of the shared data. The method returns the same instance of the Action object, so you can build chains of calls:
Here I have written a simple thread pool implementation based on SharedState. Inside is a loop in which each worker from the thread pool waits for new tasks to appear and takes them for execution. We access the SharedState, which is called _state, request modify(). It returns an Action on which we call the when method. I won’t go into detail, but the point is that we check whether there are new tasks in the queue, and internally we try to access shared data.
This code doesn't look perfect. When we make an extract, we have to use the template keyword – this is a C++ requirement. And you also have to write a lot of parentheses. All these are disadvantages (however, if you use Cpp2/CppFront, these disadvantages are leveled out :)). But when the same function was written using mutexes and condition variables, it looked even worse. Now we have moved away from communicating with mutexes, and the code has become cleaner and more understandable.
Let's see what's under the hood of the SharedState class.
It's very simple. There is some optimization here for the language standard – if we use the 17th standard, shared mutexes are available to us, and we can allow several threads to access data for reading. In some cases, such as when data is read frequently but rarely modified, this can improve performance.
Here we check in the constructor that this inner class with shared data can be created with the parameters that were passed. Next we describe the view method, to which we pass std::function. The prototype described there is passed a constant reference to the shared data. And here the compiler helps – if we request read access, but try to modify the data, it will complain.
In the modify method, we already exclusively lock the data, so the link that is passed there is not a constant one. This method is more complex and returns an Action.
Here we passed the Action SharedState to the private constructor. It gets locked, and the shared data is blocked for the duration of the Action. Internally we do access – this is an easy way to modify data. NotifyOne and notifyAll are provided. And when is the condition variables method, where the predicate is passed. It stops the execution chain until the predicate is satisfied on the shared data.
What examples with SharedState look like
This is what the first example from the beginning of the article will look like if we rewrite it using SharedState.
We place the general data that was supposed to be protected by the first mutex in one structure, and the data protected by the second mutex in another. We create two instances of SharedState, specifying them with the structure Data1 and Data2. Our _ids are stored separately because they do not need protection.
To calculate the size of a vector, we specify view with the result we want to return, in this case size_t, and get a constant link (we don’t write its type here, we use auto). Next, we access the structure, which is accessible via a constant link, and call the size() method. If we try to call a modifying method (without the const specifier), the compiler will help find this error.
Essentially, we are correcting the problem of the first example here – we are not locking _id. Explicit code blocks help us understand what kind of common data we are accessing. You don’t have to think about which mutex you need to lock them with. We simply access the data, and the protection is carried out somewhere under the hood.
Advantages of the approach
What benefits do I see in using this approach:
- We are moving away from explicitly creating synchronization primitives such as mutexes, lockers and condition variables, and at the same time from manually managing them.
- Shared data is a separate entity. It seems that this is bad – we generate unnecessary entities. But if you look at it from a slightly different angle, previously shared data was represented by separate fields scattered throughout the classroom. However, they had some kind of logical connection (we fumble them between threads). And thanks to a separate structure, we materialize this connection. Conceptually, this is how it should be – data lives and is managed together.
- Conceptually, we are moving away from the “locked/unlocked” categories and are starting to think in terms of data access: “view/modify/wait.”
- The author of the code clearly expresses his intentions with respect to the shared data. For example, it declares directly using code that it wants to view general data in the following lines. This self-documentation helps others to support him.
- By using std::shared_mutex (C++17), multiple threads can read the same data simultaneously, which can improve overall performance in some cases. We don’t do anything special for this—we just express our intention to read the data.
- All SharedState methods are implemented inline and declared directly in the header file, so performance should not suffer after switching to SharedState.
Note: I tested clang's assembly output on Linux and it was clear that the compiler was able to effectively inline lambda expressions where they were used. However, for implementation in production, I recommend more thorough, systematic performance studies.
Rules for using SharedState
Instead of summarizing, I would also like to mention that certain errors can occur when using SharedState. Here's what you need to consider to avoid encountering them:
- The shared data object should be simple and not contain complex/unobvious logic. It is advisable to do without references to external objects. The main purpose of a shared data object is to be a trivial container, in C++ the best term is “structure”.
- To improve performance, you need to minimize the amount of code in shared data access lambdas and limit yourself to retrieving the necessary information or modifying it. You should not call any heavyweight operations inside protected blocks. This is true for any work with mutexes. But in this case, we should not forget that everything is locked. Therefore, it is better to do exactly what you requested access for. We took read access – we read it, perhaps saved it into a local variable, and then we continue to work with it. With modification it’s the same: go in, modify, leave. All complex logic should remain outside. You also cannot access the same SharedState instance. Mutexes are not used recursively, so otherwise a deadlock will result.
- Complex manipulations with the extracted data are best done in local variables outside the lambda expression block. However, do not forget that the connection of the retrieved data to its source in the shared data object is immediately lost after exiting the lambda, since another thread can immediately change it.
- You cannot save links to a shared data object and work with it in circumvention of the regulations. We can quite easily go into the lambda, save pointers to common data and work with them externally. But in this case the contract is broken and nothing is guaranteed.
- I recommend not saving (for example, in class fields) an instance of the Action class that returns SharedState, because during its existence the internal mutex will be locked. It is better to use it solely in a chain of calls (as in one of the examples above) or, as a last resort, as a local variable in a limited scope (such as a mutex in a block of code explicitly created for it).
Examples of using SharedState
A simple Thread Pool implementation:
Abstract cache implementation: https://sourceforge.net/p/cpp-mate/code/ci/default/tree/src/main/public/CppMate/Cache.hpp
However, there are alternative solutions:
* The Boost library includes Synchronized Data Structures and, in particular, Synchronized Values, but their support is still in an experimental stage
* There are similar mechanisms in the Folly library, described here: https://github.com/facebook/folly/blob/main/folly/docs/Synchronized.md
If you like the idea and want to try it in C++, I'm ready full implementation of SharedState with documentation (Doxygen).
But in general, if you like to tinker with such things in C++, come to us at Kaspersky Lab. You can go through all stages of interviews in a couple of days. “Pros” is one of the key languages in our technology stack, so the range of possible tasks is huge, as is the list of new features. And legacy is not there.
A Here You can test your C++ knowledge in our smart city game.
