several important nuances in the new C++ standards

Undefined behavior (UB) is a pain that every experienced developer is familiar with; a kind of “Schrödinger code” when you don’t know whether it works correctly or not. Fortunately, the C++20/23/26 language standards have brought something new regarding undefined behavior. And quite important, if you are a software architect, and “pluses” are the key stack of your company (for more information on how and why we at Kaspersky Lab use C++ a lot, read

Here

).

In this article, I am from my positions as Senior Software Architect and Security Champion in microkernel operating system KasperskyOS I'll look at the pitfalls that can be encountered in almost any standard, and show what changes in C++20/23/26 – whether the number of cases with undefined behavior is reduced, and whether C++ is becoming safer.

What is undefined behavior

Let's start with the basic definitions: what is undefined behavior, how many of them are there, is it good or bad. Especially since the world of undefined behaviors in the “pluses” is wide and diverse.

The standard has three basic definitions that can be associated with undefined behavior.

1. Implementation-defined behavior is implementation-defined behavior, i.e. not defined in the standard. Everything is pretty good with him, because the full list is in a special section at the end of the standard Index of implementation-defined behavior (https://timsong-cpp.github.io/cppwp/n4868/impldefindex). There are quite a lot of cases with examples, in particular:

   • the size of the signs is different everywhere;
   • definition of macro NULL;
   • signedness of the char type (signed or unsigned – depends on the compiler);
   • size of base types except char.
   • …

2. Unspecified behavior is when the standard defines several options. In general, this can be called implementation defined, but the standard does not provide a complete list of unspecified behavior, and you have to look for cases yourself. Examples include:

   • the order of evaluation of the arguments in a function call (except for clearly defined ones – the “and”, “or” and ternary operators);
   • the order of evaluation of the operands of the operators +, -, =, *, /, except &&, ||, ?:;
   • …

3. Undefined behavior. If the previous two types of variable behavior assume that the program is still correct, there are no errors, then with undefined behavior it is no longer valid. The standard does not impose any requirements, stating that anything can happen. It gives a fairly general definition and the list of this undefined behavior is quite wide. This article will focus on undefined behavior. Examples:

   • access outside the array;
   • null pointer dereference;
   • integer division by zero;
   • integer overflow;
   • memory usage after freeing;
   • use of an uninitialized variable;
   • endless loops without side effects;
   • race;
   • …

How many undefined behaviors are there in C++?

The answer to this question is disappointing – no one knows for sure.

If you take the C99 standard, undefined behavior is described in a separate section: J.2 Undefined behavior https://port70.net/~nsz/c/c99/n1256.html#J.2 (there are 193 cases there). All this applies to C++.

But C++ has its own specific undefined behavior. You can use lists like this:

• P1705R1 Enumerating Core Undefined Behavior (36 cases) https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1705r1.html — this proposal has not yet been accepted, i.e. work on it is not completed. Its task is to list all the official types of undefined behavior for C++ and add them to the standard;
• reference books of varying degrees of completeness https://github.com/Nekrolm/ubbook (~60 cases). Everything is described here in some detail;
• rules in orderlies: clang ubsan (~35 rules) https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#id4 – can be used if you approach the question more formally.

But even all these lists together will not provide a complete list.

But can anything really happen?

The standard says that undefined behavior can result in anything. Is that true? The answer is not very comforting – it is indeed true.

Usually they give negative examples, like spilled coffee or Armageddon. I tried to find a more positive one – it turns out that with undefined behavior you can win the lottery.

The most popular case of undefined behavior in companies involved in information security is a buffer overflow. It is actively used by attackers, since now most lotteries are electronic. A random number generator is launched on the server and a random winner is selected. We can take advantage of a buffer overflow vulnerability on the server to add shellcode that affects the generator and run it. Result: we unexpectedly became the winner of the lottery and won, for example, an “AAAA car”!

In essence, we can say that undefined behavior in its original version was the reason for the positive outcome. There are an infinite number of such scenarios that can be created.

Why is UB bad?

Safety

The main difficulty is the security problem. To estimate how many vulnerabilities arise due to undefined behavior, you can take 25 TOP CWE (

https://cwe.mitre.org/top25/archive/2023/2023_top25_list.html

– data for 2023) – rating of problems in software products that lead to vulnerabilities. The TOP for 2023 includes five vulnerabilities directly related to undefined behavior:

The first place is traditionally occupied by out of bounds write – buffer overflow, also known as stack overflow. In this case, the “score” is a kind of integral rating that indicates the severity of the vulnerability, and exploitability is the number of exploits.

The list only includes those vulnerabilities that are directly related to undefined behavior. But in many other vulnerabilities, undefined behavior can also be an indirect cause.

Unexpected optimization

Optimization is the strength of the C++ compiler. But it may hold some surprises. Many people have probably heard that the compiler can optimize in very interesting ways. Here is a textbook example that is often used to describe the problem:

The function searches for an element in an array. After optimization, it may unexpectedly become simpler.

The compiler throws out the loop and if, because initially there is an error in the code – we go beyond the boundaries of the array. The compiler sees undefined behavior and, based on this, optimizes the code in a way that is convenient for it (so that the program runs as quickly as possible). Here it assumes that the element will be found outside the array boundary in any case, so it returns true, and the loop and everything else are thrown overboard.

It's worth noting here that this kind of weird execution-changing optimization only occurs if the program was buggy in the first place. In a normal situation, when there is no undefined behavior in the code, optimization is performed correctly.

There are quite a few cases of such strange optimization. There are sources that collect them:

Why is UB good?

Undefined behavior is not always bad. In most cases, something good can come from this.

Speed

The first and main advantage is the speed of the compiled program.

Traditionally, the C++ compiler considers the programmer smart enough not to allow undefined behavior. So by default it doesn't do:

• zero optimization;
• checking counters and links;
• buffer bounds checks;
• checking preconditions and other restrictions.

All this is considered unnecessary. And the absence of these checks leads to aggressive optimization, when you can create inline functions, throw out loops, change the order of execution of instructions, and so on. This leads to a big performance boost.

Variety of platforms

Code that is compiled in C and C++ runs on a wide variety of hardware. And it must work everywhere, despite different architecture, addressing, memory handling, differences in number representation, etc. Therefore, the compiler leaves itself some room for maneuver in the form of undefined behavior.

Yes, there are many positives, but if UB has the potential to cause harm, then all the good stuff isn't worth it.

UB in modern standards

Let's look at specific examples of undefined behavior in modern standards. And let's start with C++20.

Signed Integers in Two's Complement in C++20

In proposal P0907R4: Signed Integers are Two's Complement

https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1236r1.html

Signed numbers are now represented in two's complement notation.

Before the C++20 standard, no one talked about what the representation of a signed integer should be. It could be represented in any code, which is why signed number overflow was undefined behavior. And starting from the 20th standard, we use additional code that always behaves the same: when it overflows, it loops back, i.e., the maximum value becomes the minimum.

Unfortunately, in C++ not everything is so simple. Even taking into account the known way of representing a signed number, its overflow still remains undefined behavior – and this is clearly noted in the proposal.

Note: Overflow for signed arithmetic yields undefined behavior (7.1 [expr.pre]). — end note

Due to the fact that this is undefined behavior, there are many optimizations in the compiler. However, C++ has added some minor innovations in bit shifts (<<, >>). They will be useful to those who use bit operations.

• You can shift negative numbers (previously this was undefined behavior).

  int x = -1 << 12;

• Left shift causes zero padding.
• When shifting right, the sign bit is filled.
• The number of shifts must be positive.
• The number of shifts must not exceed the number of bits in the number.

The mentioned restrictions existed before for positive numbers. Violating them will still cause undefined behavior.

Depricate volatile in C++20

In C++20, the volatile keyword was deprecated: P1152R4: Deprecating volatile

https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1152r4.html

.

This keyword has a bad reputation in both C and C++. Previously, no one understood how to use it, so it was used incorrectly. It looks like it should finally go away now.

True, this is not entirely true…

The proposed deprecation preserves the useful parts of volatile, and removes the dubious / already broken ones.

Only some cases are prohibited, which were already broken – they either did nothing (were ignored), or demonstrated incorrect use of volatile, for example as atomic. As a result, volatile is preserved, it is possible to declare it.

Complex assignment and increment/decrement were depricated because these are compound operations.

volatile int x = 0; 

x += 10; // C++ 20 deprecated
x++; // C++ 20 deprecated

If someone used volatile as atomic out of habit, expecting atomic operations, it was always non-atomic. Now it will be clearly highlighted during compilation.

Disabled volatile arguments and return value of functions.

volatile int func(volatile int arg); // C++ 20 deprecated

This didn't work before, and was simply ignored. For example, if volatile is set for an argument, one might think that some optimizations would be disabled in this case (the argument would be passed not through a register, but through the stack, and this would be forced through volatile). But volatile never changed calling conventions, i.e., in the end, it didn't affect anything. The same story with the return value of the function.

Another prohibited case is volatile in structure buildings. This is a kind of mechanism for specifying aliases for a structure member or array element.

struct Foo {int val;} bar;
volatile auto [val] = bar; // C++ 20 deprecated

But in this case, the alias declared with volatile did not affect the original element of the structure or array in any way, but only misled. That is, in essence, it did not work either.

Cases in which volatile did not work or was simply ignored actually came from the fact that it was always paired with const. But now they are separated and, in my opinion, this is a big plus.

For those who want to study this topic in more detail, I recommend a rather interesting talk from CppCon 2019: Deprecating volatile — JF Bastien https://www.youtube.com/watch?v=KJW_DLaVXIY&ab_channel=CppCon.

Signed function ssize() in C++20

C++20 introduces the signed ssize() function: P1227: Signed ssize() functions

https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1227r2.html

.

To understand why it is needed, I offer an example:

template <typename>
bool has_repeated_values(const T& container) {
  for (int i = 0; i < container.size() - 1; ++i) {
    if (container[i] == container[i + 1])
      return true;
  }
  return false;
}

Here is a function that searches for duplicates in the container. Judging by the code, the container should come already sorted. But there is an error in the function: reading beyond the array if a zero container size comes. And this error causes undefined behavior – instead of a negative value, we get the maximum, we sort through all the memory and as a result something incomprehensible happens.

You can fix this case using the ssize() function. It turns the unsigned type std::size_t into a signed type std::ptrdiff_t. If size_t allows you to address all available memory for a 64-bit system (and this is 64 bits), then ptrdiff_t, by its semantics, allows you to represent the difference in memory addresses. This difference can be negative, resulting in a value that is one bit less (63 significant bits, one sign). True, there is still the possibility of undefined behavior, which occurs if the container size exceeds PTRDIFF_MAX, but is less than SIZE_MAX.

It can be argued that containers the size of half of all addressable memory in a 64-bit system are not that common. And in general, it is unlikely that such a container will be created, which means that the undefined behavior case will be quite rare. That is, in general, using ssize() will still fix the basic errors.

Fixing ranged for in C++23

Let's move on to more recent standards. One of my favorite innovations is that C++23 fixed the range-based for loop: P2644R1: Fix for Range-based for Loop

https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2718r0.html

. It turns out that it has been working incorrectly since its introduction in C++11.

Explanations about the error will have to start from afar.
Let's say we have a GetVector function that returns a vector of strings.

std::vector<std::string> GetVector() {
  return { "str1", "str2" };
}

C++ has several mechanisms that allow temporary objects to live until the end of the scope. The first is a constant reference to a value that lives until the end of the scope. In this case, we get str correctly.

{
  const auto& val{ GetVector() };
  std::cout << val.front() << std::endl;  // str1
}

The second mechanism is an autolink, also known as a forward reference, also known as a universal link, which extends the lifespan of temporary objects.

{
  auto&& val{ GetVector() };
  std::cout << val.front() << std::endl;  // str1
}

Range's for loop can also extend the lifetime of temporary objects. In initialization, we can create a temporary vector that will live until the end of for.

for (auto& val : GetVector()) {
  std::cout << val << std::endl;  // str1 str2
}

Everything will be fine here until we want to iterate not only over the vector of strings, but also over the characters in the string.

for (auto& val : GetVector().front()) {
  std::cout << val << std::endl; 
}

Here we get undefined behavior. However, it is worth noting that undefined behavior may occur if GetVector returns a vector of zero size. But here I wanted to focus on something else.

If we talk about ranged for in the context of extending the life of temporary objects, then this construction is not monolithic. It expands into a regular for:

auto&& rg = GetVector().front(); // !!!
auto pos = rg.begin();
auto end = rg.end();
for ( ; pos != end; ++pos ) {
  char c = *pos;
  …
}

First, initialization occurs here (universal reference declaration). It should have extended the lifetime of the temporary object, but, unfortunately, here access to the temporary object occurs through a chain of calls. In C++, chain extension has never worked. It turns out that ranged for never worked either.

The problem is precisely in the range for, because it hides implementation details. For an ordinary programmer — a client of for — it is unclear how it is structured inside and what it expands into. This secrecy raises doubts, since the developer needs to know at least what temporary objects are saved, regardless of whether they are in the chain or not.

The proposal does not specify how this is fixed. It provides examples of how this can be done. For example, you can replace simple expansion with a lambda and pass the final temporary object to it. It will be saved until the end of the lambda execution, which will solve the problem. But this is not the only way.

An important conclusion: in C++23, all temporary objects created in a ranged for chain (no matter how many there are) will be saved. This is a definite plus.

Using string with a null pointer in C++23

C++23 prohibits using string with a null pointer: P2166R1: A Proposal to Prohibit std::basic_string and std::basic_string_view construction from nullptr

https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2166r1.html

What's the point? One of the most popular string constructors in C++ is from a null-terminal string. There is an explicit restriction that a null-terminal string must be valid, otherwise undefined behavior will appear.

If we are talking about nullptr, then this is an invalid interval. According to the standard, there should be undefined behavior here:

std::string str{ nullptr };

To fix this, an elegant solution was used: the constructor from nullptr was explicitly prohibited. If there is no compilation, then there is no undefined behavior. It would seem that everything is fine. But, unfortunately, if you use a variable with the same nullptr, then there will still be undefined behavior.

Example:

char *ch = nullptr;
std::string str(ch); // UB

In fact, initialization with nullptr has runtime checks on many compilers. At least on clang there will be an exception. But this is not always the case. For example, the Microsoft compiler behaves differently, so it is better not to tie logic to this.

Using Exclusive Mode for File Streams in C++23

Another interesting case is the use of exclusive mode for file streams: P2467R1: Support exclusive mode for fstreams

https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p2467r1.html

.

Before we move on to its description, a little clarification.

There are cases when you need to check something in a file and only then open it. In this case, the presence of the file is checked. The file is opened only if it is not there:

void CheckAndCreate(const std::filesystem::path& p) {
  if (!std::filesystem::exists(p)) {
      std::fstream f(p.string(), std::ios_base::in | std::ios_base::out);
      f << "data" << std::endl;
  }
}

This is necessary, for example, when creating configs. During the first launch, the application checks for the presence of a config, and if it is not there, it creates a file, adding default values ​​to it (if there is a config, then the application reads it).

But this case has a problem. Between the moment the file is checked and the moment it is used, there is a race window in which an attacker can wedge himself. For example, he can make a symlink to the path to some system file, and as a result, the program will overwrite the system file. There are, of course, many “ifs” here, since in order to overwrite a system file, the program must have the appropriate rights. In addition, it must have external input to write to this file what the attacker needs. But if all the stars align, then such a vulnerability really does exist.

This is solved by a special flag, which in C++ is called noreplace. It is set when creating a file stream and atomically checks the presence of the file. If the file does not exist, then only in this case will it be opened.

This check is atomic, and this flag is far from new. It has been in the C standard for quite some time. Now this pleasant innovation has appeared in C++.

void CheckAndCreateNoRace(const std::filesystem::path& p)
{
  std::fstream f(p.string(),
    std::ios_base::in |
    std::ios_base::out |
    std::ios_base::noreplace);
  f << "data" << std::endl;
}

Unfortunately, this does not fix all cases. For example, if you need to check not just the presence, but some attribute of a file, say, its extension or recording time, this is still done along the path and not atomically, i.e. there will still be a race.

Of course, we can still argue here whether this is a race or not. Let's just consider it a race. But in fact, such cases that check something on the way, and then use the file separately, are all broken. The entire std::filesystem library has such problems.

void CheckAndUse(const std::filesystem::path& p) {
    if (std::filesystem::is_regular_file(p)) {
        std::fstream f(p.string(), std::ios_base::in | std::ios_base::out);
        f << "data";
    }
}

There are a couple of proposals that should solve this problem radically – instead of a path, they suggest using file handles.

These proposals completely rework the entire std::filesystem library. They are scheduled for C++26, i.e. have not yet been adopted:

Accessing containers in C++26

C++ containers have always had two access options: the bracket operator and the At method. The first option did not check the range, the second did check and throw an exception. Both methods were implemented in all containers except span. Since the introduction of std::span in C++20, there has not been an At method.

Proposal P2821R4: span.at() https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p2821r4.html has few technical details, but asks many questions. Is it intentional that neither C++20 nor C++23 lack this method? If so, why wasn't the method removed from other containers (or at least depricate)? The answer is given there, and it's trivial:

Ultimately, this becomes a stereotypical example of how C++ traditionally handles safety.

This is an example of how security cases are traditionally handled in C++. Everything happens slowly and in a flurry. Sometimes some things can simply be forgotten.

Only in C++26 will at() appear in span.

Saturation Arithmetic in C++26

Finally, one more interesting case: arithmetic with saturation: C++ 26, P0543R3: Saturation arithmetic

https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p0543r3.html

.

Saturation arithmetic is a version of arithmetic in which all operations, such as addition and multiplication, are limited to a fixed range between a minimum and maximum value.

Let me remind you that unsigned integer overflow is not undefined behavior, it has always been a cyclic return or modulo arithmetic. Signed integer overflow in C++ was undefined behavior. And saturated arithmetic allows us to solve this issue radically. From the point of view of saturated arithmetic, all operations with numbers must occur in a certain range of values. If we go beyond this range, only the maximum or minimum number will be returned (depending on which side we went beyond the range).

The use of saturation arithmetic is nothing new. It is quite developed and really helps in some cases. It really makes sense, for example, in graphics or 3D modeling.

The use of this arithmetic is very simple. In C++26, there will be only four additional functions that duplicate the arithmetic operations of addition, multiplication, division, subtraction, as well as the Saturate_cast function – a type conversion that takes into account either the maximum or minimum.

template<class T>
constexpr T add_sat(T x, T y) noexcept;

template<class T>
constexpr T sub_sat(T x, T y) noexcept;

template<class T>
constexpr T mul_sat(T x, T y) noexcept;

template<class T>
constexpr T div_sat(T x, T y) noexcept;

template<class T, class U>
constexpr T saturate_cast(U x) noexcept;

One might wonder how fast this will work. In theory, we could add these checks (if) to all operations manually – essentially creating such functions ourselves. But, according to the proposal, the functions will use hardware acceleration – commands that are found in all modern processors.

Most modern hardware architectures have efficient support for saturation arithmetic on SIMD vectors, including SSE2 for x86 and NEON for ARM.

This will work no slower than ordinary arithmetic.

Let's sum it up

It is gratifying that security issues and, in particular, undefined behavior in C++ are being raised more and more often. This cannot but please, since there are many proposals and features related to safety and security, and they all require due attention.

In modern C++, there is less and less chance of catching undefined behavior in practice – due to inattention or ignorance. That is, UB ceases to be hidden, and this is also good.

But, unfortunately, cutting out undefined behavior does not happen quickly. Still, the standard is red tape, coordination, discussion, many different nuances and legacy code.

However, what we can be sure of is that flying on a rocket called C++ is becoming safer and there is no limit to perfection (or perfection is not the limit).

And also that C++ is not the best language to shoot yourself in the foot with 🙂