how gcc and clang handle statically known undefined behavior
Recently, our team had a discussion about undefined behavior (UB) in C. Let me remind those who don’t know: if we write code whose effect (and events during its execution) are not strictly defined in the language specification, then undefined behavior occurs. Thus, when encountering such code, the compiler can act at its own discretion, and there is no guarantee that the execution of this code will follow a predictable path. Therefore, undefined behavior should be avoided at all costs, since not only can it lead to program glitches, but it often becomes a source of vulnerabilities and a security threat. Examples of code that exhibits undefined behavior: array indexing beyond the bounds, integer overflow, division by zero, dereferencing a null pointer [1].
Compilers often use the vague semantics of a language to make assumptions about a program. For example, if you write something like int x = y/zthe compiler may assume that z cannot be zero, since division by zero results in undefined behavior, and the programmer clearly did not intend to write such code. Based on this information, it may attempt to further optimize the program as follows:
Program
int main(int argc) {
int div = 5 / argc;
if (argc == 0) {
printf("A\n");
} else {
printf("B\n");
}
return div;
}gcc -O2
.LC0:
.string "A"
.LC1:
.string "B"
main:
mov eax, 5
xor edx, edx
push rbx
idiv edi
mov ebx, eax
test edi, edi
jne .L2
mov edi, OFFSET FLAT:.LC0
call puts
.L1:
mov eax, ebx
pop rbx
ret
.L2:
mov edi, OFFSET FLAT:.LC1
call puts
jmp .L1clang-O2
main:
push rbx
mov ebx, edi
lea rdi, [rip + .Lstr]
call puts@PLT
mov eax, 5
xor edx, edx
idiv ebx
pop rbx
ret
.Lstr:
.asciz "B"As this example shows, clang relies on the fact that division by zero is undefined behavior. Accordingly, argc can never be equal to zero. Accordingly, the condition if (argc == 0) is completely excluded, since it is known that such a case will never occur [2].
Statically known undefined behavior
Yes, I knew that compilers can intelligently optimize a program if they assume that undefined behavior cannot exist in it. But I wondered what a compiler does if statically detects undefined behavior in a program — in other words, when we force the compiler to compile code that definitely contains undefined behavior (and both we and the compiler know about it). I really wanted to find a reason to use Compiler ExplorerI quickly ran a few experiments. The results will not surprise many of you (and these experiments, even if they deserve the name, are not exhaustive anyway), but my curiosity was satisfied. So I decided to share them here – in the hope that my readers will also be able to derive something useful from them.
Needed Goal zero
The simplest program I could come up with provokes undefined behavior in C by forcing division of a constant by zero. Here is the program and its output from gcc (v14.1) and clang (v18.1) compiled on x86_64:
Program
int main(int argc) {
int ub = argc / 0;
return ub;
}gcc -O2
main:
ud2clang-O2
main:
retDuring compilation, both gcc and clang issue a warning:
:2:17: warning: division by zero [-Wdiv-by-zero]
2 | int ub = argc / 0;
| ^True, while gcc compiled this program down to a single (invalid) instruction ud2clang reduced it to ret. In the case of undefined behavior, both approaches are acceptable, but they are very different: one approach crashes the program, while the other ignores the problematic code. [3].
What if we change the program a little by replacing the constant with a variable in the division operation?
Program
int main(int argc) {
int i = 0;
int ub = argc / i;
return ub;
}gcc -O2 -Wall
main:
ud2clang-O2-Wall
main:
retEven though both programs are unchanged in compiled form, we no longer get warnings (even с -Wall), even though both compilers can easily statistically (for example, by folding constants) find out that division by zero occurs in the program [4].
No guarantees
Let's add a few more lines before the division by zero and see how that affects the output:
Program
int main(int argc) {
int i = 0;
printf("before");
int ub = argc / i;
printf("%d", ub);
return ub;
}gcc -O2
main:
sub rsp, 8
mov edi, OFFSET FLAT:.LC0
xor eax, eax
call printf
ud2clang-O2
main:
push rax
lea rdi, [rip + .L.str]
xor eax, eax
call printf@PLT
lea rdi, [rip + .L.str.1]
xor eax, eax
pop rcx
jmp printf@PLTIt's not surprising that gcc persists with the crashing approach. Note, however, that it inserts a crash only after it compiles the division by zero, not before — for example, not at the beginning of the function. In turn, clang compiles both outputs, both before and after the division, simply removing the division operation itself. As with the code containing the division by zero, there are no guarantees about the code that causes this operation. Simply because there is undefined behavior in the program, no rules apply, and the compiler, at its own discretion, can crash the function immediately after entering it. [5].
If there is undefined behavior in a program, but no one uses it, are its echoes noticeable?
Do compilers take into account code that has undefined behavior but is never used in the program? This brings to mind a philosophical question – Can you hear the sound of a falling tree in a forest if no one is around? Let's try to figure it out:
Program
int main(int argc) {
int i = 0;
int ub = argc / i;
return 1;
}gcc -O2
main:
mov eax, 1
retclang-O2
main:
mov eax, 1
retAs we can see, the answer to this question is yes, and now both compilers have optimized away the division operation. Most likely, if the program used dead code pruning, the division operations would have been removed before the compiler had time to determine that this is undefined behavior. Again, it is important to understand that compilers ourselves choose this strategy (and only if we enable optimizations, otherwise the division is compiled as is). Even if undefined behavior is “not used”, this does not mean that the program does not contain undefined behavior. We were just “lucky” that the compiler removed the dead code before it realized that it was undefined behavior. This is not guaranteed to be the case with other compilers, nor is this behavior guaranteed to be consistent across compiler versions. It might just as easily crash the program or open the CD drive.
This meaning is poison
So, there are two questions left to answer: 1) why do we often not get warnings about undefined behavior in a program, even if the compiler was able to find it, and 2) why are clang (and sometimes gcc) so lenient about handling undefined behavior. Why do they compile (and execute) the code, rather than cause it to crash (e.g. by inserting an invalid instruction into it)?
The answers to both questions are given in post Chris Lattner. On the topic of warnings, Lattner explains that often a compiler could issue so many warnings that it would be useless (while still producing many false positives). It is also hard to determine who would and would not want to receive so many warnings (for example, no one cares about undefined behavior in dead code). As for the softness of the checks, especially in the programs above, Lattner sums it up well in the following line from his post:
“It is believed that arithmetic operations on undefined values result in undefined values, not undefined behaviors. The difference is that an undefined value will not format your hard drive or have other unwanted effects.”
Nowadays, LLVM uses mostly “poisoned» values that open the door to more optimizations than just 'undef', but the idea is the same: just because a value is the result of undefined behavior does not mean that any code using it should be immediately invalidated. For example, if you take a poisoned value and do and 0 on it, you can assume that the result will always be 0, no matter what the poisoned value is.
This is useful, for example, when the result of an undefined operation does not affect the execution of the rest of the program, as shown in the following example:
Program
int main(int argc) {
int i = 0;
// Разыменование нулевого указателя
int ub = *(int*)i;
int p = ub | 1;
printf("print");
if (p) {
printf("%d", ub);
}
return 1;
}gcc -O2
main:
mov eax, DWORD PTR ds:0
ud2clang-O2
main:
push rax
lea rdi, [rip + .L.str]
xor eax, eax
call printf@PLT
lea rdi, [rip + .L.str.1]
xor eax, eax
call printf@PLT
mov eax, 1
pop rcx
ret
.L.str:
.asciz "print"
.L.str.1:
.asciz "%d"Conclusion
Since a bitwise or with a non-zero value always evaluates to true, the if condition will always succeed, regardless of the specific value of ub. In LLVM, an arithmetic operation on poisoned values does not necessarily yield a poisoned value as a result. Here, the compiler is free to get rid of the condition in question. On the other hand, Gcc got rid of ud2 immediately when it noticed the null pointer dereference.
Acknowledgments: Thank you Edd Barrett And To Lawrence Tratt for comments.
Notes
[1] Although the examples given seem completely obvious, There are also more complex and difficult to resolve cases of undefined behavior.
[2] When I prepared this example, I was quite sure that gcc would do the same, and was surprised when it didn't. The situation itself is interesting, but beyond the scope of this article.
[3] The latter case may or may not have unpleasant consequences, depending on how exactly this value will be used after the return (and at the moment described, it will, in any case, be in the RAX register).
[4] You can force both gcc and clang to produce runtime errors by setting the option -fsanitize=integer-divide-by-zero. But this will have a negative impact on performance, and otherwise will not change the program in any way: gcc will still crash with ud2and clang ignores the division operation.
[5] In this situation, I wondered whether the tool could arbitrarily abort compilation if it could prove that there was undefined behavior in the code and that it would be executed. There is some discussion about this, but I have not found a definitive answer to my question. Of course, I can imagine that they don't do this simply because most programs would otherwise not compile at all.
