Give me 15 minutes and I'll change your view of GDB

The material was prepared based on the speech from CppCon 2015 “Greg Law: Give me 15 minutes & I'll change your view of GDB” (available at link ). I changed and corrected many points, so it’s worth considering that the translation is quite free.

And yes, let’s put aside the question of how convenient or inconvenient a program GDB is in general, and what is, in principle, better to use for debugging: this article will discuss working with GDB.

This article will look at debugging C code on Linux.

Introduction

GDB is an incredibly powerful tool, and although it is very easy to get started with, GDB is not intuitive: many of the utility's features are hidden from the user's view. And to start using GDB to its fullest, you need to spend a lot of time studying the documentation.

However, there are some things about how the debugger works that you just need to see once to significantly improve your experience.

This article will look at some of them.

Typical GDB usage example

So let's see how we would most likely use GDB for a trivial task.

Let's say we have the following code in C:

#include <stdio.h>

int main(void)
{
  int i = 0;
  printf("Hello world!\n");
  printf("i is %d\n", i);
  i++;
  printf("i is now %d\n", i);
  return 0;
}

The example is not much more complicated than a typical “Hello World”, but it has a few lines, so we can look at it in the debugger.

Let's compile the program and run it in GDB:

gcc -g hello.c
gdb a.out

In the debugger itself we are faced with something like this interface:

In this interface, to navigate between breakpoints, or to view information about the code of the executing program, we will execute commands like stepi, next, disas or list. And constantly use list And disasto put it mildly, not very convenient.

And let's be honest, this interface is quite sloppy, and as if it came to us from the seventies.

Text User Interface

Activation and what is it?

And here this mode of using GDB comes to our aid: TUIor Text User Interface (which, of course, is not a good name, because by default GDB already uses some form of text interface).

To activate TUI, you need to press the keyboard shortcut Ctrl+X A (don’t ask why this is, or just start the debugger with the command gdb -tui.

After activating TUI we see the following:

Here we have pseudo-graphics in GDB with a preview of the executable program code!

Of course, this interface is also made in retro style, but it is convenient and functional.

In the window with the program code, we are shown breakpoints and the current line of executing code.

Standard output and window redraw

This interface also has its drawbacks: if you enter next For our test program, its output will break the text interface a bit:

But this can be easily fixed by pressing the shortcut Ctrl+Lwhich “redraws” the GDB screen.

Window configuration

The program source code and command line are not all the information that can be viewed in TUI. If you press Ctrl+X 2then we will see the assembly code of the executable program:

If you press more Ctrl+X 2then we will have other TUI modes open with other windows.

You can also change the display of some windows directly through the command line, for example using the command tui reg float.

You can use the up/down arrows to scroll through the program's source code. Shortcuts are used to navigate through GDB commands. Ctrl+P And Ctrl+N (Previous and Next).

This way you can very easily customize the debugger interface for yourself without wasting extra time.

Python interpreter

Yes, GDB (starting from version 7) has a built-in Python interpreter!

Using the built-in interpreter, you can write “general purpose” Python programs, that is, those that could work separately from the debugger.

For example, this is what getting the PID of the current process looks like:

(gdb) python
> import os
> print("my pid is %d" % os.getpid())
> end
my pid is 5228
(gdb)

This way you can do quite a lot of things, for example, define functions that can then be called by name.

But the Python interpreter is not just exists inside GDB, it is actually closely related to debugging and can use data about the current session. For example, using the built-in interpreter, you can view a list of current breakpoints:

(gdb) python print(gdb.breakpoints())
<gdb.Breakpoint object at 0x.....> <gdb.Breakpoint object at 0x.....>

Or create a new breakpoint (in this example on the 7th line):

(gdb) python gdb.Breakpoint('7')
breakpoint 4 at 0x....:  file hello.c, line 7.

Reversible Debugging

Let's take for example a new test program, bubble_sort.c:

#include <stdlib.h>
#include <time.h>
#include <stdbool.h>

void sort(long* array)
{ удобнее
  int i = 0;
  bool sorted;

  do {
    sorted = true;

    for (int i = 0; i < 31; i++)
    {
      long *item_one = &array[i];
      long *item_two = &array[i+1];
      long swap_store;

      if (*item_one <= *item_two)
      {
        continue;
      }

      sorted = false;
      swap_store = *item_two;
      *item_two = *item_one;
      *item_one = swap_store;
    }
  } while (!sorted);
}

int main()
{
  long array[32];
  int i = 0;
  srand(time(NULL));
  for (i = 0; i < rand() % sizeof array; i++)
  {
    array[i] = rand();
  }

  sort(array);

  return 0;
}

We have a very simple program that sorts an array of 32 random numbers like long bubble method.

But the problem is that, although rarely, this program throws an error Segmentation fault (core dumped) (or how this error appeared on my machine, *** stack smashing detected ***: terminated).

The normal course of action in this situation for debugging would be:

ls -lth core*
gdb -c core.xxxxx

When looking at the information about the program crash, we find that we do not have any information about the program stack, so most likely the problem is with some kind of bug that breaks the stack. That is, the generated core file will not be of any use.

So, in such a situation, reverse debugging in GDB will help us.

We need to find contextual information about the program at the moment when it happens segfaultbut at the same time, so that this segfault happened, we need to run the program quite a few times. What to do?

So, to start, let's just launch GDB:

gdb a.out
(gdb) start
(gdb) next

Then we will set breakpoints at the entrance to the main function and at the point _exit.c:30 (service file, the code from which is called when the program exits).

(gdb) b main
(gdb) b _exit.c:30

And then, for these breakpoints, we need to write the following code:

(gdb) command 3
run
end
(gdb) commnand 2
record
continue
end
(gdb) set pagination off

What did we write?

• When program execution reaches breakpoint 3 (that is, before _exit.c:30), the program starts its execution from the beginning

• When program execution reaches breakpoint 2 (that is, the entry point into the main function), GDB will begin “recording” events occurring in the program and continue execution

• What a team set pagination off just makes the GDB output a little more readable.

And now when the command is executed runGDB will run the source program in a loop until it encounters the error we need.

After some time of waiting, we will still stumble upon an emergency situation. Here we need to use contextual information to understand what actually happened wrong.

1. We turn on TUI mode to make sure at what stage of execution we are. At first it will show us some incomprehensible line from libc. To get useful information about the work bubble_sort.c using the command reverse-step let's get to the moment when we executed the last line of the function main.

2. At this stage, we look at when we went into an emergency state: including viewing the current assembler code through Ctrl+X 2 We clearly see after which instructions we go into an emergency state:

3. If you press further several times stepwe will see that we go to code that is already writing an error message (*** stack smashing detected ***: terminated). It turns out that the error is precisely that someone damaged the stack. Let's look at the stack pointer:

4. Here we see the address that the stack pointer refers to. Let's look at the history of interaction with data at this address:

(gdb) watch *(long**) 0x......
(gdb) reverse-continueCppCon

…or we can place breakpoints during the program execution, and then using command <4/5/....> configure for each breakpoint to display the current address to which the stack pointer refers.

5. Thus, we will “move back in time” and see who broke our stack. As a result, we come to the conclusion that the stack change occurred on line 39, where we write the data to the array array (which was basically expected). After viewing the output of the command print i it becomes obvious that when filling out array random numbers counter i went beyond the boundaries of the array.

The error lies in the expression i < rand() % sizeof arraywhere we should count amount of elements in an array, not number of bytes.

Thus, using GDB, we were able to detect which part of the code was causing the error in a given example.

useful links