Unsafe multithreading or Race Condition

As I like, we will start with the conditional basics and gradually move deeper and deeper. Towards the end we will figure out how to use it. Many different sources were used when writing this article. What is this for? And besides, I’ll start with the words that I found in Boom’s article – more about him later. Well, enjoy reading.

Please remember that the use of acquired knowledge and skills must be within legal and ethical boundaries, and interfering with others’ networks without permission is unacceptable and illegal.

Table of contents

How does it work

A hacker walks into a hookah lounge, quest or bar, and he says you have a race condition!

Omar Ganiev

To begin with, as elsewhere – a definition.
Race condition is a situation in which multiple threads (or processes) simultaneously attempt to perform read or write operations on shared resources without proper synchronization. You can present them in a queue format, one by one.

The photo is not negative

Difficult? I understand you, when analyzing topics, it’s always like this. Let’s try to look at an example from life.
Imagine a site like Ya.market, where there are coupons for a 10% discount “best10”. Two threads could simultaneously query the database and confirm that the discount code “best10” was not applied to the cart, then both would try to apply the discount, causing it to be applied twice. Note that racing conditions are not limited to a specific web application architecture. It’s easiest to think about a multi-threaded application with a single database, but in more complex systems state is typically stored in even more places.

Single-threaded systems such as NodeJS are slightly less vulnerable, but there is still a possibility of similar problems occurring. In the last article, where I discussed the concept of race condition in WS, I was corrected that it does not depend specifically on NodeJS, and this interested me. And I began to look for where I could study something like this using an example, or better yet see it with a piece of code…And found. Although the article is from 2021, it shows how race conditions occur in NodeJS. The most interesting things happen there in the comments, as always on habr.

Typical race conditions

Almost all programs and web applications today use the so-called “multi-threaded» processing in which they are able to perform several actions simultaneously. While this allows applications to run significantly faster, it introduces potential errors if more than one process (or “thread”) tries to access the same data at the same time.

Typically, race conditions follow a certain pattern:

• Critical section: This is the section of code in which shared resources are accessed and modified.

• Synchronization: Lack of proper synchronization mechanisms can allow multiple processes to enter the critical section simultaneously.

• Unpredictable result: Due to concurrent execution, the final state of the shared resource becomes uncertain.

For example, when the SQL system performs an upgrade on a database, it creates a temporary file during the upgrade process. It is this temporary file that replaces the data in the database over time. In a well-timed attack, attackers can replace the temporary SQL update file of the administrative access table with their own, giving themselves administrator rights in the system.

Types of Race Condition

This part was written using material from the most important site for every pentester hacktricks

Limit-overrun / TOCTOU

This is the simplest type of Race Condition, where vulnerabilities appearing in places limit the number of times an action can be performed. For example, using the same discount code in an online store several times. A very simple example can be found in this article (vpn required) or in this report h1.

There are many variations of this type of attack, including:

• Redeeming a gift card multiple times

• Multiple product ratings

• Withdrawal or transfer of funds exceeding the account balance

• Reusing one CAPTCHA solution

• Anti-brute-force bypass

Hidden substates

Other, more complex RCs will use subsystems in the machine’s state, which could allow an attacker to abuse states they should not have access to, but there is a small window for the attacker to access them.

1. Predicting potential hidden and interesting subsystems.

   The first step is to identify all the endpoints that either write to or read data from it, and then use that data for some important purpose. For example, users may be stored in a database table that changes when they register, edit a profile, initiate a password reset, and complete a password reset. We can use three key questions to rule out endpoints that are unlikely to cause collisions. For each object and its associated endpoints, ask the following questions:

• How is the state stored?
  Data stored in a persistent server-side data structure is ideal for exploitation. Some endpoints store their state entirely on the client side, such as password resets done by sending a JWT via email – these can be safely skipped.
  Applications often store some state in the user session. They are often protected from nested states to some extent – more on this later.

• Are we editing or adding?
  Operations that edit existing data (such as changing an account’s primary email address) have a greater potential for “collisions”, while actions that simply append existing data (such as adding a secondary email address) are unlikely to be vulnerable to anything ‑except for limit‑overrun attacks.

• What is the operation based on?
  Most endpoints work with a specific record that is looked up using a “key”, such as a username, password reset token, or filename. For a successful attack, we need two operations using the same key. For example, consider two plausible password reset implementations:

  

2. Search for clues

At this point, it’s time to run a few RC attacks on potentially interesting endpoints to try to find unexpected results compared to normal ones. Any deviation from the expected response, such as a change in one or more responses, or a second-order effect, such as different email contents or a visible change in the session, can be a clue that something is wrong.

3. Prove the concept
   The final step is to prove the concept and turn it into a viable attack.

When sending a batch of requests, you may find that an early couple of requests cause a vulnerable end state, but subsequent requests overwrite/unauthorize it and the end state is rendered unexploitable. In this case, it is necessary to exclude all unnecessary requests – two are enough to exploit most vulnerabilities. However, if you reduce the number of requests to two, the attack becomes more time-sensitive, so the attack may need to be repeated many times or automated.

Time Sensitive Attacks

Sometimes a race condition may not be detected, but a timed request delivery technique may still reveal the presence of other vulnerabilities.

One such example is the use of high-resolution timestamps instead of cryptographically secure random strings to generate security tokens.

Consider a password reset token that is randomized by timestamp only. In this case, you can initiate two password resets for two different users who use the same token. All you have to do is set the time for the requests so that they generate the same timestamp.

To confirm, for example, the previous situation, you can simply request 2 password reset tokens at the same time (using a single packet attack) and check if they are the same.

Detection

The discovery process is quite simple. In general terms, all you need to do is:

• Identify a one-time or rate-limited endpoint that has some security impact or other useful purpose.

• Issue multiple requests to this endpoint in quick succession to see if you can exceed this limit.

The main challenge is timing the requests so that at least two race windows coincide

Even if you send all requests at the same time, in practice there are various uncontrollable and unpredictable external factors that affect when the server processes each request and in what order.

I really liked the diagrams from portswiggerI really don’t know if this will violate their rules, but I will translate them into Russian for better understanding by the audience.

jitter – literally – “jitter”. That is, an unpredictable and unregulated change in some parameter.

This part was taken from the same portswigger, and you can purchase or use the community edition of burp itself or use the version free from greed (carefully stick out the cracker, but I advise 0daylab

Burp Suite 2023.9 ​​adds powerful new capabilities to Burp Repeater that make it easy to send a group of parallel requests in a way that significantly reduces the impact of one of these factors: network jitter. Burp automatically configures the technique used depending on the version of HTTP supported by the server:

• For HTTP/1 The classic last byte synchronization technique is used.

• For HTTP/2 uses a single-packet attack technique first demonstrated by PortSwigger Research at a conference Black Hat USA 2023.

A single-packet attack allows you to completely neutralize network jitter interference by using one TCP packet to simultaneously execute 20-30 requests.

While often only two requests can be used to launch an exploit, sending a large number of requests helps reduce internal latency, also known as server-side jitter. This is especially useful during the initial discovery phase.

I would especially like to note that the same jitter is not the only problem. Many of you have encountered high ping in games, here the operating principle is similar. The closer you are, the more likely you will be able to operate the RC.

Operation Race condition

Here you can say a simple phrase – “Let’s just turn on the intruder or turbo intruder in many threads and everything will work out. Where is our $300 per BB?“. Yes, there is some truth in it, but it is not always effective. It is clear that this remains the main method, but there is more. To explain, I will take the article Boom aka the best mustache of the community.

Original articleI highly recommend checking it out.

Splitting an HTTP request into two parts

First, let’s remember how an HTTP request is formed.
Well, as you know, the first line is the method, path and protocol version:

GET / HTTP/1.1

Next are the headers before the line break:

Host: google.com
Cookie: 

a=1

But how does the web server know that the HTTP request has ended?

Let’s look at an example, enter nc google.com 80, and there

GET / HTTP/1.1
Host: google.com

After you press ENTER, nothing happens. Click again and you will see the answer.

That is, for the web server to accept an HTTP request, two newlines are required. A correct request looks like this:

GET / HTTP/1.1\r\nHost: google.com\r\n\r\n

If this were a POST method (don’t forget about Content-Length), then the correct HTTP request would be like this:

POST / HTTP/1.1
Host: google.com
Content-Length: 3

a=1

or

POST / HTTP/1.1\r\nHost: google.com\r\nContent-Length: 3\r\n\r\na=1

Try sending a request like this from the command line:

echo -ne "GET / HTTP/1.1\r\nHost: google.com\r\n\r\n" | nc google.com 80

As a result, you will receive a response, since our HTTP request is complete. But if you remove the last character \nthen you won’t get an answer.

In fact, for many web servers it is enough to use as a wrapper \nso it is important not to swap \r And \notherwise further tricks may not work.

What does this give? You can simultaneously open many connections to a resource, send 99% of your HTTP request and leave the last byte unsent. The server will wait until you send the last line feed character. After it is clear that the main part of the data has been sent, send the last byte (or several).
This is especially important when it comes to a large POST request, for example, when a file needs to be uploaded. But even in a small request, this makes sense, since delivering several bytes is much faster than simultaneously delivering kilobytes of information.

Again, I highly recommend reading it, there are some great videos of trains there

How to hack

In general terms, all you need to do is:

• Identify a one-time or rate-limited endpoint that has some security impact or other useful purpose.

• Issue multiple requests to this endpoint in quick succession to see if you can exceed this limit.

The main challenge is timing the queries so that at least two race windows coincide, causing a “collision”. Often this window is only milliseconds, but it can be shorter, as we discussed above.

As stated earlier, even if you send all requests at the same time, in practice there are various uncontrollable and unpredictable external factors that affect when the server processes each request and in what order.

Tools

Typically, a turbo intruder is used for operation, but we will also look at racepwn.

Burp Turbo Intruder

We can try to exploit the Race condition through Repeater, by sending grouped HTTP requests, but why do this if there is a Turbo intruder

And here the question arises – “Why do we need a Turbo Intruder if we have a standard Intruder?“. To put it as briefly as possible, Turbo Intruder is Intruder at maximum speed, with anabolic steroids (22 built-in) and the ability to customize attacks using Python (if your option is not in the list, then you must have skills in writing scripts in this language).

As in a regular intruder, if you select part of the request before sending it to the extension, it will be replaced in the request window with %s. This is analogous to symbols § §and you can move it anywhere in the request.

Now a little about its features (as in advertising):

• Fast: Turbo Intruder uses an HTTP stack hand-built from the ground up with speed in mind. As a result, it can seriously outperform even fancy asynchronous Go scripts on many targets (in fact, you can choose a stack, and most of them will be familiar to you).

• Convenience: Boring results can be automatically filtered out using an advanced def algorithm adapted from Backslash Powered Scanner. This means you can launch an attack and get useful results in two clicks.

• Scalability: Turbo Intruder can achieve flat memory usage, allowing for robust multi-day attacks. It can also be run in a headless environment via the command line.

• Flexibility: attacks are configurable using Python. This enables complex requirements such as signed requests and multi-step attack sequences. Additionally, a custom HTTP stack allows you to handle malformed requests that break other libraries.

For detailed study – Link

racepwn

RacePWN is a utility written in golang that implements the librace interface by setting parameters through a configuration written in json.

[
    {
        "race": {
            // Установка параметров гонки
            "type": "paralell", // режим гонки
            "delay_time_usec": 10000, // временная задержка между двумя частями запроса
            "last_chunk_size": 10 // размер куска по последнему запросу
        },
        "raw": {
            "host": "tcp://localhost:8080", // имя хоста и порт
            "ssl": false, // использовать флаг ssl
            "race_param": [
                {
                    // параметры гонки
                    "data": "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n", // raw HTTP запрос
                    "count": 100 // количество пакетов
                }
            ]
        }
    }
]

Two operating modes are supported. It works as a console utility and a web service.

console utility:
In this operating mode, a configuration file is supplied to the application input. The application makes requests to the server.

racepwn < config.json

web service:
In this mode, the utility starts working as a web service. To work, you need to make a POST request along the /race path with the contents of the configuration file. The hostname and port must be specified using the -hostname flag.

racepwn -host "localhost:8080"

Example:

curl -i -X POST localhost:8080/race --data @<(cat config.json)

How to protect a web application

• One of the most effective ways to prevent race condition vulnerabilities is to use locking mechanisms. Locking mechanisms ensure that only one process can access a shared resource at a time, preventing other processes from interfering with the resource.

• Semaphores is a type of locking mechanism that is often used to prevent race conditions. Semaphores work by assigning a value to a resource indicating whether it is available or not. When a process tries to access a resource, it checks the value of the semaphore. If a resource is not available, the process waits until it becomes available.

• Mutexes is a type of locking mechanism similar to semaphores. Mutexes work by allowing only one process at a time to access a shared resource. When a process tries to access a resource, it checks the state of the mutex. If another process is already accessing the resource, it waits until the resource becomes available. Most programming languages ​​have built-in data locking functionality; for example, Python has “threading.Lock” and Go has “sync.Mutex”.

• Atomic operations are a type of low-level synchronization mechanism. It ensures that process operations are completed in one, indivisible step. This prevents other processes from interfering with the resource while the operation is in progress.

Race Condition in BugBounty

In this part of the article I will attach a list of 15 reports on h1 (I hope that disclosions from ru platforms will appear soon).

Where can you practice?

Conclusion

In conclusion, I would like to say thank you to all those who helped with this article. There are many authors who may not know that I mentioned them here, but still. I hope the material was not too loaded and complicated.

PS You can find more similar information and good memes here