How to fuzz a REST API using RESTler. Part 3
Fuzzing Modes
The first thing you need to understand is how RESTler chains requests. A fuzzer can use various strategies to create sequences. Let's look at each of them in detail.
BFS
The first strategy is BFS (breadth first traversal). The basic algorithm used in this mode looks like this:
The sequence of requests increases at each iteration as requests with satisfied dependencies are added to the end of the chain. Sometimes not all parameters for sending a request are met; in this case, you must first create an object using the POST method, and only then request it.
For each added request, the parameter is specified if its type in the grammar is specified as restler_fuzzable. Combinations of such criteria are formed according to the specified values in the fuzzing dictionary. Next, a chain of requests is executed and the server response is checked: if it has a valid status code, then the new sequence is saved. Otherwise, it is rejected, and the resulting error code is recorded for analysis and debugging.
Dynamic objects created during the execution of request chains are saved and, if necessary, used in subsequent requests. By default, every query with multiple restler_fuzzable types generates every possible combination of values, so large dictionaries can have an astronomical number of combinations.
Let's look at the picture above again: in the EXTEND function, for requests with satisfied needs, all valid combinations of request chains of length n+1 are formed. Since this function is called at each iteration of the main loop (line 8), the algorithm combines all sequences of requests – this is a breadth-first traversal.
BFS-Fast
When using BFS-Fast in the EXTEND function, each request is appended to at most one sequence. The result is a smaller newSeqSet that covers (i.e. includes at least once) every request, but does not generate all valid request sequences.
Let's return to the algorithm in the picture and see that BFS-Fast, like BFS, checks each restler_fuzzable type of query being executed on all iterations of the main loop on line 8. This mode still provides full grammar coverage, but with fewer query sequences, which allows it to increase the length of chains faster than with breadth-first traversal (BFS).
RandomWalk
In the RandomWalk strategy in EXTEND, two cycles of lines 16 and 17 are eliminated. Instead, the function returns a new sequence, which consists of requests with the necessary parameters and is generated by randomly selecting one seq request chain in seqSet and a request in reqSet. The function randomly selects such a pair until all its dependencies are satisfied. Thus, this strategy will explore the search space of possible query sequences deeper and faster than BFS or BFS-Fast.
When RandomWalk cannot extend the request chain further, it starts with an empty sequence. Note that RandomWalk does not retain past chains between restarts and may regenerate them in the future.
There is no clear answer to the question of which of the 3 modes to choose. Different search strategies may well complement each other. When running the fuzzer, BFS mode is used by default. But you can specify the strategy in the console using the parameter
--search_strategy [bfs-fast(default), bfs,bfs-cheap,random-walk]
or in the Engine_settings configuration file (the next time the fuzzer is launched, the path to the file is specified in the –settings parameter):
“fuzzing_mode”: “bfs”;
Annotations
There are many tools on the market that can receive specifications as input and send queries. You might have a question: why is RESTler better than other fuzzers?
The answer is simple: RESTler allows us to provide annotations with information about the dependencies between producer-consumer requests. This is useful when the fuzzer cannot automatically infer dependencies due to lack of “linking”. In order to add such a connection, you need to add an annotation at the RESTler compilation stage in the following way:
In json format we indicate:
producer_endpoint: the “path” of the function that “produces” the value for the consuming function;
producer_method: HTTP method of the producing function;
producer_resource_name: The name of the value produced. Can be in the response body, response header, or request path;
consumer_param: name of the parameter in the consumer request;
consumer_endpoint: user function path;
consumer_method: HTTP method of the consumer function;
except: list of functions that are not consumers;
description: description.
After compiling the specification, we get the following grammar:
As we can see, RESTler saves the IDs of the created projects for subsequent requests.
In the figure below we can see that the previously saved command ID is written into the required field:
This feature sets RESTler apart from its competitors because it allows you to:
increase source code coverage;
avoid a large number of rejected requests;
build more efficient request sequences;
make it easier for the fuzzer to build chains of requests;
solve the problem of an empty application.
Problems in preparing for fuzzing and ways to solve them
Creating or reconfiguring a specification using parsers can lead to a lot of errors, since they work quite mediocre. Let's look at a few problems that may arise when preparing for fuzz testing and tell you how you can solve them.
Preparing the circuit
In the example below you can see that the address initially contains the specified username. When compiling such an example, RESTler will send all requests specifically to the address /rest/user/username , since the function parameters do not contain a description that this is an inpath parameter and can be changed.
Result after manual correction:
After these operations, we can check the resulting grammar and make sure that the username will now be fuzzed correctly:
Lack of typing
Often the specification does not specify the type of data that can be passed in a particular parameter. Of course, RESTler will try to change this criterion itself to test validators. However, during long-term testing, we will send a large number of false requests, which has a bad effect on testing efficiency.
Pre-built web application
We need to have a non-null application. This issue complicates the testing phase when we want to determine which handles can even work with RESTler. For example, there is an interface that returns information about the user, but there is no user itself. In this case, we will receive a 4xx error for each request and will not be able to understand whether this function can be tested at all. Even if we create the user manually, when we run RESTlertest again we will receive a response (using a different interface) that “such a user already exists.”
To solve this problem, it is necessary to create a specification with such interfaces separately and before that run RESTlertest, which will perform a kind of unit tests, thereby creating the necessary data in the database. You also need to add interfaces for deleting this data to return the application to its original state. Annotations partially cope with this task.
Specification parsing
In most cases, when sending any data, we use the JSON format, which poses some difficulties when conducting fuzzing. Let's say we have a user creation interface /user/create/. This is what the json we send looks like:
{
"login":"admin",
"password":"admin",
"info":{"age":"99","city":"Moscow","sex":"male"}
}
Now let's look at it in the specification:
/v2/user/create:
post:
tags:
- General
summary: api/v2/users/
requestBody:
content:
application/json:
schema:
type: object
parameters:
- in: body
name: login
example: 'admin'
name: password
example: 'admin'
name : info
age: 99
city: 'Moscow'
sex: male
When compiling the specification, RESTler will not understand that it received a structure as input. Because of this, we will end up with the following grammar:
That is, with each mutation, the JSON itself will change, and not its variables. This means that all requests will be rejected at the stage of checking the validity of the body of the JSON itself and the coverage in this case will be minimal.
Here's what the correct specification should look like:
/v2/user/create:
post:
tags:
- General
summary: "Create a new user"
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
login:
type: string
example: "admin"
password:
type: string
example: "admin"
info:
type: object
properties:
age:
type: integer
example: 99
city:
type: string
example: "Moscow"
sex:
type: string
example: "male"
Here, during compilation, we receive parsed JSON, in which only the value of the parameters will mutate, and the structure itself will maintain its integrity:
As you can see, the body of the sent data is divided into restler_static And restler_fuzzable. You can guess by the name what the difference is between these types. This way we can create very complex structures and mutate them correctly. This approach will help RESTler determine the parameters and automatically remove some of them or add your own during the fuzzing process.
In the example in the picture above, we will perform grammatical fuzzing at the level of the JSON itself. This will allow you to discard a large number of invalid queries and increase coverage. In addition to format validators, handler functions will be used, which will have a positive effect on the quality of testing.
Converters
The problem with converters is that the vast majority of them simply don't work. And before using them, you have to mask the information in the specification.
Problems during fuzzing and ways to solve them
The problems discussed above mostly related to the preparatory stage, that is, they affected the specification itself, and not the fuzzing process.
All applications implement response statuses differently. For example, 200 might contain LogTrace in the answer. Sometimes we receive an authorization token, but the status is 4**. In such cases, it is worth initially analyzing the application itself, taking into account the features at all stages of testing. RESTler allows you to manually specify which responses may potentially contain errors, and also gives you the opportunity to work with them. This can be done using the parameter custom_bug_codes:
{
"custom_bug_codes":["201","203"]
}
Specification breakdown
To optimize the work of a wrestler, it is recommended to break the API scheme into subsets, taking into account the functionality of the interfaces themselves. Using user-rest as an example:
GET:/user/login – obtaining information about the user using the login;
PUT:/user/login – changing user information;
GET:/user – obtaining a list of all users taking into account filters;
POST:/user – creating a user;
POST:/user/status – status change;
POST:/user/password – change password.
This breakdown will solve the following problems:
a large number of false requests at launch;
creation of low-quality request chains (combining requests with completely different functionality into chains);
low performance.
Files in response
Quite often you can find problems in the files that the server sends us. For example, when we test the filter report interface, it may respond with a PDF of the wrong size, content, or timing.
Another problem may be data encoding. For example, RESTler will fail with an error when trying to decode any data received from the server:
RESTler solves this problem by skipping decoding errors. This can be done using the parameter "ignore_decoding_failures": True. For these types of requests, it is also recommended to increase the response time.
Conclusion
So, we described the stages and nuances of testing, looked at what vulnerabilities RESTler helps to detect, what problems arise when preparing and conducting fuzzing, and how they can be solved.
As a result of our research on API fuzzing, we can safely conclude that RESTler is the first automatic tool for analyzing the state of cloud services through their REST API, allowing you to customize the testing process as accurately as possible. Thanks to this, the fuzzer can be used for many purposes:
Grammar-based fuzzing. Allows you to build a request structure with the addition of dynamic and static parts, as well as dynamically generated values, such as tokens;
Generating tests based on feedback. Makes it possible to save information about an object that was created/modified/deleted/moved for use in subsequent requests;
White box fuzzing. By analyzing the source text of the program, it helps to more accurately configure the grammar, input value generator and dictionary;
Penetration testing. The fuzzer finds API vulnerabilities by analyzing the errors found, and also makes it possible to use your own dictionaries, which can contain pre-prepared loads for automatic pentesting;
Unit tests. RESTler allows you to start fuzzing in TEST mode, essentially running a kind of unit test using only the specification;
Stress Testing. Given that fuzzing involves a huge number of requests, the tool can be used to generate high traffic with random values and monitor the applications under test in terms of performance by pre-configuring the request generation time, the number of parallel requests and the timeout.
All of the above makes RESTler stand out from its competitors. The tool opens up a wide range of possibilities for improving the security of application APIs.
