Introducing DevSecOps into the Development Process. Part 4. Test-time Checks Stage, Tools Review

Hello! This is Oleg Kazakov from Spectre.

In the previous part of the article I talked about security control of build artifacts during security testing. Today we will talk about the next stage of DevSecOps — Test-time Checks.

The Purpose of Test-time Checks

Based on the results of the previous articles, we can check the code for security, build secure builds, deploy these builds on test benches and encounter new vulnerabilities.

The purpose of Test-time Checks: functionality testing. Usually these are classic autotests and manual testing, but other tools can be used. Let's look at them in more detail.

DAST

DAST (Dynamic Application Security Testing) is a method of application security testing. It is used to identify vulnerabilities in web applications and services in real time. Unlike static code analysis (SAST, which was written about in the second article), which analyzes the source code of an application, DAST tests the application in its running state. It simulates attacks and checks how it responds to different types of input.

This is an emulation of the actions of attackers.

OWASP ZAP

The most popular DAST tool is OWASP ZAP.

DAST is in GitLab, in its Ultimate version, but inside it uses the same OWASP ZAP.

Similar to Gemnasium, which we talked about in the previous partthis tool is open source, meaning you can safely use it: https://gitlab.com/gitlab-org/security-products/dast.

The only problem is that it is difficult to understand the script and how it works. But here comes good documentation the service itselfall that remains is to transfer this to the capabilities of GitLab itself.

After examining the DAST template in GitLab and the command parameters in the OWASP ZAP documentation, we get the following.

We take the configurations from the previous article as a basis (there are already build and test stages). We create our own stage (dast_custom):

stages:
  - build
  - test
  - dast_custom

We create our own task so that there is no conflict with the standard ones in GitLab, we specify the image, we specify the artifacts:

dast_custom: 
 stage: dast_custom
  image: registry.gitlab.com/gitlab-org/security-products/dast
  variables:
    #DAST_FULL_SCAN_ENABLED: "true" # включение полного сканирования (пассивное и активное)
    #DAST_BROWSER_SCAN: "true" # использовать браузерный сканер GitLab DAST
  tags:
    - docker
  allow_failure: true
  artifacts:
    when: always
    paths: [report.html]
  script:
    - mkdir /zap/wrk/
    - /analyze -t $APP_LINK -r report.html
    - cp /zap/wrk/report.html .

The script itself creates a working folder, then the artifact is taken out of there. Differences from the tasks in the previous articles:

• the APP_LINK variable contains the address of the deployed stand that we will “break”;

• This utility already outputs information about the tests performed and vulnerabilities found inside the job, so there is no need for additional processing;

• The utility allows you to generate a visual html report, which we will definitely use.

You can use the original GitLab wrapper instead ZAP imagebut let's leave that for later.

Example setup in GitLab

So, the task is written, it needs to be tested on something. For a visual example, let's try scanning a service from the same OWASP, which is called WebGoatIt is essentially a Penetration Lab, a deliberately insecure web application that aims to teach practical web application security.

I would also like to point out separately that it is often impossible to get into the application being tested without authentication, but you want to check for vulnerabilities. For this, variables in GitLab will come to the rescue, which allow you to pass authentication:

You can simply add the variables block to the task, but I don't recommend doing this. It's better to put it in environment variables in GitLab itself. Below is an example of filling in variables:

variables:

#DAST_FULL_SCAN_ENABLED: “true” # enable full scan (passive and active)

#DAST_BROWSER_SCAN: “true” # use GitLab DAST browser scanner

DAST_AUTH_URL: “http://webgoat.example.ru/WebGoat/login”

DAST_USERNAME: “test123”

DAST_PASSWORD: “test123”

DAST_USERNAME_FIELD: “id:exampleInputEmail1”

DAST_PASSWORD_FIELD: “id:exampleInputPassword1”

DAST_AUTH_VERIFICATION_SELECTOR: “id:restart-lesson-button”

The following are specified here: the link where the authorization form is located; login and password; selectors of DOM elements where the login and password should be entered; a selector by the presence of which it can be understood that the authorization was successful.

After setting up and starting, we see the scanning result, which is also displayed in the task itself:

Below is what the html report looks like. Here is the output of the number of vulnerabilities by levels, then a list of all vulnerabilities with the output of the number of repetitions of these vulnerabilities and detailed information on each vulnerability: description, location, links to a detailed description in various known lists of vulnerabilities, such as CWE.

OWASP ZAP Modes

Above in the job code you may have noticed two variables commented out — DAST_FULL_SCAN_ENABLED and DAST_BROWSER_SCAN. They allow you to select the testing mode. OWASP ZAP allows testing in two modes:

By default, DAST in GitLab scans in passive mode. Passive scanning does not modify requests or responses in any way and is therefore safe to use.

But there is also active scanning, in which case the script performs “attacks” and can potentially run for a long time, depending on the size of the application. Therefore, I do not recommend using it often.

The DAST_FULL_SCAN_ENABLED variable specifies whether to scan in passive or active mode.

GitLab also offers to use its browser scanner, for this you need to set DAST_BROWSER_SCAN to true.

You can read more about scanning modes in GitLab here.

Below is the output of the task during a full scan:

Below is a full scan report – we see that a high-level vulnerability has appeared – SQL injection:

This is a way to test a deployed web application. Passive testing can be safely integrated into your CI/CD, as it is performed quite quickly.

Active testing can take a lot of time and is unlikely to be practical within a standard CI/CD pipeline, but you can do such checks regularly, regardless of the release cycle – either completely separate from CI/CD, or do checks, for example, on a schedule, this functionality is in GitLab There is.

DAST API

What I wrote above is essentially scanning the front end. How do we check the back end? Here, the following tool from the same OWASP comes to our aid — API Scan.

API Scan — a tool for performing scanning of APIs defined in OpenAPI, SOAP or GraphQL via a local file or URL.

It works in a similar way to ZAP itself, moreover, about a year ago ZAP was combined with API Scan into one common image, that is, the same image can be used for both tasks.

ZAP – Scan API integration in GitLab

Let me give you an example of integrating ZAP — API Scan into GitLab.

Add a stage:

stages:
  - build
  - test
  - dast_custom
  - dast_api_custom

Add a task:

dast_api_custom:
  stage: dast_api_custom
  image: ghcr.io/zaproxy/zaproxy
  tags:
    - docker
  artifacts:
    when: always
    paths: [report.html]
  script:
    - mkdir /zap/wrk/
    - cp examples/dast_api/openapi.json /zap/wrk/openapi.json
    - zap-api-scan.py -t openapi.json -f openapi -r report.html -I -d
    - cp /zap/wrk/report.html .

Overall it is very similar to OWASP ZAP, the only differences are as follows:

• here the image used is not from GitLab, but from OWASP itself, since GitLab does not have it in the public domain;

• instead of the stage URL, we can specify either the documentation URL or a file with the documentation description. Our example uses reading documentation in OpenAPI format from a file;

• there is an interesting parameter “-I”. In essence, this is ignoring errors. By default, if at least one vulnerability is found, the script will end with an error and this will lead to the failure of the entire task. But we need a report, so we add this parameter.

We see the result of the check: 96 checks passed, 5 vulnerabilities found:

Let's look at the report, again everything is very similar to DAST:

This way we can scan the backend API.

Conclusion

Today we talked about one of the final stages of DevSecOps. All the code developments are in this repositories. And in the final part, I will talk about the process of checking the infrastructure – Deploy-time Checks: about the tools and methods of checking at this stage.