how to organize testing of backing on a project

Hi all!

My name is Grebenyuk Guzel, I am a QA-head of the testing group at AERO. We are engaged in the development of eCommerce and data solutions for large businesses.

In this article I want to talk about how we organized back-end testing on projects.

The main testing tool chosen was Postman. The checks have gone through various stages of evolution. At first, we used this tool only for visual check of individual backend methods. The check consisted of importing either a yaml file with a collection of a list of methods of a microservice, or as an import of a separate url request. In this case, various combinations of header checks, response and request bodies, response codes, etc. were checked.

Then we began to use environment variables for testing on different stands with different sets of test data, but still these checks remained manual and consisted of visual checks of query responses in collections.

The next step was to form e2e chains of methods by receiving variable values ​​from one request and passing them as input parameters to the next request. This gave impetus to the active use of the Test tab in Postman and the formation of snippets for parsing responses and obtaining the necessary values. As a result, we formed templates for basic tests, which manual testers began to use on all projects.

As part of these tests, we checked response codes, response times, field types, json schemes, and requirements for restrictions on the received values. This gave a good increase in regression speed and testing quality.

Next, the task arose to integrate the launch of these tests into CI, and save the results of the runs into our TMS AllureTestops. This task was decomposed into the following steps:

1. Created collections of related queries in Postman;

2. Uploaded the exported collection and environment variables to the created repository;

3. Created gitlab_ci.yml file.

List of files in the repository:

Env.postman_environment.json	
Test.postman_collection.json	
gitlab-ci.yml	
README.md

Setting up gitlab-ci.yml

The following settings are required for gitlab-ci.yml:

1. Set stage;

2. Set variables to connect Allure and form a clear launch name;

3. Install an interface for running launchers and unloading reports allurectl;

4. Install newman and configure the command to launch the corresponding collections of services and regression, with the formation of reports in the allure-report format;

5. Configure the uploading of the results obtained at each launch to AllureTesops.

stages:
   - test
   - allure


variables:
   ALLURE_ENDPOINT: https://своё название.qatools.cloud
   ALLURE_PROJECT_ID: 444(id проекта)
   ALLURE_RESULTS: allure-results
   ALLURE_LAUNCH_NAME: ${CI_PROJECT_NAME}-${CI_JOB_NAME}-${SERVICE_NAME}-${CI_PIPELINE_ID}-${ENV} 


.deploy:
   before_script:
       - wget https://github.com/allure-framework/allurectl/releases/latest/download/allurectl_linux_amd64 -O ./allurectl
       - chmod +x allurectl
   script:
       - ./allurectl upload allure-results
   after_script:
       - ./allurectl job-run stop


postman_test:     
   stage: test
   image: node:16   
   script:
       - npm install -g newman
       - newman --version
       - npm install -g newman-reporter-allure
       - newman run --verbose "Tests.postman_collection.json"  -e "Env.postman_environment.json" --reporters allure,cli --reporter-allure-export allure-results
   allow_failure: true    
   artifacts:
       when: always
       paths:
           -  allure-results

allure_test:
   rules:
     - when: always
       variables:
           ALLURE_LAUNCH_NAME: "Postman tests - REGRESSION TESTING"       # Необязательно задавать, если задали в переменных выше
   extends:
   - .deploy
   stage: allure
   allow_failure: true

If there is a microservice architecture and there are separate collections of checks for services, then it is possible to launch separate checks and a separate set of checks for smoke or regression. To do this, you need to add another stage and slightly rewrite the stage for allure:

service_test:
   rules:
     - if: $CI_PIPELINE_SOURCE == "pipeline" # Согласно данному правилу, этот процесс запускается в случае, если пайплайн триггериться
     - !reference [.not_gitrun_rules, rules]
   stage: test
   image: node:16
   script:
       - npm install -g newman
       - newman --version
       - npm install -g newman-reporter-allure
       - newman run --verbose "Postman_Collections/${SERVICE_NAME}_Service_API.postman_collection.json"  -e "Postman_Collections/env/${ENV}.postman_environment.json" -g "Postman_Collections/env/workspace.postman_globals.json" --reporters allure,cli --reporter-allure-export allure-results
   allow_failure: true
   artifacts:
       when: always
       paths:
           -  allure-results
   after_script:
       - echo ${ENV} 


allure_test:
   rules:
     - if: $CI_PIPELINE_SOURCE == "pipeline"
       when: always
     - when: always
       variables:
           ALLURE_LAUNCH_NAME: "Postman tests - REGRESSION TESTING"      
   extends:
   - .deploy
   stage: allure
   allow_failure: true 

Also add to the .gitlab-ci.yml service the rules for launching a trigger when it changes:

stages:
- test


Test Stage:
 stage: test
 when: always
 variables:
   SERVICE_NAME: Какой то сервис
   ENV: stage
 trigger:
   project: qa/postman_workshop
   branch: master

Setting up postman tests to run in CI, with results uploaded to AllureTestops

Setting up variables.

1.1. Create a token in Allure Testops:

1. In Allure Testops, click on your avatar and go to your profile;

2. In the API Tokens section, click Create;

3. Enter a name for the token (e.g. “Token for GitLab”), then click “Submit”;

4. In the dialog box that appears, click the Copy icon to copy the token to your clipboard. You will need this token in the next step.

1.2. Specify the token in GitLab:

1. In GitLab, open the project and go to Settings → CI/CD;

2. In the Variables section, click Add Variable;

3. In the dialog that appears, fill in the fields:

   • The key is “ALLURE_TOKEN”.

   • The value is the API token you received in step 1.1.

   • Mask variable – must be checked.

Setting up running postman tests from AllureTestops

2.1 Enable integration with GitLab server in Allure Testops:

1. Login to Allure Testops using an administrator account;

2. Click on your avatar and go to Administration → Integrations;

3. Click Add Integration in the top right corner of the page;

4. In the dialog that appears, select GitLab;

5. Fill in the fields:

*Name – a name that will help you identify the GitLab server, for example “My company's GitLab”.

*Endpoint is the URL of the GitLab server, for example “https://gitlab.example.com/“

6. If your GitLab server uses a self-signed SSL certificate, select the Disable certificate verification checkbox;

7. Click Add Integration.

2.2. Create a token in GitLab:

1. In GitLab, click on your avatar and go to Settings;

2. In the menu on the left, click “Access Tokens”;

3. If the token creation form does not appear, click “Add new token” above the list of tokens;

4. Fill in the fields:

* Token name – a name that will help you recognize the token, e.g. “Token for Allure Testops”.

* Validity period – any date within a year from the current date. After this date, the integration will no longer work and you will need to create a new token to continue using the integration.

* Select scopes – “api” and “read_api”.

* Role of Maintainer

5. Click Create Personal Access Token. The page will reload and the “Your new personal access token” field will appear at the top. Click the Copy icon next to it. You will need this token in the next step.

2.3. Add the token to the Allure Testops project:

1. In Allure Testops, go to the project page;

2. In the menu on the left, click Settings → Integrations;

3. In the Available Integrations section, find the GitLab integration and click Add Integration next to it;

4. In the dialog that appears, specify the Token that you received in step 2.2. Check that the credentials are correct;

5. Click Test connection. After a few seconds, the message “Connection established” should appear;

6. Click Add Integration to close the dialog box and save your settings.

2.4. Set up a job in Allure Testops:

1. In Allure Testops, go to the project page;

2. In the menu on the left, click “Jobs”. The page should contain a new job that was automatically added and configured when running tests in Gitlab;

3. Click the three-dot task menu, then select “Configure”;

4. In the dialog that appears, edit the fields:

*Name – a name that will help you recognize the job.

* Build Server is the name of the integration you added in step 2.1.

* The task can be used to run tests – If this checkbox is checked, users will be able to run this task from Allure Testops.

* Parameters – parameters that can be passed to GitLab via environment variables

/* Проверка на статус-код 200 */
pm.test("Status code is 200", function () {
   pm.response.to.have.status(200);
});


pm.test("Response time is less than 900ms", function () {
   pm.expect(pm.response.responseTime).to.be.below(900);
});




/* Валидация по json-схеме tv4
(подойдет если необходима компактность и выполняется работа с небольшими схемами и данными) */
var schema = {
   "$schema": "https://json-schema.org/draft/2019-09/schema",
   // body of schema
}
let jsonData = pm.response.json();
let result = tv4.validateMultiple(jsonData, schema);
if (result.valid) {
   pm.test('JSON is valid', function () {
       pm.expect(result.valid).to.be.true;
   });
}
else {
   pm.test('JSON is invalid', function () {
       pm.expect(result.valid).to.be.true;
   });
   console.error('JSON is invalid');
   console.log(`In JSON on '${result.errors[0].dataPath.slice(1)}' ${result.errors[0].message}`);
   console.log(`In JsonSchema on '${result.errors[0].schemaPath}' may be error`);
};




/*  Валидация по json-схеме ajv
(подойдет если важна актуальность валидатора, производительность и поддержка новых стандартов JSON Schema */
let Ajv = require('ajv');
var schema = {
   // body of schema
}
let jsonData = pm.response.json();
let ajv = new Ajv();
let validate = ajv.compile(schema);
let isValid = validate(jsonData);


if (isValid) {
   pm.test('JSON is valid', function () {
       pm.expect(isValid).to.be.true;
   });
} else {
   pm.test('JSON is invalid', function () {
       pm.expect(isValid).to.be.true;
   });
   validate.errors.forEach(error => {
       console.log(`In JSON on '${error.dataPath}' ${error.message}`);
       console.log(`In JsonSchema on '${error.schemaPath}' may be error`);
   });
};




/* Проверка на соответствие значения Тела Ответа конкретному ожидаемому значению */
pm.test("Body contains 'ok'", function () {
   pm.expect(pm.response.text()).to.include('ok');
});


/* Проверка на соответствие значения ключа в Теле Ответа регулярному выражению */
pm.test("RegExp match", function () {
   pm.expect(jsonData.headers['x-amzn-trace-id'])
   .to.match(/^[a-zA-Z0-9\s!@#$%^&*()-_=+[\]{}|;:'",.<>/?]{40}$/);
});


/* Проверка на соответствие значения ключа в Теле Ответа конкретному значению */
pm.test("Field includes", function () {
   pm.expect(jsonData.args['foo2']).includes("bar2");
});


/* Проверка на соответствие значения в Заголовке Ответа конкретному значению */
pm.test("Connection is present in headers", function () {
   pm.expect(pm.response.headers.get('Connection')).to.include('keep-alive');
});


/* Проверка на соответствие значения в Заголовке Ответа регулярному выражению */
pm.test("RegExp match", function () {
   pm.expect(pm.response.headers.get('ETag')).to.match(/^[a-zA-Z0-9\s!@#$%^&*()-_=+[\]{}|;:'",.<>/?]{35}$/);
});




/* Пропуск теста */
pm.test.skip("Пропуск теста по причине");




/* Таймер для вычисления времени прохождения теста */
let start = Date.now();
// Здесь вставить тест
let finish = Date.now();
console.log(`Время прохождения теста: ${finish - start}`);




/* Получение значения конкретного поля из тела запроса */
// Разбираем тело запроса как JSON
const requestBody = JSON.parse(pm.request.body.raw);
// Получаем значение "value" из первого элемента массива characteristics
const value = requestBody.characteristics[0].values[0].value;
// Выводим значение в консоль Postman
console.log(value);

At the same time, such minimal tests as response code and response time can be moved to the folder level, and all runs of methods in the collection folder inherit these tests.

Results:

• An easier and faster way to automate back-end testing for manual testers

• It is possible to simply run tests locally in Postman to get a quick answer about the state of the backing

• One-time setup of integration with Gitlab CI and AllureTestops allows you to automate the launch of different sets of tests and the collection of historical information on past and current sprints on different environments.