Automate review selection with GitLab CI and Danger JS
Hello everybody! My name is Mikhail Avdeev and I work in the Cloud Mail.ru project! I will talk about how I solved the problem of speeding up the verification of merge requests (MR) in our team. Why was this even necessary? Because the developers lazy save energy and usually do not tend to take new MRs for testing, or choose something simpler. So I decided to make a bot that would automatically prioritize and assign a reviewer for each MR.
Idea
At first, we tried to solve the problem with the help of a bot, the idea for which we got from the Calendar team. He had to select open merge requests and send them to the working chat. We wrote a bot in TypeScript and created a chat. But the scheme worked while the team was small, and when it grew and the number of MRs increased, people looked at the list of tasks in the chat and tried to choose the one that was smaller and simpler. As a result, I decided to make a bot for GitLab that would forcefully distribute duties in a team of highly motivated and highly intelligent knowledge workers. At the same time, the bot would save the authors from the creative anguish of choosing verifiers for their code.
The bot also had to be universal so that it could be used not only in our team. To do this, it was necessary to think over its settings and integration with the VK Teams service. The idea was that the bot not only assigns a reviewer for MR, but also sends a message about this to him and the author of the code.
Solution
I already had experience in creating bots, so I chose the technologies I was familiar with: Gitlab CI and Danger-js. How long it can take you to create such a tool, I can’t tell you, it all depends on your CI. I needed to integrate the bot with VK Teams, and you may have another messenger, Slack or Telegram, for example. In the settings, I put the ability to choose the number of reviewers for each MR so that the bot fits different work pipelines.
My bot sends the following messages to the messenger:
And in the GitLab pipeline – these:
The bot also allows you to add some tag to each MR, so that it is convenient to classify changes in the code. In our project, for example, the need-review tag is used. If this is provided for in your project settings, you can select a special tag that allows you to skip checking a specific merge request. Also, the bot should skip launching if the MR is marked as draft or wip.
The bot is integrated into our internal Mail Core CI platform so that it can be used by many other teams in the company, but you can also customize it to fit your environment. How we do it:
# Джоба для вызова dangerjs в пайплайне через нашу обертку (npx ts-node cli danger)
.mail-core:job:danger-init:
stage: test
needs: []
variables:
DANGER_ID: "<<required>>"
DANGERFILE: "<<required>>"
script:
- echo "danger:" $([[ -f ./node_modules/.bin/danger ]] && (./node_modules/.bin/danger --version 2>/dev/null || "[--version not found]") || echo "[file not found]")
- echo "dangerfile:" $DANGERFILE
- time [[ -d cli/command/danger ]] && npx ts-node cli danger ci --dangerfile=$DANGERFILE --id=$DANGER_ID -f
# Джоба для вызова ревью рулетки
.mail-core:job:review-roulette:
extends: .mail-core:job:danger-init
variables:
# Минимальное количество Review Approvers
REVIEW_ROULETTE_MIN_APPROVERS: 1
DANGER_ID: "review-roulette"
DANGERFILE: "./node_modules/@mail-core/ci/dangerfiles/review-roulette/index.js"Bot logic
When a new merge request appears, Gitlab CI automatically launches its runner. The bot looks at the project settings:
const {REVIEW_ROULETTE_LABEL, REVIEW_ROULETTE_MIN_APPROVERS} = process.env;
const {mr, approvals, api, metadata} = danger.gitlab;
const {repoSlug, pullRequestID} = metadata;
const {iid, author, reviewers, web_url, title, description, labels, draft} = mr as MR;
const {suggested_approvers, project_id, approved_by, approvals_required} = approvals as Approvals;
const mrLink = `[${repoSlug}!${pullRequestID}](${web_url})`;
const approvalsCount = approvals_required || Number(REVIEW_ROULETTE_MIN_APPROVERS);And checks if it is possible to skip the check (depending on the assigned tag, the number of reviewers, etc.):
const skipReview = draft || isWip || hasApprove || hasSkipReviewLabel || reviewers.length !== 0;If it is impossible to skip, then the bot takes a list of currently available reviewers:
// алгоритм подбора рекомендуемых апруверов (у нас он реализован на основе файла CODEOWNERS)
const approvers = getSuggestedApprovers();Randomly selects the required number of people:
// алгоритм выбора ревьюеров на ваше усмотрение
const reviewers = getReviewers(approvers, approvalsCount);and sends them messages to messengers (if they are integrated) and to MR itself.
// Достаем из массива ревьеюров только их id (необходимы для отправки в gitlab api)
const reviewerIds = reviewers.map(({id}) => id);
const reviewerNames = reviewers.map(({name}) => name).join(', ');
const reviewerCountText = reviewers.length > 1 ? 'ревьюеров' : 'ревьюера';
// Получаем на основе ревьюеров email
const reviewerEmails = await getReviewerEmails(reviewers, api.Users);
const mentionedUsers = reviewerEmails.map((v) => `@[${v}]`).join(',');
// Добавляем выбранных ревьюеров в MR
await api.MergeRequests.edit(project_id, iid, {reviewer_ids: reviewerIds});
// Информируем автора MR о выбраных ревьюерах в VK Teams
await communicator.sendMessage(
'author',
`
${REVIEW_ICON} Я подобрал для *${mrLink}* (${title}) ${reviewerCountText}: ${mentionedUsers} 👏👏👏
`,
);
// Информируем ревьюеров в VK Teams
await communicator.sendMessage(
reviewerEmails,
`
${REVIEW_ICON} Вы выбраны ревьюером 🎉
*${mrLink}: ${title}*
${description}
`,
);
// Отправляем сообщение о выбранных ревьюера в MR
message(
`Я подобрал Вам ${reviewerCountText} 🎉 \n` +
'\n' +
`Встречайте бурными аплодисментами - ${reviewerNames}! 👏👏👏`,
{icon: REVIEW_ICON},
);And this piece of code is responsible for error handling:
// Получаем ссылку на документацию об ошибке
const docLink = getDocLink('ci', {fragment: '#review-roulette'});
// Информируем автора MR об ошибке В VK Teams
await communicator.sendMessage(
'author',
`
${REVIEW_ICON} Мне не удалось выбрать ревьюеров для ${mrLink} 😔
${errorMessage}
Подробнее можно почитать в [документации](${docLink})
`,
);
// Отправляем ошибку в MR
fail(
`Мне не удалось выбрать ревьюеров: \n` +
`${errorMessage} \n` +
'\n' +
`Подробнее можно почитать в [документации](${docLink})`,
);Local launch for testing
In order not to test the bot in Gitlab CI, you can run it locally and check the settings and correct operation. This is much faster, because otherwise you will have to create a test merge request in the project, wait for it to load, push and hope that the connection does not fall off. And for a local launch, a few commands are enough.
The first step is to set up the local environment:
To do this, set up environment variables.
export DANGER_GITLAB_API_TOKEN=токен
export DANGER_GITLAB_HOST=урл/репозиторияYou will also need a link to the merge request so that the bot can take the necessary information from it.
Local testing is started with the command npx danger pr “ссылка/на/mr”--dangerfile ‘путь/до/файла/с/ботом”. Execution result in the console:
Pipeline integration
The bot is debugged, it’s time to put it into operation. First, let’s generate and add an environment variable DANGER_GITLAB_API_TOKEN to your project.
You can read about token generation herebut about adding to the project – here .
Then we integrate the bot.
Problems and solutions
The first problem was that when several scripts with a bot are executed in a pipeline, the last one running overwrites the results of all previous ones. Therefore, I added an identifier to the bot, passed through an environment variable.
npx ts-node cli danger ci --dangerfile=$DANGERFILE --id=$DANGER_ID -fThe second problem turned out to be more serious. The DangerJs authors claim TypeScript support, but it actually works if you don’t use modules. Otherwise, this is what happens:
Danger: ⅹ Failing the build, there is 1 fail.
## Failures
Danger failed to run `./dangerfiles/review-roulette/index.ts`.
## Markdowns
## Error SyntaxError
Cannot use import statement outside a module
./dangerfiles/review-roulette/index.ts:3
import { Communicator } from 'communicator';
^^^^^^
SyntaxError: Cannot use import statement outside a module
at wrapSafe (internal/modules/cjs/loader.js:1001:16)
at Module._compile (internal/modules/cjs/loader.js:1049:27)
at requireFromString (/Users/mikhail.avdeev/ci/node_modules/require-from-string/index.js:28:4)
at /Users/mikhail.avdeev/ci/node_modules/danger/distribution/runner/runners/inline.js:157:68
at step (/Users/mikhail.avdeev/ci/node_modules/danger/distribution/runner/runners/inline.js:52:23)
at Object.next (/Users/mikhail.avdeev/ci/node_modules/danger/distribution/runner/runners/inline.js:33:53)
at /Users/mikhail.avdeev/ci/node_modules/danger/distribution/runner/runners/inline.js:27:71
at new Promise (<anonymous>)
at __awaiter (/Users/mikhail.avdeev/ci/node_modules/danger/distribution/runner/runners/inline.js:23:12)
at Object.runDangerfileEnvironment (/Users/mikhail.avdeev/ci/node_modules/danger/distribution/runner/runners/inline.js:118:132)
```
### Dangerfile
```
--------------------^
```This is solved by compiling TS to JS files.
Another problem was that the bot cut itself automatically (using regular expressions).
// 1. Оригинальный TS
import {fail, message, warn, danger} from 'danger';
// 2. Преобразование в JS
const dangerjs_1 = require('danger');
// 3. Библиотека вырезает импорт регуляркой
dangerjs_1 = 🎃;
// 4. Ошибки вызова библиотечных функций
dangerjs_1.fail // Error
dangerjs_1.message // Error
Dangerjs_1.warn // ErrorSo that the environment does not “swear”, I registered the type declarations and moved the dangerjs import to the very top of the file:
import type * as dangerjs from 'danger';
declare const danger: typeof dangerjs.danger;
declare const message: typeof dangerjs.message;
declare const fail: typeof dangerjs.fail;
declare const warn: typeof dangerjs.warn;Conclusion
As Salvador Dali said: “He who does not want to imitate anyone will not create anything.”
So go ahead, create your own bots!
