How to make CI on Github for a modern frontend
Very soon, on November 6 and 18, we will start new streams JavaScript course and
course “Profession Web Developer”, especially for their start, we share with you a useful tutorial on how to set up Github Actions for real front-end projects with a lot of linters and UI testing, as well as workflow notifications in Slack. Details and a repository under the cut.
GitHub Actions makes it easy to automate all of your workflows, now with world-class CI / CDs. Build, test, and deploy your code right from GitHub. Make checks, branch management, and issue organization work the way you want them to.
But that sounds like an introduction from the GitHub welcome page, right? So what we really need here is to show the process of configuring the CI pipeline for the modern web. This is what we will do. Here we will rely on repository with pre-configured GitHub Actions.
Introduction
It’s no secret that the modern interface is based not only on the combination of HTML + CSS + JS, which is stored and given to the browser by the server side. A lot of sophisticated architecture has been built to deliver high quality code to the end user. We must do this day in and day out.
Now, as front-end engineers, we are responsible for storing many preprocessor scripts, pipelines along with the business codebase, linting conventions, testing, security. This allows you to take your mind off quality control and focus on coding.
Note. Continuous deployment (CD) processes are not discussed here because it is another part of the CI / CD ecosystem. Our goal is to figure out how to automate annoying routine tasks when writing code or preparing pull requests.
Let’s say there is no one on the project with a defined DevOps role. It is sad. Sometimes this is fine for business, because even then you can still create some kind of pipelines with ESLint, TSLint, Prettier, Jest, etc.
While it is quite easy to do it locally based on the documentation and JavaScript, it is still quite difficult to figure out how to work with external CI and YML workflows.
This is where we’re going to dig next.
Basic GitHub CI Configuration
God bless GitHub and their passion for templating everything: pull requests, issues and questions, financial support, etc. So in the end the CI / CD setup doesn’t look much more complicated. Here’s what we have after activating the basic NodeJS workflow on GitHub:
# This workflow will do a clean install of node dependencies, build the source code, and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actionsname: Node.js CIon:
push:
branches: [ master ]
pull_request:
branches: [ master ]jobs:
build:
runs-on: ubuntu-latest strategy:
matrix:
node-version: [10.x, 12.x, 14.x]
steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }} - run: npm ci
- run: npm run build — if-present
- run: npm test
It can be seen that there is not much logic here. You can already mark friends npm, ci, build, test
…
Between npm
and yarn
there is no difference. I prefer yarn
, so this particular manager appears in the examples.
Here we must understand that among all fields: name
, on
, jobs
, run-on
, strategy
, steps
, build
only one thing is valuable to us in terms of updating the base template. And this steps
… They provide us with the ability to add, remove, change scripts (actions) throughout the entire workflow. Also, as far as you can see, there is no word on how to automate linting, testing, or other intermediate processes.
Custom CI configuration
Once we figured out how to set up GitHub Actions, let’s move ahead and create our own CI configuration.
The very first problem in understanding how CI works is that it is impossible to believe that it is possible to run the same scripts as on our local machines, for example yarn eslint ./someAwesomeFile.ts
or npm run jest ./someAwesomeFile.test.js
– and so on.
The only major difference is that you need to figure out how to run them on pending commit files. In the end, it works the same way as the pre-commit hook lint-staged
and husky
… But we’ll come back to this at the end.
Creating a Testing Workflow in CI
So let’s dive deeper into CI! Here’s the entire testing workflow:
name: Unit + UI Testing
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
unit-ui-testing:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [12.x]
steps:
- uses: actions/checkout@v2
- name: Staring Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- name: Bootstraping packages
run: yarn install
- name: Testing Shared Utils
if: always()
run: yarn jest ./shared
- name: Testing Storybook UI
if: always()
run: yarn storybook:build
Whereas everything above the Starting Node.js server launch (including the launch itself) seems familiar from the GitHub pattern, we could focus only on the basic test scenarios described below.
GitHub Action CI requires the same pre-configuration as before running the repository locally. For example, here’s the download of the required packages:
- name: Bootstraping packages
run: yarn install
Installing Jest + Enzyme
Ok, now we’re one step away from creating our own test configuration:
- name: Testing Shared Utils
if: always()
run: yarn jest **/*.test.*
Are you wondering what the flag does if: always()
? The answer is simple. If you have several independent steps, but at the same time connected by a common abstraction, they can still work, even if some of them do not work. Conditions if:
together with the internal GitHub API provide a flexible scripting interface.
Core team run: yarn jest **/__test__/*.test.*
will test the entire repository. Our goal is to test not only the files that were in the commit (even if they passed some of their own tests), but to test the files globally, taking into account all the impact on the code. Therefore, we test everything every time. It’s always easy to switch a configuration to check only certain files. You decide!
Setting up UI testing in Storybook
Please note that you can skip this step if you are not using Storybook UI testing at all.
- name: Testing Storybook UI
if: always()
run: yarn storybook:build
We keep this code here to ensure that our common UI codebase is not affected by any new feature in the commit. So basically, getting success information run: yarn storybook:build
we hope the build goes well!
Creating a linting workflow
Let’s move on to linting. This is where we finally roll up our sleeves and are sure to have more fun! It’s time to get familiar with the real CI configuration with Prettier, ESLint, TSLint and StyleLint. So far, so good. As with the testing workflow, let’s start with an example:
name: Lintingon:
push:
branches: [ master ]
pull_request:
branches: [ master ]jobs:
linting:
runs-on: ubuntu-latest strategy:
matrix:
node-version: [12.x] steps:
- uses: actions/checkout@v2
- name: Staring Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }} - name: Bootstraping packages
run: yarn install - name: Get file changes
id: get_file_changes
uses: trilom/file-changes-action@v1.2.4
with:
output: ' ' - name: Echo file changes
id: hello
run: |
echo Added files: ${{ steps.get_file_changes.outputs.files_added }}
echo Changed files: ${{ steps.get_file_changes.outputs.files_modified }}
echo Removed files: ${{ steps.get_file_changes.outputs.files_removed }} - name: Prettier Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn prettier --config ./prettier.config.js --ignore-path ./.prettierignore ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix - name: ESLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn eslint --config ./.eslintrc.js --ignore-path ./.eslintignore ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix - name: TSLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn tslint --config ./tslint.json -e "**/*.(js|jsx|css|scss|html|md|json|yml)" ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix - name: StyleLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn stylelint --config ./.stylelintrc --ignore-path ./.stylelintignore --allow-empty-input ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix - name: Commit changes
if: always()
uses: stefanzweifel/git-auto-commit-action@v4.1.2
with:
commit_message: Apply formatting changes
As usual, omitting all of the above in the section Bootstraping, let’s move on to discussing the rest. For clarity, I will clarify: the main thing is to linting only files prepared for commit. It doesn’t make sense to check the entire repository every time. Otherwise, it would take us quite a long time to check and fix each specific file. This is nonsense.
Configuring file changes
We will collect all the files from the commit for further operations. I think in our case the action is very useful trilom/file-changes-action
which could “pick up” all changed filenames.
- name: Get file changes
id: get_file_changes
uses: trilom/file-changes-action@v1.2.4
with:
output: ' '- name: Echo file changes
id: echo_file_changes
run: |
echo Added files: ${{ steps.get_file_changes.outputs.files_added }}
echo Changed files: ${{ steps.get_file_changes.outputs.files_modified }}
echo Removed files: ${{ steps.get_file_changes.outputs.files_removed }}
In turn, along with the setting echo
of these files in the next step, we could create a string consisting of file paths. And in the end, create the ability to send all files further to eslint and other conveyors. So the boundaries Get file changes
and Echo file changes
Is our strategically important point for all further verification steps.
Configuring Prettier Validation
Everything is correct! We have collected the files, the processing of which is definitely necessary. Now it’s time to linting. And the first step is optimization Prettier:
- name: Prettier Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn prettier --config ./prettier.config.js --ignore-path ./.prettierignore ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
It looks a little scary, right? In addition, upon closer inspection, it turns out that there is only the operator if
and script run
… Not too much to be scared. Let’s start with if:
… This seems a little tricky compared to what we saw inside the testing workflow. There are no surprises here, only individual files need to be tested, and in the testing workflow, testing is done across the entire codebase.
Thus, the condition if:
indicates to start scanning only if there are several new or changed files. In case of deletion, there is definitely nothing to check. By the way, it should run even if some steps fail (just like with Jest or Storybook). As for the team run:
, it is exactly the same as the command you run locally (based on the Prettier CLI schema).
Configuring ESLint Validation
The same as in the previous step. Just a native interface:
- name: ESLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn eslint --config ./.eslintrc.js --ignore-path ./.eslintignore ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
Configuring TSLint Validation
Another similar interface:
- name: TSLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn tslint --config ./tslint.json -e "**/*.(js|jsx|css|scss|html|md|json|yml)" ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
The only difference here is that you cannot use the global file .tslintignore
because there is no such file in TSLint yet. We literally need to put all the exception rules in the CLI arguments.
Configuring StyleLint Validation
Everything looks the same as in the case of ESLint and Prettier. Just a custom interface run:
- name: StyleLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn stylelint --config ./.stylelintrc --ignore-path ./.stylelintignore --allow-empty-input ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
Configuring the Commit changes action
Fuh, we did it! The conveyor consists of 4 independent checks. Great job! Now it’s time to move on. Let’s talk about ways to save processed changes (files). There is a pretty useful package called stefanzweifel/git-auto-commit-action
:
- name: Commit changes
if: always()
uses: stefanzweifel/git-auto-commit-action@v4.1.2
with:
commit_message: Apply formatting changes
It aims to automate the commit of prepared files. Thus, all changes to the conveyor are saved at once. And here it is, the second workflow file. Now you have all the automation and codebase enhancements in your hands.
Bonuses – Yarn caching and Slack notifications
Just in case you want to continue, I would recommend integrating a few more Github Actions.
Yarn caching
As with local development, it takes some time to build and install all the packages in the repository. There is a way to reduce the boot time from 15 to 1 minute.
It is of course possible to cache each installed package on the CI side. To do this trick, simply replace the bootstrap step with an action actions/cache@v2
…
- name: Restoring Yarn cache
uses: actions/cache@v2
with:
path: '**/node_modules'
key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}- name: Bootstraping packages
if: steps.yarn-cache.outputs.cache-hit != 'true'
run: yarn install
Turn on Slack notifications
If you are using Slack as a source when communicating with a team, it is important to have CI / CD process notifications. Thanks to 8398a7/action-slack
you can easily manage which notifications and their texts we need. Just add this step to your workflows right after all of the above.
- name: Slack Notification
uses: 8398a7/action-slack@v3.8.0
if: failure()
with:
status: custom
fields: workflow,job,commit,repo,ref,author,took
custom_payload: |
{
username: 'Awesome-CI',
icon_emoji: ':react:',
author_name: 'Linting Test',
attachments: [{
color: '${{ job.status }}' === 'success' ? 'good' : '${{ job.status }}' === 'failure' ? 'danger' : 'warning',
text: `CI Task: ${process.env.AS_WORKFLOW}ncommit: (${process.env.AS_COMMIT}) ${{ github.event_name }} ${{ job.status }}. Initiated by ${process.env.AS_AUTHOR} in ${process.env.AS_TOOK}`,
}]
}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
MATRIX_CONTEXT: ${{ toJson(matrix) }}
I must say that to be able to work with notifications in Slack, you must create a webhook URL (code shown above) and specify it in Slack settings. This is how all workflows look like:
# This workflow will do a clean install of node dependencies, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
name: Linting
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
linting:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [12.x]
steps:
- uses: actions/checkout@v2
with:
ref: ${{ github.head_ref }}
- name: Staring Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- name: Restoring Yarn cache
uses: actions/cache@v2
with:
path: '**/node_modules'
key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}
- name: Bootstraping packages
if: steps.yarn-cache.outputs.cache-hit != 'true'
run: yarn install
- name: Get file changes
id: get_file_changes
uses: trilom/file-changes-action@v1.2.3
with:
output: ' '
- name: Echo file changes
id: hello
run: |
echo Added files: ${{ steps.get_file_changes.outputs.files_added }}
echo Changed files: ${{ steps.get_file_changes.outputs.files_modified }}
echo Removed files: ${{ steps.get_file_changes.outputs.files_removed }}
- name: Prettier Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn prettier --config ./prettier.config.js --ignore-path ./.prettierignore ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
- name: ESLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn eslint --config ./.eslintrc.js --ignore-path ./.eslintignore ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
- name: TSLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn tslint --config ./tslint.json -e "**/*.(js|jsx|css|scss|html|md|json|yml)" ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
- name: StyleLint Checking
if: ${{ always() && (steps.get_file_changes.outputs.files_added || steps.get_file_changes.outputs.files_modified) }}
run: yarn stylelint --config ./.stylelintrc --ignore-path ./.stylelintignore --allow-empty-input ${{ steps.get_file_changes.outputs.files_added }} ${{ steps.get_file_changes.outputs.files_modified }} --fix
- name: Commit changes
if: always()
uses: stefanzweifel/git-auto-commit-action@v4.1.2
with:
commit_message: Apply formatting changes
# branch: ${{ github.head_ref }}
- name: Slack Notification
uses: 8398a7/action-slack@v3.8.0
if: failure()
with:
status: custom
fields: workflow,job,commit,repo,ref,author,took
custom_payload: |
{
username: 'React-Apps-CI',
icon_emoji: ':react:',
author_name: 'Linting Test',
attachments: [{
color: '${{ job.status }}' === 'success' ? 'good' : '${{ job.status }}' === 'failure' ? 'danger' : 'warning',
text: `CI Task: ${process.env.AS_WORKFLOW}ncommit: (${process.env.AS_COMMIT}) ${{ github.event_name }} ${{ job.status }}. Initiated by ${process.env.AS_AUTHOR} in ${process.env.AS_TOOK}`,
}]
}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
MATRIX_CONTEXT: ${{ toJson(matrix) }}
# This workflow will do a clean install of node dependencies, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
name: Unit + UI Testing
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
unit-ui-testing:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [12.x]
steps:
- uses: actions/checkout@v2
- name: Staring Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
- name: Restoring Yarn cache
uses: actions/cache@v2
with:
path: '**/node_modules'
key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}
- name: Bootstraping packages
if: steps.yarn-cache.outputs.cache-hit != 'true'
run: yarn install
- name: Testing Shared Utils
if: always()
run: yarn jest ./shared
- name: Testing Storybook UI
if: always()
run: yarn storybook:build
- name: Slack Notification
uses: 8398a7/action-slack@v3.8.0
if: failure()
with:
status: custom
fields: workflow,job,commit,repo,ref,author,took
custom_payload: |
{
username: 'React-Apps-CI',
icon_emoji: ':react:',
author_name: 'Unit + UI Integration Test',
attachments: [{
color: '${{ job.status }}' === 'success' ? 'good' : '${{ job.status }}' === 'failure' ? 'danger' : 'warning',
text: `CI Task: ${process.env.AS_WORKFLOW}ncommit: (${process.env.AS_COMMIT}) ${{ github.event_name }} ${{ job.status }}. Initiated by ${process.env.AS_AUTHOR} in ${process.env.AS_TOOK}`,
}]
}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
MATRIX_CONTEXT: ${{ toJson(matrix) }}
Summarizing
Today, we have created CI workflows that can do a lot of mundane tasks without wasting your personal time. As you can see, it’s quite easy to manage linting and testing. Thanks to GitHub CI, you hardly need any third-party software, hooks, Actions analogs in other CIs. Try to use your scripts that only sync with package.json
… It looks like at some point it will be possible to live freely without any manual actions associated with committing.
And in case you are planning to change the field or improve your qualifications – a promotional code HABR will give you an additional 10% to the discount indicated on the banner.
- JavaScript course
- Profession Web developer
- Data Science profession training
- Data Analyst training
- Data Analytics Online Bootcamp
- Java developer profession
- Python for Web Development Course
- SQL for data analysis
- C ++ developer
- Data Analytics Course
- DevOps course
- The profession of iOS developer from scratch
- Profession Android developer from scratch
- Machine Learning Course
- Course “Mathematics and Machine Learning for Data Science”
- Advanced Course “Machine Learning Pro + Deep Learning”
Recommended articles
- How to Become a Data Scientist Without Online Courses
- 450 free courses from the Ivy League
- How to learn Machine Learning 5 days a week for 9 months in a row
- How much data analyst earns: overview of salaries and vacancies in Russia and abroad in 2020
- Machine Learning and Computer Vision in the Mining Industry