pipeline for a website on AWS S3 Bucket

We translated an article about creating a pipeline for deploying a static website on AWS S3 Bucket using Gitlab CI / CD as an example, in order to quickly understand the basics of the technology and start applying it at work. The article discusses the basic concepts of CI and CD, as well as the stages of the CI/CD pipeline.

From the author

I’ve been fortunate to be part of some professional teams, each with multiple DevOps practices. And I was struck by how the quality of the code, the speed of development, and the positive attitude of the team correlate with the CI / CD pipeline.

In my opinion, the maturity of the pipeline can be a great indicator of the experience of the developer, the quality of the code, and the effectiveness of the entire team.

In many cases that I have seen, the pipelines were built either by a DevOps engineer or a separate DevOps team. And the latest report State of CD 2022 demonstrated that only 22% of developers create pipelines.

My goal is to increase this number: to help developers take responsibility for pipelines, build a continuous delivery process and create quality code.

The article discusses the fundamental concepts of CI and CD.

What is CI/CD?

Many businesses use Agile frameworks because they allow you to change priorities and increase delivery speed. Among other things, this approach improves the atmosphere in the team and helps to increase profits.

If your company is following the Agile path, then adopting the culture, philosophy, and practices of DevOps will be a big advantage.

A buzzword of recent decades, DevOps is now considered a true industry standard. CI/CD is a DevOps practice that helps software developers deliver code changes with high frequency and reliability.

“Quick build, quick test, quick fail”

With automated tests in place, teams gravitate toward general task automation and frequent, reliable code deliveries. Creating a CI/CD pipeline in this case can lead to several benefits.

Businesses benefit from lower costs and higher productivity, faster Time to Market and adapt to changing market demands.

The team benefits from rapid feedback, improved development efficiency, reduced bottlenecks, and improved employee engagement and satisfaction.

CI and CD phases

CI stands for continuous integration. Continuous integration allows you to commit changes to the main branch of your codebase many times a day.

Given the limited human cognitive abilities, CI incentivizes developers make small changes to the codewhich are easier to review, cover with automated tests, and release frequently.

This avoids stressful and merge conflict-filled pre-release days with tons of manual testing.

CD – continuous delivery. The next step after CI allows you to ensure that the codebase is constantly ready for deployment, and it can be deployed with the click of a button.

It does not matter what you are working with: a large distributed system, a complex production environment, etc. The key point is automation.

CD – Continuous Deployment. The last stage of a mature CI/CD pipeline, where all code changes are automatically deployed to production without manual intervention.

Of course, this requires a large number of well-designed automated tests. State of CD 2022 States that “47% of developers use CI or CD, but only one in five use both approaches to automate build, test, and deploy code“.

Book Accelerate sums up a multi-year study using State of DevOps reports based on 23,000 company datasets worldwide. As you can see, high-performing teams can deploy on demand (or several times a day).

Stages of the CI/CD pipeline

Source code stage – this is where the pipeline starts. This usually happens after a change in the Git repository, which manifests itself in opening a new Pull Request or pushing to a branch. Another way is to set up the CI/CD toolkit to run the pipeline via an automatic schedule or after another pipeline has started.

Assembly stage – the stage during which the verification and assembly of the code takes place. This is where tools such as Docker: they provide a homogeneous environment.

Testing stage – CI / CD is impossible to imagine without automated tests. After all, everyone wants to be sure that code changes won’t break production.

Deployment stage – at the last stage (after successfully passing all the previous stages), the code can be deployed in the selected environment.

Gitlab example

This example will use Gitlab CI/CD, but the concepts are similar for the rest of the tools, so they can be applied to other repository hosting services.

Exists several CI/CD toolssuch as the world famous Jenkins. This tool requires some setup and configuration, while others are provided by repository hosting services (such as GitHub Actions And Bitbucket Pipelines) with presetting.

Therefore, if your code is hosted on Gitlab, then it is easiest to use Gitlab CI/CDbecause code and CI/CD management are on the same platform.

How can all this work without settings?

To answer this question, it is worth diving a little into the architecture of Gitlab, namely, into instances and runners. Instances store application code and pipeline configurations. Runners act as agents that perform operations in pipelines.

In Gitlab, each instance can be connected to one or more runners.

Gitlab.com is a managed instance with multiple runners that Gitlab itself supports. Therefore, if you use this instance, you get everything you need out of the box.

Let’s get to work

Gitlab offers several templates when creating a new project. The default Gitlab CI/CD pipeline configuration is located in the .gitlab-ci.yml file in the root directory.

Let’s say we want to create a simple pipeline that checks that the code is written, tested, and deployed. Here are a few concepts and terms to familiarize yourself with before you get started.

Pipeline

A pipeline is a set of tasks divided into stages. Gitlab offers various types of pipelines, such as parent-child or multi-project. Full list see here.

Stage

A stage is a step in the pipeline that provides information about which tasks to run (build, test, etc.). One stage may include one or more tasks.

Job

The task is the main block of the pipeline (compilation, linting, etc.). For each job, a name and a script must be defined. After completing all the tasks at the stage, the pipeline proceeds to the next one.

Now – to the code

We build a Gitlab CI / CD pipeline that builds, tests and deploys a static website in AWS S3 bucket.

First let’s create a new .gitlab-ci.yml

1. Define variables

variables: # variabiles definitions for easier reuse of values.
 CI_NODE_IMAGE: "node:16.13.2"

2. Define the stages

# Pipeline stages
stages:
 - install
 - build
 - test
 - deploy

3. Define tasks at each stage

#install job definition
install:
 stage: install
 image: "$CI_NODE_IMAGE" # variable reference
 script: # Shell script that is executed by the runner.
   - npm ci
 cache: # List of files that should be cached between subsequent runs.
   key:
     files:
       - package.json
       - package-lock.json
   paths: # directories to cache
     - node_modules

# Build Job definition
build:
 stage: build
 image: $CI_NODE_IMAGE
 script:
   - npm run build
 artifacts: # list of files and directories that are attached to the job
   paths:
     - dist/
 cache:
   key:
     files:
       - package.json
       - package-lock.json
   paths:
     - node_modules
   policy: pull

# Test Job definition
test:
 stage: test
 image: $CI_NODE_IMAGE
 script:
   - npm run test


# Deploy Job definition
deploy:
 stage: deploy
 image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest # a Docker provided by GitLab that includes the AWS CLI
 script:
   - aws s3 cp --recursive dist s3://bucket-name # copies the dist folder to an aws s3 bucket

That’s all, thanks for your attention.

You can learn how to work with pipelines, builds and artifacts on the Gitlab CI/CD course in Slurm. You will learn what Gitlab consists of and what features and settings it has, as well as analyze the best practices for building a pipeline, features of templating and working with variables.

The video course is always available. View program: https://slurm.club/3JUKdzT

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *