Speed ​​up the CI / CD pipeline with Kubernetes in Docker (KinD)

In our new translated article, we understand KinD with a practical example.

Cluster creation Kubernetes it gets easier over time. There are several turnkey solutions available on the market, and now no one chooses the difficult path!

It is worth noting that Minikube was one of the main clusters that developers used to rapidly develop and test containers. While Minikube currently supports multisite clustering on an experimental basis, it is not yet available in the General Access (GA).

Consequently, this limits the ability to integrate and test components, which is why most organizations use Kubernetes managed cloud services for this.
To integrate Kubernetes into the CI / CD pipeline (continuous integration and deployment) and perform testing, you need the following tools: Terraform, depending on the cloud provider, and of course, a CI / CD tool like Jenkins, GitLab, or GitHub.

For large companies with sufficient budgets, these are good options, but developers are often looking for something to help them get started quickly. Deploying a Kubernetes cluster in the cloud also takes some time (~ 10 minutes), which can be an obstacle for CIs where assemblies need to be fired quickly.
Kubernetes in Docker or KinD Is an implementation of the Docker-in-Docker (DinD) approach for Kubernetes. This tool creates containers that act as Kubernetes hosts and you only need to install Docker on your machine.

It allows you to deploy a multi-node cluster in a couple of minutes without depending on other tools or cloud providers. This makes it useful not only for local development, but also for CI / CD.

KinD architecture

Kubernetes in Docker uses the Docker-in-Docker (DinD) approach to run a Kubernetes cluster. It runs several Docker containers that function as Kubernetes hosts. Docker containers mount volume docker.sock in Docker running on your machine to interact with the underlying container runtime.

KinD has passed the compliance test and received the CNCF certificate. It uses Kubeadm to bootstrap the cluster and also generates Kube configuration files for the user through whom you manage your cluster, allowing you to use kubectl to interact with clusters. Other Kubernetes components such as Helm and Istio also work great on KinD clusters.

The disadvantage of KinD is that it doesn’t work with services Loadbalancerso you will have to use NodePortto forward your services from the outside.

Also, DinD is currently not the most secure solution, so use KinD clusters only on local development machines and CI / CD pipelines. Never use KinD in a production environment!

Installing KinD

KinD consists of a simple command line utility that you can download and move to your path. Then you can interact with KinD using the commands kind:

sudo curl -sL https://kind.sigs.k8s.io/dl/v0.9.0/kind-linux-amd64 -o /usr/local/bin/kind
sudo chmod +x /usr/local/bin//kind

Then you can create your cluster using the following command:

kind create cluster --wait 10m

This command will create a single node cluster. But if you want to define a multi-node cluster, you can use a cluster configuration file similar to the one below:

# three node (two workers) cluster config
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker

Then create a cluster with a configuration file using the following command:

kind create cluster --wait 10m --config kind-config.yaml

You can also create clusters with multiple management levels by specifying multiple roles in the nodes section.

Since KinD automatically creates a Kube config file, you can use the commands kubectlas with other clusters.

Removing a KinD cluster is also easy. Run the following command:

kind delete cluster

Getting Started

Without further ado, let’s figure out in practice how the CI / CD pipeline uses KinD. We’ll take GitHub Actions as our CI / CD tool because it’s easy to use, doesn’t require additional infrastructure, and can be run by anyone with a laptop and an internet connection.

Let’s create a simple application NGINX with the words “Hello World”.

We perform the following actions:

1. Create a dev version of the application.

2. We start testing the components in the KinD cluster.

3. If the test is successful, we translate the image into release and push it to Docker Hub.

The necessary conditions

  • GitHub account
  • Docker Hub account

Quick Start Guide

1. Fork this repository.

2. Go to the repository and create two secrets: DOCKER_USER and DOCKER_PW… They should contain your Docker Hub username and account password, respectively.

3. Go to GitHub Actions and rerun the tasks. Alternatively, you can make changes to the file README.md and click on it to start the action.

Long version

Let’s take a look at the file build-pipeline.yml on GitHub Actions to understand how it works:

name: Docker Image CI

on: [push]
     # Environment variables available to all jobs and steps in this workflow
env: # Or as an environment variable
      docker_username: ${{ secrets.DOCKER_USER }}
      docker_password: ${{ secrets.DOCKER_PW }}

jobs:

  build-docker-image:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
      with:
        fetch-depth: 0
    - name: Build the Docker image
      run: docker build -t $docker_username/nginx:dev .
    - name: Login to Docker
      run: echo "$docker_password" | docker login -u "$docker_username" --password-stdin
    - name: Push the docker image
      run: docker push $docker_username/nginx:dev

  kubernetes-component-test:
    runs-on: ubuntu-latest
    needs: build-docker-image
    steps:
    - uses: actions/checkout@v2
      with:
        fetch-depth: 0
    - name: Run KIND Test
      run: sudo sh build-test.sh $docker_username
  
  promote-and-push-docker-image:
    runs-on: ubuntu-latest
    needs: kubernetes-component-test
    steps:
    - uses: actions/checkout@v2
      with:
        fetch-depth: 0
    - name: Pull the Docker image
      run: docker pull $docker_username/nginx:dev
    - name: Tag the Docker image
      run: docker tag $docker_username/nginx:dev $docker_username/nginx:release
    - name: Login to Docker
      run: echo "$docker_password" | docker login -u "$docker_username" --password-stdin
    - name: Push the docker image
      run: docker push $docker_username/nginx:release

The assembly pipeline runs three tasks in sequence:

1. Task build-docker-image Builds a Docker image for development and pushes it to Docker Hub upon successful build. In this task, you can run your unit testing.

2. Task kubernetes-component-test sets up a KinD cluster and runs a component test for the application.

3. Task promote-and-push-docker-image pulls an image for development, marks it to release, and sends the release to Docker Hub.

Let’s take a look at the Dockerfile to understand what it creates:

FROM nginx
RUN echo 'Hello World' > /usr/share/nginx/html/index.html

The second step is key, it runs the script build-test.sh… Now let’s take a look at the script:

#! /bin/bash
docker_username=$1
set -xe
curl -sL https://kind.sigs.k8s.io/dl/v0.9.0/kind-linux-amd64 -o /usr/local/bin/kind
chmod 755 /usr/local/bin//kind
curl -sL https://storage.googleapis.com/kubernetes-release/release/v1.17.4/bin/linux/amd64/kubectl -o
chmod 755 /usr/local/bin//kubectl
curl -LO https://get.helm.sh/helm-v3.1.2-linux-amd64.tar.gz
tar -xzf helm-v3.1.2-linux-amd64.tar.gz
mv linux-amd64/helm /usr/local/bin/
rm -rf helm-v3.1.2-linux-amd64.tar.gz
kind version
kubectl version --client=true
helm version
kind create cluster --wait 10m --config kind-config.yaml
kubectl get nodes
docker build -t $docker_username/nginx:dev .
kind load docker-image $docker_username/nginx:dev
kubectl apply -f nginx-deployment.yaml
kubectl apply -f nginx-service.yaml
NODE_IP=$(kubectl get node -o wide|tail -1|awk {'print $6'})
NODE_PORT=$(kubectl get svc nginx-service -o go-template="{{range.spec.ports}}{{if .nodePort}}{{.node
sleep 60
SUCCESS=$(curl $NODE_IP:$NODE_PORT)
if [[ "${SUCCESS}" != "Hello World" ]];
then
 kind -q delete cluster
exit 1;
else
 kind -q delete cluster
echo "Component test succesful"
fi

What the script does:

1.Downloads and installs the utility kind, kubectl and helm to the CI server.
2.Creates a multi-node cluster using a file kind-config.yaml
3. Builds a Docker image for development using docker build.

4. Uploads a Docker image to a KinD cluster. The download provides access to the image for all KinD nodes so that they don’t have to pull the image from Docker Hub.
5.Unpads the container in deployment and throws it through the service NodePort NodePortservice.
6. Get the IP address and port of the host and run a test to see if the application returns the phrase “Hello World”.
7.If the test succeeds, it removes the KinD cluster, outputs “Component test successful” and returns a success code. If the check fails, it deletes the KinD cluster and returns an error code.

results

When we start working with the pipeline, GitHub Actions automatically starts the entire pipeline.

This is undoubtedly an improvement and a convenient way to perform continuous integration and deployment using Docker and Kubernetes. Kubernetes in Docker not only simplifies local development, but is also a great tool for CI / CD.

Thanks for reading the article! I hope you enjoyed it!

Similar Posts

Leave a Reply