C.I.

In the first article we will provide an overview of the key elements of the platform. In the following articles we will look at each component in detail. We welcome constructive criticism, comments and suggestions that will help eliminate shortcomings and improve the platform.

Docker containerization and Kubernetes cluster

The first most important task was to unify the launch of all platform components and projects that would subsequently be deployed on it. A universal and fashionable solution today is to run applications in Docker containers in a Kubernetes cluster.
Most software developers have created official Docker images for their products and post them on the website hub.docker.com. If there is no official application image, you can always build your own Docker image based on the official one. For example, we collect microservice images based on official lightweight Alpine Linux images.
The Kubernetes cluster for the platform is installed on bare metal or VPS (Bare Metal) with Ubuntu or Debian OS pre-installed, which allows you to build the platform from scratch in vanilla Kubernetes.

Gitea and Continuous Integration CI

At this stage, we installed the Gitea Git server in the cluster, which implements the functions of code hosting and a version control system.
This made it possible to have our own Git server as part of the platform and not to depend on third-party products like GitHub, which also encountered some difficulties in working with it.

The Git server solves 2 main problems:

The code for each microservice is stored in the main branch of the Git repository. The programmer clones the microservice repository to his laptop, creates his own sub-branch, executes the task and makes a pull request to merge changes from his branch into the main branch. A person appears in the team who is responsible for receiving merge requests from programmers and in an organized manner creating merge requests to merge changes from programmers’ branches into the main branch of the microservice being developed.

For each microservice (frontend, backend, mysql, postgresql, redis, memcached, etc.) we created a Git repository in Gitea

The microservice Git repository contains: a Dockerfile with a description of the Docker layers of the microservice image, configuration files of the program in the microservice container, a helm chart for deploying the microservice in the Kubernetes cluster, and a Jenkinsfile with a description of the pipelines for building and delivering the microservice to the Kubernetes cluster. The microservice program code is located in the src directory.

Jenkins and Continuous CD Delivery

To solve this problem, we installed Jenkins in the Kubernetes cluster and created a build and delivery pipeline for each microservice.

To make changes to the code or configuration of a microservice, just make edits to the corresponding files in its Git repository and make a push. Push will automatically force Gitea to send a webhook to Jenkins.
Using a web hook from Gitea, Jenkins will automatically download the contents of the microservice’s Git repository and execute jobs in the Jenkinsfile

Copies the microservice source code from the src directory of the Git repository into a Docker image and compiles it if the microservice is written in a compiled programming language. Builds a Docker image of a microservice using a Dockerfile. Will deploy the microservice to the Kubernetes cluster using the helm chart.

The following Jobs were implemented in Jenkins pipelines:

• review – assembly and delivery of a microservice to the developer’s dynamic environment in the development circuit

• staging – assembly and delivery of a microservice into a staging circuit

• production – promotion of a microservice from a staging circuit to production

• rollback – rollback to one of the previous versions in case of an error

Keycloak and SSO single sign-on

All participants in the project that will be deployed on the platform must be given access to web interfaces: Gitea, Jenkins, Grafana, Phpmyadmin, Pgadmin. Each of them requires authentication and has its own database of users and passwords. As the number of project participants grows, administering a large number of accounts in multiple databases will become a difficult task.
This is where Keycloak and SSO single sign-on technology came to the rescue.
We installed Keycloak into the platform and created a unified user database in it. Added 2 groups: admins and devs. The admins group was associated with the admins role, which gives group members administrative rights in all services. The devs group was associated with the devs role, which gives group members Read Only rights.
On the web interface of each service there is a button “login using Keycloak”.

The user clicks on this button and is directed to the Keycloak authentication form.

The user enters a username and password, and if authentication is successful, Keycloak sends the already authenticated user back to the service from which he came to Keycloak. Having passed authentication in Keycloak, the user gains access to the web interfaces of all platform services, without the need to re-enter the login and password in each of them.

Prometheus, Grafana and platform monitoring

The problem of platform monitoring was solved by installing the Prometheus + Grafana stack in the Kubernetes cluster. We visualized node-exporter metrics in Grafana, giving an idea of ​​the load on the main subsystems of the host on which the platform is installed: CPU, RAM, HDD, network stack, etc.

Configured Alertmanager and sending warnings about critical problems by e-mail.
We connected the collection of business metrics from a project deployed on the platform and visualized it in Grafana on a separate Dashboard.

SSL and secure access to platform services

We installed Cert-manager into the platform, which generated an infrastructure domain wildcard certificate and a CA certificate from the organization’s certification authority, which signed the domain certificate. The domain certificate was connected to the web interfaces of the platform services, and the CA certificate was issued to the project participants. This made it possible to make the platform infrastructure independent of external certification authorities. We connected a Let`s Encrypt certificate to the main domain of the project deployed on the platform and configured its automatic renewal every 3 months.

Zero Trust and network security

Network security on the platform was configured in accordance with the Zero Trust approach – all traffic, both from the Internet to the platform and local traffic in the Kubernetes cluster, was closed. From the Internet, access to the platform was opened to only three ports: 22 – ssh to the host, 443 – HTTPs to web interfaces, and 2222 – work with Git via ssh. In the Kubernetes cluster, instances of infrastructure services, development, staging and production circuits were placed in separate namespaces and isolated from each other at the network level. Allowed traffic within the namespace and the minimum traffic between namespaces necessary for the interaction of infrastructure services.

Conclusion

In the next article, we will look at the intricacies of CI continuous integration in Gitea. We ask those who currently use CI/CD on their projects to write in the comments what functionality should be added to the platform and vote below. Thank you.