I’ll tell you a little about our (DevOps) processes in the company.
Our DevOps department supports all technological processes of the Development Department.
We also fulfill requests from other departments of the company.
The version control system is Gitlab. Assemblies are assembled in it.
Developers use the .NET stack and write in C, C #.
Nuget repository and Docker Registry are also in Gitlab.
Almost all binaries for both Linux environments and Windows are collected in Kubernetes.
Clusters are built on minimal configurations: 3 masters + n..workers.
Deployed using Ansible on-demand playbooks.
All the infrastructure is in Git. The Iac approach is used.
Everything from configuring DNS servers to installing updates on Linux hosts is managed through CI.
The infrastructure consists of both Linux and Windows environments.
Everything is monitored by Prometheus, Node Exporter, Grafana + additional exporters.
Mattermost is used to communicate with developers.
For video conferencing – BigBlueButton.
Many companies use the same Slack (we also used it before) and this is enough for them.
We switched to self-hosted solutions due to the instability of the slack (like many other online services) back in 19, during the outbreak of the pandemic.
Then there were severe problems in the operation of all public online services.
The platforms simply couldn’t handle such massive influx of user flows.
But even after the loads stabilized, we still remained on self-hosted solutions.
They still work more stable, plus all correspondence and negotiations are carried out within the company.
Also, Mattermost is used to automatically collect metrics and alerts from infrastructure, assemblies, etc.
Links to releases for business are automatically dropped there at the time of release.
Many are now and have used Cloud services now.
We tried to work with them, but quickly realized that the capabilities of the cloud were not enough for our products to work, due to the specifics of the products themselves.
Therefore, we use our capacities.
The plus is that you can fully manage the entire infrastructure, loads, which cannot be fully done when working with Cloud services.
The test infrastructure is located at the facilities of VMware Vsphere in conjunction with Terraform (nodes and disk pools are monitored by the same Prometheus, Grafana).
This makes deploying the booths a lot easier as they are all deployed from Gitlab.
In minutes, the required environment is deployed and there, using the same Gitlab, you can roll the product under test, without the need for manual installation.
As a result, the tester receives a ready-made testbed and, without spending too much time on manual deployment, can do his job.
Autotests are configured on all assemblies. Written by both developers and QA and DevOps teams.
Write as many tests as possible!
This allows you to have a more stable product at the output, reduce the time for finding and eliminating errors.
The previously mentioned Mattermost receives alerts about broken assemblies, and the DevOps engineer already knows where that broke. As a result, the problem is most often eliminated even before the user wants to create a task in Jira or write about the problem in the messenger.
The Hashicorp Vault is used to store secrets and passwords.
Our main use, in CI assemblies, is to avoid storing secrets and passwords in repositories.
Interaction between teams is one of the most important mechanisms in the direction of DevOps in our company.
No matter how powerful the technological stack is and no matter how much the whole process is automated, it will be rather difficult to achieve the desired result without direct interaction between teams.
This interaction is supported by Mattermost and BigBlueButton. This is another plus for self-hosted tools.
Since they are not influenced by external factors, you can fully focus on the workflow.
I would also like to note that when building the infrastructure and its components, you should not over complicate it. Where possible, simple and reliable services should be configured. Do not write huge script-combines, so as not to waste your time and colleagues’ time later looking for the cause of the accident. And if they do exist, especially legacy ones, try to get away from these tools, rewrite, replace. Simply put, improve the technological process.
All processes taking place within the company and supported by our department are very entertaining. Sometimes you have to combine the incompatible and automate the non-automated. But the end result is a fairly stable and interesting product.
In conclusion, I would like to say.
Everything can be automated, but not everything
can need to be automated.
Although my opinion is not that DevOps is only automation.
The use of all DevOps practices and methodologies, coupled with well-established communication within the company, will bring the desired result rather quickly.
If the topic is close to you, please share your experience in the comments. Thank you for your attention.
PS the picture shows a fragment from one very funny movie called “Billy Madison” with Adam Sandler. For me, he tells that there are always many obvious phenomena and things around us, but we can give them
the faithful desired values.