History of creation and overview of the program for easy creation of DevOps reports

DevOps Report Manager logo

Hello, my name is Pavel.

I have been working as a system administrator since 2010 in different companies, in 2019 I decided to change my field of activity to a DevOps engineer.

After working as a DevOps engineer for some time, I realized that employees needed a program that would create concise reports in the form of a table about which versions of the program assembly are installed, on which test benches and in what state these assemblies are at the time of the report, taking into account that that they could be emailed to the development team.

Working in a company, I created this version of the report in the form of a job in Jenkins (Pipeline + Bash), the report was created only for certain test benches and for a certain program.

Over time I got the idea to create my own universal cross-platform Java program for easily creating DevOps reports.

To become a DevOps engineer, you needed to know Java. Writing my own program would be some kind of incentive to study it, so I decided to start studying and, if possible, create my own program.

After reading articles on the Internet about licensing when using the JDK and Open JDK, so that there are no copyright problems, I decided to use Open JDK Amazon Corretto 11.

I chose as the GUI interface JavaFX.

The program was called “DevOps Report Manager”.

Then, armed with the book “Java Beginner’s Guide” and the Internet, I began to create the first part of the program.

The first part of the program that reports on the content of the web programming context of programs

To simplify the installation of the program, it was decided not to use databases, but to store the settings in encrypted form in a text file.

I decided to store the activation settings for the full version of the program (without restrictions on the number of servers) in a separate encrypted file for ease of use in a corporate environment.

I used AES 256 as the encryption algorithm.

The minimum password length is set to 4 characters, the maximum is 256 characters.

I decided to store all files of settings, logs and reports of the program in the user’s home directory in the DevOps Report Manager files folder and the corresponding subfolders Settings, Logs, Reports.

I implemented the logs using Logger, I thought that this program does not need to store many files, I made it so that the logs are stored in one file and that it is replaced when it reaches about 7 MB.

Made the file of a certain kind of report to be replaced every time a given report is created.

Also in the program there is some protection from the “fool”, which looks like some restrictions on the number of characters for filling out forms and tables.

A few months later the first part of the program was created.

The program built reports in the form of a nice table in a single HTML file for ease of sending by mail.

The HTML file contains JavaScript for the operation of the interactive report table.

Also in the program there was a version of the report in the EML format (in full and abbreviated form).

In the EML format, the report is less beautiful, because HTML version 1.0 is used, which imposes its own restrictions on page design.

A report is generated if the program has a web program context in HTML or Json format like this { “programName”: “Test_1”, “programVersion”: “1.0.0”, “standIP”: “127.0.0.1” } and if the path to the program context was something like this https://address:port/program name/… path to the program context.

As parameters program name And program version you must use the appropriate parameter names in the program context (not parameter values).

For example, in the context “programName”: “Test_1”,

“program name” is the name of the parameter;

“Test_1” is the value of the parameter.

If you specify the name of a non-existent parameter in the program settings and create a report, the program will write “check settings” in the report.

Below is a screenshot of the first part of the program (the third tab of the program).

Below is an animated gif-screenshot with a version of the report in HTML format (if you click on the text of the program context, its page will open).

Program Context Report (HTML + JavaScript)

Below is an animated gif-screenshot with a version of the full and abbreviated report in EML format (For ease of navigation through the document, I created anchor links).

Program Context Report (EML Full report and EML Short report)

In the process of creating the first part of the program, there were thoughts that two more needed to be done: for reporting Docker And Kubernetes

Starting to think about how to collect data from servers, I came up with an option with a one-time connection to the server via ssh, executing the necessary set of commands, copying all the output from the console to a variable, and then parsing the text from the variable using regular expressions.

Later, in some reports, if there are a lot of containers or pods, we had to make sure that the program could connect to the server twice.

To connect to servers via SSH, I decided to use the open SSH client library JSch.

To connect via SSH, it is possible to use User and Password, use SSH key in PEM format, use SSH key in PEM format and SSH key Password.

For some security of reading the user and ssh key in pem format fields, I made them password-locked.

For reporting, added the ability to execute commands with Sudo rights.

In these parts of the program, the algorithm for outputting to the data table becomes much more complicated, because there are much more containers and pods than one, and each can have its own state.

For Docker containers, it was decided to create the following approximate types of states:

Server connection error;

The server does not have this command, or the user does not have permission to run this command;

All containers are in created state;

All containers are in unknown state;

Multiple containers in created or unknown state;

All containers not found;

All containers are in the stopped state;

Multiple containers in the stopped or paused state;

Multiple containers in stopped or created state;

Several containers are in stopped or unknown state;

All containers are paused;

All containers are up and running;

Multiple containers in a running or paused state;

Multiple containers in a running or stopped state;

Several containers are running or created;

Several containers are running or unknown.

For Kubernetes pods, it was decided to create the following approximate types of states:

Server connection error;

The server does not have this command or the user does not have permission to run this command;

All pods are in state (Unknown);

All pods not found;

All pods are in state (Pending or Waiting or ContainerCreating);

Multiple pods in state (Pending or Waiting or ContainerCreating) or (Unknown);

All pods are in a state (Failed or Terminating or Terminated or CrashLoopBackOff or ImagePullBackOff);

Multiple Pods in a state (Failed or Terminating or Terminated or CrashLoopBackOff or ImagePullBackOff) or (Pending or Waiting or ContainerCreating);

Multiple pods in state (Failed or Terminating or Terminated or CrashLoopBackOff or ImagePullBackOff) or (Unknown);

All pods are in a state (Running or Succeeded or Completed);

Multiple Pods in a state (Running or Succeeded or Completed) or (Pending or Waiting or ContainerCreating);

Multiple pods in state (Running or Succeeded or Completed) or (Unknown);

Multiple Pods in a state (Running or Succeeded or Completed) or (Failed or Terminating or Terminated or CrashLoopBackOff or ImagePullBackOff);

If 3 or more different container or pod states appearthen the overall status is chosen towards “big mistakes”.

Anyway you need to see a detailed technical description in the report table.

The second part of the Docker reporting program

Unfortunately, I didn’t find commands for advanced container reports in Docker (docker inspect is not the same), for example, as for Kubernetes pods, therefore, as additional technical information in Docker reports, the program uses information on container stats and the Docker server .

Below is an animated gif screenshot of interface switching With Docker Image Reports on Docker Compose reports.

The second part of the program consists of two parts:

Docker Report Generation generates a report on the status of Docker containers by Image name and Image tag.

Below is an animated gif-screenshot with a version of the report in HTML format.

Docker report by image name and image tag (HTML + JavaScript)

Below is an animated gif-screenshot with a version of the full and abbreviated report in eml format (For ease of navigation through the document, I created anchor links).

Docker report by image name and image tag (EML Full report and Short report)

Reporting Docker Compose generates a report on the status of a Docker project by Project name.

Below is an animated gif-screenshot with a version of the report in HTML format.

Docker report by project name (HTML + JavaScript)

Below is an animated gif-screenshot with a version of the full and abbreviated report in eml format (For ease of navigation through the document, I created anchor links).

Docker report by project name (EML Full report and Short report)

The third part of the program for creating reports Kubernetes

As additional information on Kubernetes, depending on the type of report, my program contains data such as:

detailed information about the pod;
detailed information about the deployment;
information about services;
information about ingress controllers;
detailed information about cluster nodes.

Below is an animated gif-screenshot of switching the interface from Kubernetes Image reports to Kubernetes Applications reports.

The third part of the program also consists of two parts:

Kubernetes reporting generates a status report of Kubernetes Pods by Image name, Image tag, and Namespace.

Below is an animated gif-screenshot with a version of the report in HTML format.

Kubernetes report by image name (HTML + JavaScript)

Below is an animated gif-screenshot with a version of the full and abbreviated report in eml format (For ease of navigation through the document, I created anchor links).

Kubernetes report by image name (EML Full report and Short report)

Kubernetes Application Reporting generates a status report for Kubernetes pods by Application name, Application version (optional), Part-of parameter (optional), and Namespace.

Below is an animated gif-screenshot with a version of the report in HTML format.

Kubernetes report by application name (HTML + JavaScript)

Below is an animated gif-screenshot with a version of the full and abbreviated report in eml format (For ease of navigation through the document, I created anchor links).

Kubernetes report by application name (EML Full report and Short report)

Backup, export and import of the program settings file

In order to be able to backup the settings file or share the program settings file between employees without transferring their password, I created the ability to reset the password when exporting settings.

To load the file of exported settings, you will need to import it (after the import of the settings file is completed, the program will close).

If an employee wants to change the password in the settings file transferred to him, he will need to reset the password on his computer when exporting the program settings and then import this program settings file.

Overview of the functionality of some program buttons

To view the desired report in the desired format, select the appropriate tab, for example, Docker report, then select the desired report format, for example, EML, and click on the button “open file”.

To reset the settings in the current tab, click the button “Reset” and type in a password.

To reset all settings of all tabs and reset the password, you need to click on the button “Reset all” and dial the password, then dial a new one 2 times.

In order to save changes in the current tab, you need to click on the button Save and type in a password.

In order to save changes to settings on all tabs, you need to click on the button “Save all” and type in a password.

Overview of the functionality of the program close buttons

To check the changes in the program settings and request to save the settings changes before closing the program, click the button “Exit” or on the button “X” in the upper right corner of the window.

The positions of all switch buttons are saved as changes to the program settings. in the tabs of the program, except for the “Block / Unblock” switch button (blocking and unblocking the visibility of the user and ssh key fields in pem format).

Keyboard shortcut Alt+F4 shows a dialog about closing the program without saving changes to the settings.