Into the future with the integration of Jenkins & Oracle APEX services

5 min


In the process of managing large amounts of data and their visualization, we are increasingly confronted with the need to implement a single tool for accessing the management of functionality in the repository. Moreover, this tool should be equally convenient and understandable to all employees of the data management unit: administrators, analysts, developers and testers.

My name is Vitaly, I am responsible in Rostelecom for the direction of test automation and the implementation of DevOps processes. In this article I will talk about an interesting solution to this problem in a large data warehouse of the company Rostelecom.

And that's how we came to this set of tools for solving this problem.

For a start, a little background. Earlier, I wrote that we wrote our own module in Python to automate the process of installing objects in the data warehouse. It is controlled by Jenkins and allows you to start the necessary functionality either manually by a button with input of the required launch parameters, or fully automatically according to a schedule without user intervention. Also in our company has already been implemented a number of applications written using Oracle APEX.

What is Oracle Application Express (Oracle APEX)?

Oracle APEX – in earlier versions of the application had the name HTML DB. Using this tool, using only a browser and having programming experience in languages ​​such as PL / SQL and JavaScript, you can develop fast, scalable, and secure web applications that can be easily deployed on any contour afterwards for development, testing, and subsequent implementation in production. .

To build a data entry form, there is no need to program the interface in a complex way. There are many ready-made templates in the application that you can reuse to develop your own solution. End users also access the application through the browser, thereby eliminating the need to install the application on the computer. And with all the above advantages, the metadata (information about the data used) in our repository lies in the Oracle database, so we could not ignore this tool.


APEX application architecture example

Oracle REST Data Services (ORDS) Is a data service replacing Oracle HTTP server and mod_plsql, based on Java ee. It provides interaction with objects through Resfull

A bit about Jenkins

Many, probably, already know what kind of animal it is, but for those who are just eyeing this tool, I will tell you what it is and how we use it in our repository.

Jenkins is a continuous integration tool, most often used for software development and application deployment at various stages of the development process. In simple terms, Jenkins provides a process automation environment. Due to its flexibility and a large set of plug-ins, it provides an opportunity to integrate into any application or technology and supports work with different version control systems.

In our data management department, it implements CI / CD processes with automated test runs. Jenkins also manages several samopisnymi modules, thanks to which many manual processes are automated.


Sample interface web-client Jenkins, clickable

The problem of misunderstanding and its solution

Having implemented many tasks and automated them, we are faced with a problem: not only DevOps engineers and testing specialists will have to manage these processes, but also people who are not confronted with such tools, but are involved in the development process. The end users were all internal department staff. The Jenkins client interface, in my opinion, is quite simple and convenient, but I look at it through the eyes of DevOps, and not everyone can look at it the same way. Since we had to run a number of tasks on the button by employees, it became necessary to come up with an interface that would be friendlier to the user. In fact, there is a Jenkins plugin called Blue Ocean that allows you to change the UI representation of the tool.


Sample interface web-client Jenkins (Blue Ocean plugin), clickable

This task could not be solved by this plugin, but as an alternative, if the standard interface does not suit you, you can reconfigure it. Most often there is a need to integrate the module in Jenkins, and not vice versa. This is the interesting problem to be solved. At the time of its solution, as I wrote, we already had a number of applications written using Oracle APEX.


An example of the interface of one of the APEX-applications, clickable

After checking that its interface is quite friendly and able to manage it, and all the parameters for running the necessary tasks in Jenkins are stored in the Oracle database, the idea arose to launch a number of Jenkins tasks from APEX.

It took very little time. The interaction between applications was implemented according to the architectural principle of building service-oriented systems such as REST. REST-architecture implies the rules of interaction of components of a distributed application in a network. APEX-application allows you to use this style and provides a ready-made template for forming the process of sending HTTP / HTTPs-requests such as REST when developing an application. As a result, we quickly picked up the application to run this kind of tasks, the launch data began to pull up directly from the database with the possibility of their choice, which eliminated the possibility of input error launch parameters. The transfer of parameters for running tasks to Jenkins is carried out by means of a POST request, in the body of which there is a JSON with the necessary parameters.


REST call form when developing an APEX application, clickable

JSON example:

{"parameter":[{"name":"SERVER","labels":"master"},{"name":"INSTANCE","value":"DEV"},{"name":"SRC_CODE_START","value":"SRC_ID"},{"name":"SRC_CODE_END","value":"SRC_ID"},{"name":"MODEL_ID","value":"MODEL_ID"},{"name":"SAVE_DATA","value":"0"}], "statusCode": "303", "redirectTo": "."}


An example of a ready-made APEX application for invoking Jenkins processes, clickable

The result of pressing the “Run process” button will be the transfer of parameters to the Jenkins job and its subsequent launch. The display of logs of successful and unsuccessful launches has been implemented, which is directly returned from the console of the Jenkins task process. The task itself may contain any automated process.

Eventually

The integration of application data in our department showed excellent results. There is no need to force the people who do not have to work directly with him to understand the Jenkins application toolkit. We were able to delineate and leave the task execution control at the Jenkins level using a custom matrix of rights, so that users have the ability to assemble tasks, but not have the ability to edit them.

Since this experiment was deemed successful, ideas of development appeared in the form of writing your own open source frameworks to simplify interaction with Jenkins and with other applications running on our site. But this is a completely different story, which I will try to tell in the following articles.


0 Comments

Leave a Reply