How the BI market has changed and why we decided to create our own BI platform

I have been working at Jet Infosystems for about 7 years, most of which I designed and implemented BI solutions and the systems built on them: situational centers, information-analytical systems and everything that was created to collect and analyze data. During this time, I have accumulated a number of stories and observations on the features of BI projects that I would like to talk about.

Story 1

The customer is a large geographically distributed company. The task is to collect information, generate reports, control panels, and also automate a couple of business processes. They planned to collect all this from the decisions of different vendors. That’s how we thought until we came to the installation meeting.

It turned out that it was required of us to implement a full-fledged situational center with a huge amount of custom code and cross about 10 vendor solutions, some of which have already been implemented. And, the most unpleasant thing was to write the kernel of the system from scratch, because it was impossible to use the metadata management system of the selected BI solution. We simply lacked what was available “out of the box”.

The problem was also that rather complex reports were often required. Somehow we managed to cope with the regular functions of vendor solutions. Then it turned out that control panels (dashboards) were also needed, and the size of the entire wall (3×9 video cubes), updated in real time and often dynamic structure (to get out of the video wall or offer the user scrolling, of course, is not permissible).

In this project, we eventually had to implement control panels manually, in pure HTML / JS, without using the vendor’s BI functionality. The decision was forced, certainly poorly supported. By the way, it was those developments that gave impetus to the creation of our own BI-system.

Let’s go back to the video wall: all this beauty was complemented by a mix of BI reports, office documents, VKS, GIS screens, pieces of our custom functionality like an incident response module, task management, shift duty, etc. It seems that they managed and even started to develop a new direction for themselves – information and analytical systems. As it turned out, the classic BI approach is far from always applicable. Just about this will be the following story.

Story 2

Years passed, and finally there was a serious shift in terms of requirements for BI platforms. Customers are fed up with the fact that BI solutions are in essence overwhelmingly autonomous tools, divorced from the general IT infrastructure. Necessary non-monolithic solutions were required. No one wanted to “sit on the needle” of one vendor. Customers demanded freedom and the potential interchangeability of one solution to another.

The market realized that it is physically impossible in one solution to combine all the functions that the customer needs, precisely with those opportunities that the vendor offers. In particular, government agencies, ministries and other departments began to understand this. Something was needed that would allow for an acceptable budget and short deadlines to provide not only visualization and analysis of data, but also automation of business analytics processes.

In particular, this affected the situational centers of senior officials at different levels of government. More and more demands were made on situational centers. The presence of video walls and switches to them has ceased to be a sufficient condition. Serious analytics was required, which was supposed to be convenient, simple, understandable and, most importantly, fast, since decisions in the SC should be made in seconds. Then we came to the conclusion that we need to do our own subsystem. At first, it was not yet a separate product, had no name, but had a clearly defined functionality and architectural principles.

We conducted a number of pilot projects in the constituent entities of the Russian Federation, polishing our platform. We combined BI functions with the functions of process automation, without trying to cover all possible processes of organizations: our goal was to create an information-analytical system, the task of which is not, for example, to automate warehouse accounting, but to give top Management and analysts have tools with which to understand what is happening in the organization, what can be done with it, how to evaluate the correctness of actions and predict what they will lead to. And more about that in the third story.

An example of a dashboard.

So it looks on the screen.

Story 3

The task is the automation of projects for the repair of complex, partly military equipment and the analysis of the progress in the implementation of these projects. The average duration of each such repair is 1 year. It is impossible to use MS Project and the like. Moreover, not only because of import substitution, but also because the amount of customization required by the customer was too large. First of all, this concerned limitations, validations, unique business processes, and the data structure itself.

And again we turned to our platform, adding to it another proprietary development, the project management subsystem. It was written by us a little earlier, from scratch, and was used in pilot projects, but it required a thorough adjustment (fortunately, the customer understood quite clearly what he needed). This subsystem was distinguished by the fact that, thanks to the capabilities of the base platform, it turned out to be surprisingly fast to implement and replicate the entire complex. From the moment the task was formulated to the commissioning, 3 months passed. It is not only about accounting and updating repair plans, but also about reports of various kinds, some of which, by the way, are quite sophisticated.

For example, it was required to create the so-called “Production Chart”. Its purpose is to monitor the workload of the workshops and identify bottlenecks. In appearance, this is a regular BI report, in which all active projects are summarized in a tree and arranged in a single timeline, each of which is a full-fledged expandable Gantt chart. If someone tells me where to find something similar in the “mastodons” of BI platforms or among OpenSource components, I will shake his hand.

We analyzed the needs for visualization, and sadly realized that we needed to write our own visualizer. I think it’s no secret to anyone that standard graphics and bagels are no longer interesting to customers. It is time to recognize that more and more tasks require an individual approach and individual solutions. Of course, the solutions of the vendors allow to increase their platforms, but, firstly, not always in the right amount, and secondly, by what efforts?

How to be

I believe that the time has come for hybrid solutions, which for simple tasks provide clear and convenient user interfaces, and for more complex ones they provide a kind of framework, or framework, with the help of which what is needed is already implemented, and without any restrictions from the vendor. Unless these are architectural restrictions (of course, the importance of the latter is undeniable).

To make such a BI platform is a real art, and I do not claim that we have already made it. This is just what we are striving for. If someone has already done something similar, I will be grateful if you share your experience in the comments on this post.

In the next post I will talk about the architecture of our platform and the technical details of the solution.

Albert Nurutdinov, architect of Jet Infosystems

Similar Posts

Leave a Reply