Market of BI (Business intelligence) systems in Russia. How are platform ratings drawn?

When choosing a platform, many companies are faced with the question of why this or that platform is better compared to other similar platforms. And depending on the availability of free time and resources, clients take different paths. Someone launches competitive pilots between different platforms and their partners, someone, having decided on the choice of platform, begins to look for a reliable contract executor, someone looks at a neighbor’s project and this is enough for him to buy similar technology for himself. But almost all clients want to see the overall picture of the market and turn to different ratings.

During the reign of the big three analytical platforms (Qlik, Tableau, Power BI) on the Russian market, the annual opinion of the authoritative Gartner was enough for everyone. As a last resort, we also looked at BARC, Aberdeen Group, G2 or IDC. Everywhere, by the way, the picture quite clearly identified the industry leaders and highlighted the strengths and weaknesses of the platforms. At the same time, getting into Gartner was a prestigious, difficult and expensive task. Of all the Russian platforms, there was only Forecast, which went bankrupt in 2017. But he managed to motivate Western vendors to write battle maps. If we return to Gartner, then this is a whole machine for studying technologies on the market, with its own system for evaluating vendors. Clear criteria and weights for each element of the platform, determination of the vector of technology development for several years ahead, deep analysis of the market and, of course, the magic quadrant, where everyone wants to be in the upper right corner. Moreover, this entire assessment was supported by customer feedback, which Gartner collected separately. Of course, everyone tried to provide contacts of the most loyal customers and influence the final rating, but no one canceled the test purchase. The chance of receiving a higher rating increased significantly after purchasing Gartner’s own service packages. We will still consider this model to be a reference model and fair to the market. And so let’s voice the problem: there is a market for domestic BI software, but it is not clear which ratings can be trusted, and which ratings have too much error?

Next, it will be quite difficult not to offend the feelings of the authors of various ratings, but I will try to explain the pros and cons as correctly as possible without unnecessary personalities.

1. Large IT marketing platforms and publications. Something dot ru. There are two options for the appearance of ratings: either collecting reviews according to the same G2 principle (but without the negative ones), or a simplified system of criteria and weights. In the case of reviews, the vendor’s marketing machine works and, over a certain period of time, it collects, as a rule, a positive set of success stories, which must also be taken into account as a plus for the platform. When they try to draw ratings based on criteria, it comes out like a “pirated” copy of Gartner or BARC with obvious cheating of numbers. Because when you read them and compare them with real combat experience among friends, you are amazed at how much progress has been made. I already wrote earlier that the business of such ratings is built on the loyalty of vendors and their participation in site events. But when the market was not stormy, the ratings coincided with the opinions of the Leaders and there were simply no questions. Now I have at least two big questions for the authors: 1) Do you have real independent BI experts on your staff who drew up the ratings? 2) How do you carry out such complex analytical work in such a short time and without deploying platform stands, followed by testing?

2. Separately, I would like to highlight the UNprofessional approach to ratings. There are two cases: 1) non-core publications are trying to play at being BI experts 2) the vendor himself begins to do strange things on his website and compare himself “independently” with competitors. In the second case, I was simply surprised by this approach and the result of the Experts’ work. I will not dwell on this point in detail, and I do not recommend taking such works into account.

3. Professional approach. When an integrator tries to make a platform rating, it often coincides with its vendor partner statuses. The higher the status, the higher the place in the ranking. Lack of partnerships results in lower positions in the platform rankings. The more a company has invested in the development of a vendor’s technology, the higher it will be in the ranking, since this advertising should encourage the client to buy what is in the integrator’s line. At the same time, the rating itself can be done at a very professional level, with a large number of criteria and will consider more than a dozen platforms

4. I will write separately about Gromov’s rating. The first thing I liked was the humor of the Creator, who named the rating after himself. But, to be honest, I was shocked by the professionalism of my colleagues… The guys are really trying to repeat Gartner’s story locally. They look at all the platforms under a magnifying glass, set up stands, test the platform in several stages, give vendors the opportunity to defend the low score and prove the opposite, and issue a real report with a detailed assessment of the platforms. Of course, all this is sold for money. The only thing that is alarming is the presence of a pocket IT company that deals with BI projects, without real Enterprise experience. If we discard the latter, then so far this is the only rating on the market with which I can agree and believe that what is drawn can be trusted. But, this is my subjective opinion! I would like to believe that the grades received are INDEPENDENT of the presence of partner statuses! Because the paid story can burst in one moment.

In conclusion, I would like to share my personal opinion. Ratings are needed, at a minimum, to understand the main players in the market. But I also recommend paying attention to large corporations, see what they buy for themselves. As a rule, large businesses independently test and study platforms. He spends the necessary and sufficient time on this, tests platforms professionally for information security, which all the above players do not have. He conducts not just tests, but full-fledged pilots, which he then watches under load. Yes, large corporations can afford it, because the cost of error is very high. But ratings also help them, at least to determine 2-3 candidates in this race.

Choose wisely!