Business expert and programming. Combine cannot be shared

Now imagine what it is like to implement DAAS (data as a service) if your users only interact with each other using Excel and Power Point. There is a gap: those who have programming skills do not possess the subject area to properly demonstrate all the benefits of modern technology, and business experts are in a state of bewilderment because any known problem can be solved using “Small Data “In the face of Excel.

In order to bridge this gap, and ideally eliminate it altogether, we can approach the problem from which sides. First, recruit analysts with knowledge of SQL and Python. Secondly, to teach existing users programming languages. And it seems like the first way looks more logical and simpler, right? But there are pitfalls, namely:

• “Narrow neck” of a limited resource of analysts;
• there are additional labor costs for the business in a detailed statement of the problem (actually double);
• the task flow will be endless, since any data analysis is an iterative process, when you get the following questions when you get an answer;
• since the responsibility for the calculation is correct on analytics, the business does not have a need to ask itself “for data quality” and a large number of minuses, etc.

Of course, the second approach also has disadvantages – it’s not entirely clear how people loaded with current work can be taught programming skills, it’s by default long, difficult and unclear why, if everything can be done in Excel. Plus, we did not have the existing distinguished function of analysts who would have mastered modern skills in working with tools.

Therefore, we thought and launched the School of Analytics.

Her idea arose as an attempt to answer the following questions:

• as a successful business expert who knows the mathematical equipment well, to teach not only how to work with the code, but also expand the horizons of his imagination: what tasks can you solve with the help of modern tools; now you don’t know something, but what if you knew?
• how to ensure that employees do not immediately forget what they were learning, but apply the acquired skills in practice (for example, automate their routine tasks, freeing up time for new opportunities);
• how to make the implementation of the School of Analytics payback (for example, you can compare the cost of creating a dashboard by a contractor and its endless maintenance and self service or create a prototype tool to clarify whether it is needed and which one).

And in 2019, we started testing the format of the School of Analytics in SIBUR for our employees. We needed to learn to program in Python and SQL people who for the most part solve business problems. It’s not so easy for anyone to get into our School, even if you really want to. The maximum that we can give to everyone who is interested in learning in one way or another – advise the right courses on Coursera or give a stack of useful books. If you want to study, please study.

The school’s team leader was Andrei Telyatnik, who is keen on solving optimization problems and is the winner of one large American competition. Now his optimizer is calculating the electricity market in Russia, in SIBUR Andrey participated in a project to optimize the supply chains of our polymer business.

Only an employee of the company who already has a real business case who is in work right now can study at the School of Analytics. Then the head has a reason to devote part of the time to training. At admission, we ask uncomfortable questions like “Why are you doing this?”, But the main selection is not even at admission, but in the learning process. Those who cannot or are not ready to make sufficient efforts fall off themselves.

In fact, the School of Mentoring was implemented in our School – an employee who has undergone training is committed to helping the next stream learn. And if suddenly he does not cope with the problem of a beginner, you can always call your own former mentor. In this way, a safe environment is formed within which you can always try new things, but you cannot treacherously deviate from the task.

In the learning process, we help students overcome fear of terrible and terrible programming and form proactive thinking in them. The main thing at the very beginning is to overcome the stupor of a clean slate when you need to write something from scratch. Experienced mentors help identify tools and get started, and then help may not even be needed at all, since all the information is on the Internet, you just need to be able to formulate your question. If a student is faced with some kind of problem or is simply stuck in some place, he must first try to figure it out, sort out several options, and if they do not work, call a mentor. He will hold his hand, listen to the problem and the ways in which the student tried to solve it, nods understandingly and tells you how to make it work again.

Thus, people become more confident in themselves and their abilities, form critical thinking in themselves. We are following our graduates, and so far no one has stopped at the scripts that they wrote during their studies, and continue to create new ones as part of more and more new tasks. So far, we have released three streams of students and collected several cases from our graduates – business analysts, and are ready to share them with you.

A bit about the format

All training lasts an average of three to three and a half months, one lesson per week or two. The first two months there was only study, and then, in parallel with it, cases were decided directly. The student agreed with the mentors from the School of Analytics about the meeting, talked about problems, immersed them in the problems and expectations of the result, and mentors shared tools and approaches to solving the problem.

Everything happened in the most free format – meetings, correspondence in the mail and chat rooms, discussions over the phone – without any formal deadlines and frequency. At the same time, students continued to carry out their usual duties in parallel, and the School was an additional burden for them, which they decided to assume.

What was their motivation? Everything is simple – having mastered new tools, business analysts will get rid of them with the help of routine and frequently repeated operations, which, to be honest, are enough in their work. And if, on the one hand, they free up time for solving more creative tasks, on the other hand, by developing tools based on a data lake, you can make marketing data transparent, accessible at all times, and uniform for all users.

What specific tasks did the School of Analytics help graduates to solve? They shared their stories themselves. All of them are specialists of the marketing department.

Specific data processing

Alyona Vartanskaya, chief specialist, business analytics

From the first day of study, the focus of our attention was tuned into three blocks. The first is working with a marketing data lake. We mastered the tools that allowed us to go to the lake and visualize this data on a dashboard. We learned to correctly analyze the quality of the data, look for the missing ones, and in some cases make up for them. Pay attention to the hierarchy, how the data is built and how it is stored, how from different sources to correctly pull data on one specific product.

Secondly, actually, the digital tools themselves – SQL and Python, with their help we learned to write algorithms. And thirdly, basic knowledge of dashboards.

The problem that I came to solve at the School of Analytics was related to clients and customer focus, because a client for business is certainly the most important participant in the relationship. In my case, we tried to learn more about the client, and using stream analysis from different sources, as well as using text analysis to understand how and in which regions he sells his products, to distribute products by brand.

Thus, we will be able to understand in what periods the client may have difficulties in terms of product sales or segment development, and whether we can somehow support him, for example, with package offers. Or vice versa, we will see that everything is fine with the client now, he is developing intensively, opening up new segments, and we can offer him to do some projects together.

And in addition to automating the entire process, I needed to dive deeper into the specifics of the client. Previously, such things were done point by point on request, taking an average of 1-2 hours of work. Now the matter is on the rails – I can always open the dashboard and quickly collect the necessary data: how the client changes his strategy, where, how and when the products are brought, etc. Thus, it takes 10-15 minutes to complete one request.

Requests from colleagues from marketing are most often associated with the analysis of data on a particular product in which they are engaged. Understand the volume of sales, distribution channel or segment of application. If you correctly build a dashboard in the framework of frequently encountered queries, all statistics can be quickly pulled out.

Yes, quite a lot of time is spent on the first dashboards, but over time you do some operations faster. In any case, the entire system is built once, and then you just need to choose the appropriate information, which is also automatically updated.

To complete a request, you usually need to collect a bunch of data from a variety of sources, then you also need to link together a large array of documents, data from different periods, aggregate large excel files.

Now this is a single marketing lake of data, combining internal sources such as SAP and external. You can see the history by month, starting from the 2000s, get a breakdown by product.

Previously, it was necessary to prepare such a summary of data every month and store it somewhere. And locally, only at home. And now everything is transparent, any employee can use the lake if he needs this information.

Another plus of transferring data to the lake is uniformity. Data formats in different databases are different. I learned in the framework of certain groupings to combine these data so that the product immediately pulls up data from different sources for certain keys, so that I can further connect and use them in my work.

Automation of operational reporting

Vitaliy Malakhai, expert, business analytics

Weekly, in the function of Marketing and Sales, an operative is conducted at which the current market situation, main events, trends are discussed. We thoroughly prepare for such meetings and every week we collect materials in the format of an analytical report with detailed information on our products (macro parameters, quotes, prices, comments, etc.).

Previously, all information was collected from a variety of PDF reports and excel files, which were processed manually, and from there it got into the help. Naturally, we spent a lot of time preparing materials, plus all this information remained local. At the School of Analytics, using Python, we automated data collection and made it possible for information from PDF reports and excel files to be stored in a database and then visualized in Tableau.

So we reduced labor costs for the routine collection of information, plus now we can see the whole picture. Unfortunately, we have not yet learned to write comments automatically, but I think it will be an interesting task.

Optimization and reduction of routine operations in data collection and primary processing

Arseny Korshunov, chief expert, advanced analytics

So that a marketer can draw conclusions about what is happening with the market, sometimes it’s not enough to read a report or build diagrams in Excel. You need to make a forecast and understand what will happen if you change one or more input parameters. I am engaged in the construction of mathematical models for the analysis of the real market.

To do this, you need to collect a bunch of input data of different formats from different sources and accumulate them in one place, study the patterns between them and build statistical models. In fact, my task is to build adequate functions of the input variables, which transparently and clearly give the user the result in the form of forecasts, market behavior scenarios.

My task at the School was to automate the collection of certain sets of information, and then upload them to Vertica. The solution itself was divided into three blocks, of which only the central is ready. The first block was to have the system automatically log in to the site, download the archive with 20-30 excel files and save it to disk. The second is to go through these 20-30 files and parse them by data type, creating templates for integration with the database. He is just ready. And the third is to upload data to Vertica.

If earlier for my forecasting models I inserted blocks of information and regularly updated it once a month, now I can log in and download the archive, run Python, and continue to use ready-made templates for uploading to Vertica.
General impressions

This is not to say that the School of Analytics is a series of courses. Rather, it is a platform where from the first day you can use new tools to solve operational problems. At the same time, there are mentors who are always in touch and ready to help.

We discussed our ideas with mentors, and they shared their vision and suggested solutions. This helped a lot, because we usually looked at the problem from the point of view of the business, and mentors from the technical point of view, and thus a new solution could be found that would not have occurred to us in ordinary circumstances.

It was great to learn a lot of life hacks even when working with familiar tools, because we could observe the work of mentors, notice some things in practice and then actively use them in our projects. Now we are actively developing our data lake, and we are spending the free time on automation of more and more new tasks.