Where to apply machine learning?

Hi all! In the past article I talked about the life cycle of machine learning, when its implementation occurs spontaneously. Now let’s talk about a more conscious approach, which I came to through chaotic insertion and pain.

Below will be instructions on how to find a project for an ML engineer with PM ambitions or for a PM with an understanding of the limitations of the ML sphere. Although for those who “just ask”, there is also something to read.

Where to begin

Let us first understand the general state of affairs about which we will reason. For the birth of an ML project, an initiative is needed. It can go both “from below” and “from above”. From its source and the energy of the initiator globally depends on whether the project will survive to the stage of implementation.

With the initiative “from above”, everything seems relatively simple – there is a big boss who, for his own reasons, moves the project forward. For an ordinary ML engineer or PM, this is a given that we face every day. Our task here is also relatively simple – to help move the project towards success and it is desirable to highlight the pitfalls to all interested in advance.

When it comes to the initiative “from below”, the situation becomes a little more interesting. It is interesting because in this case there is no administrative resource that allows you to remove some barriers to the movement of the project. Instead of an administrative resource, the “social capital” that the initiator has accumulated is used. In essence, these are resources that the initiator can obtain through their personal connections and relationships in the company.

At the start, it would be good to assess whether there is enough social capital to implement the project. If in doubt, it’s still worth a try, after enlisting the help of more “influential” colleagues.

However, one initiative is not enough to start a project; the business value of the project is also needed.

How to identify the business value of an idea?

It can take a lot of resources to find those interested in the project, or maybe the idea will be so cool that you will be asked to do the project “yesterday ago”. When looking for such an idea, I start by researching current business processes and identifying places where machine learning can be useful. Of course, my idea of ​​usefulness is limited by the depth of understanding of the process and the qualifications of an ML engineer, because of this, the first stage of the funnel of ideas is formed as widely as possible. A coffee break helps in the search: colleagues are always ready to share their pain on the project, offer ideas or problems that can be solved using machine learning.

If the company already has implemented projects with ML under the hood, then new places can be identified through competitors or experts in the domain. Often good ideas within a domain can be found at professional conferences or within the community. It is important to understand here that knowledge of the domain sometimes creates “blindness” to innovation, which is why the collection of ideas is better not to be limited to one domain area. For example, if you work for a logistics company, then your ideas may revolve around logistics and there is a risk of losing a good idea from marketing or customer support.

Okay, let’s say we’ve collected a pool of ideas. How to rank them by business value and choose the best one?

In this matter, the introduction of a value metric is indispensable. Of course, not everything can be measured, but we will try.

I came to the conclusion that the metric can be formed through a 5-point scale. If the company has a pyramid of metrics, with its help you can translate many improved indicators into user happiness. The scale used includes:

• improving the efficiency of the business process: automation gives an increase in labor productivity, which saves human resources;

• direct cost reduction or revenue growth: we improved the algorithm for arranging goods on the shelf – we received more revenue;

• product quality improvement: a more abstract metric than money, but also calculable. For example, one can use the Kano model (article) to assess customer satisfaction with the use of the product;

• non-economic benefits: improved customer experience or company reputation;

• alternative solutions: is there an easier way to implement changes without using ML?

Depending on the company’s strategy, each item can be given different weights in the final happiness metric.

Many people forget about assessing the value of already implemented projects. But ML is impossible to forget, projects require attention and updating. Therefore, value assessment should be performed regularly – this allows you to determine whether you need to invest additional resources in development or retrain models. The existence of alternative solutions can also be easily missed, because we are ML engineers with a hammer in our hands, which means that we see nails everywhere.

So, we have a list of projects and even an estimate of the value of each of them. What to do next?

Determination of readiness for the implementation of the project

In theory, business value assessment looks logical – we have defined the criteria, evaluated them in a happiness metric or other important indicator, and we can go to a brighter future!

In practice, the project will be implemented not only by its initiator, but also by colleagues from other departments with their own list of tasks and a set of KPIs. For colleagues, the value of the project may not be what the initiator sees it. To assess the real possibility of launching a project, it is worth looking at these factors:

1. resources: availability of financial, human and technical resources for project implementation;

2. data: the availability of sufficient and high-quality data for training models, the availability of tools for delivering data to models;

3. strategy: does the project meet the strategic goals towards which the company is moving;

4. culture: openness of the company and colleagues to the use of new technologies and the introduction of changes;

5. maturity of affected business processes: there is no point in automating processes that are not yet ready for automation and integration with new tools.

Yes, here you can also take a 5-point scale, I think you already got the idea.

Two items on this list are more difficult to evaluate than others – this is the company’s culture and the maturity of the processes. In my mind, these are two sides of the same coin. Culture generates processes, processes support culture. In companies with a positive and innovative culture, employees are more open to new technologies and more collaborative. Ideally, when the company culture allows you to take the risks of new technologies without consequences for your career. If the company has established horizontal communications and the company is committed to making decisions based on data, then you are in a dream company and at this moment you can rejoice.

But if there is no item in your company, this is not a reason to be upset. After all, it depends only on you what the culture in the company and the atmosphere in the team will be like.

But what about the technical task?

Each project has readiness attributes. Even if the project does not yet exist, some artifacts for it may already appear. They especially like to show roadmaps, terms of reference, or other visualizations of a plan to conquer the world. A small part of these documents live in place with the project and remain relevant at all its stages.

This topic deserves a separate discussion, and I will share a quick solution − Document template for the design of ML systems from Irina Goloshchapova, well-known in our circles, and her colleagues. If your project requires extensive preparation, this template will help to take into account many nuances. In my opinion, in small projects, the template is redundant, but it is better with it than without it.

Kickoff meeting or how to start a project in 15 minutes

We do remember that any initiative gives nothing without real action, right? I saw cases when companies started “empty” projects, but could not promote really cool initiatives. In many ways, the fate of the project was decided at the kick-off meeting, for which, for some reason, the initiators did not prepare. Perhaps because of the conviction that the value of the project is obvious to everyone around. Many things that are obvious to the initiators will not be obvious to their colleagues. That is why it is so important to involve colleagues who may be affected by the project in the future. After all, such preparation is needed not by the initiator of the project, but by colleagues in order to realize that very business value.

The project launch algorithm is as follows:

1. We evaluate the costs of our “social capital”.

2. We identify the business value of the project using the metric that is important to you. You can use the metric suggested in the article.

3. We determine the degree of readiness of the project for implementation and possibly fill in the ML System Design Doc.

4. 15 minute meeting with presentation of proposals based on previous points.

According to this algorithm, I have launched the last 3 projects out of the 3 proposed.

All this is necessary for the project to receive priority and try to bring real business benefits. If you have gone through all the stages, then at the end you will most likely hear the phrase: “We are ready to implement, let’s go!“.

You still have a lot of preparation to do, but globally you have “sold” the idea, congratulations!

In the next article I will tell you how to bring the project to real business benefit, what demons you can face and what to do with zombie projects.

PS More thoughts about the everyday life of an ML engineer in the tg-channel “Notes of a Data Satanist”: https://t.me/my_datascience