The horizontal structure of the development team, or How to bring ML projects to production
introductory: Let’s talk about a team with a horizontal structure. There is no lead in such a team, and the voices of all developers have the same weight in making decisions.
Who is there?
An important point to pay attention to is responsibility. On any project, questions will arise: “who decides on the direction of the team’s work?”, “how does the team make a decision by voting?” and others.
For example, someone wants to store models locally, while someone is closer to MLflow. Of course, there are situations when a decision is made by voting on the basis of rational criteria, but in a situation where the votes are evenly distributed, a better study of the issue is required. Then it is worth discussing the issue in several approaches, and sometimes it is better to revise the problem.
Another point about responsibility is the definition of metrics and project deadlines. We have people in the company who determine the current project of the team, but the metrics and deadlines are determined as a weighted average between the desire of the business and the capabilities of the team. A team with a horizontal structure may not have a person who could communicate the expectations of the project to the business and at the same time be responsible for the result of the team’s work. This can lead to the fact that the deadlines requested by the team for the project are reduced, the methods for calculating metrics are not always fixed.
To solve this issue, we introduced the design of ML projects at different stages of approval. This approach allowed the customer to synchronize his understanding, desire and confidence in the project, as well as to determine clear criteria for the work of the team, while remaining in a horizontal structure.
How to do everything?
When the direction is new and interesting, a sharp increase in the number of projects is almost inevitable. At a certain moment, and specifically after the first successful project, the excitement around the team’s activities can be compared to the opening of the first McDonald’s in Moscow.
The lack of established development practices and ready-made infrastructure can lead to a situation where existing projects take longer to maintain than new ones. Proactive and very important decisions on the part of our team were:
rollout of the MLOps platform, which reduced the time to maintain current projects and bring new ones into production;
development of general rules for creating a code base.
Someone can rightly say that it would be nice to have some kind of funnel for cutting off projects from the “Wishlist” category and prioritizing really profitable projects for the company. We discuss such issues with business, come to a decision how to do the project and whether it is worth doing it at all in terms of profit. The solidarity of the team around a common solution and the conviction in the position regarding the project allow the team to defend their ideas and convey values for the business.
We analyze tasks
When the project is finally approved, the team starts its implementation. Tasks are created and distributed by the team based on the goals and deadlines of the project. Here the question may arise: “How to distribute tasks when the weight of the vote is the same for everyone, and the votes are distributed evenly?”. Now let’s find out.
Since there is no lead in the team, tasks are distributed according to the following scheme:
Everyone chooses tasks that are interesting to him in the context of business value and the vector of his own development.
After the previous stage, there may be some tasks that, according to the team, have low business value, so they say “I”ll be back” and go to the backlog to return after decomposition.
To speed up the process, it is worth highlighting the areas of competence of developers and agree within the team on areas of responsibility. This approach will improve the distribution of tasks and balance the atmosphere in the team.
Human
After all tasks are distributed according to the scheme above, important qualities of the developer’s personality interfere in the process of writing code – the ability to self-motivate, the ability to organize one’s own work and prioritize tasks.
Timely motivation from the team + some help in solving problems greatly accelerates the movement of the task to the target column “DONE”. SCRUM masters who work with several teams in our company help to maintain motivation and organize the planning process.
TL;DR
Features of the horizontal team structure:
Responsibility for the timing and metrics of the project falls on the whole team, and not on an individual, as well as prioritizing projects when they become +100,500, as well as tracking progress on tasks.
The distribution of tasks takes place taking into account the value for the business and the development vector of the developer.
To form the position of the team on the project, a deep study of the issue is carried out, taking into account the opinion of each developer.
The horizontal structure allows developers to develop the skills of self-motivation, self-organization and task prioritization.
There is an opportunity to experiment with technologies and tools that allow you to create value for your business.
Rapid growth of competencies is possible due to the effective exchange of experience and knowledge.
The team rallies around joint decisions.
PS
Having gone through these situations, we successfully transformed the horizontal structure of the team and adapted it to the existing format of the company’s work. This allowed us to create effective business processes that give developers the opportunity to experiment, accelerate the delivery of solutions, create a comfortable atmosphere in the team and help move forward.
PPS
I write about the life of the ML team, development and news from the DS world on my channel.
How do you feel about the horizontal structure, what experience do you have?
