Hi, my name is Max, I am a product of the Self-Service team in the Tinkoff mobile application. My team has three main goals for creating a service: contactless, proactive and self-service.
This means that we try to make the processes invisible to the user: remove what can be removed, and make invisible what can be made invisible. We solve problems proactively if we can predict the problem, and we simplify the process through technology.
Today I want to talk about how we in the team approach the assessment of tasks and their prioritization. And also about how not to do unnecessary work in a complex product with a lot of dependencies.
If you are responsible for product development, then in the process of studying competitors, qualitative and quantitative analytics, communicating with colleagues or stakeholders, you get ideas, and there are always enough people willing to throw them on.
Let’s imagine that you write everything down in notes or a notebook. For a healthy product, a logical question arises: what tasks should be solved in the first place?
The main goal of a good backlog is to reduce the level of uncertainty and give the user as much value as possible in less time. The main thing here is not to put only numbers in the foreground. Otherwise, in a year you will have nothing but a backlog if the product is large and complex. And after a year, the backlog will be out of date.
I used to use the classic ICE method, but Tinkoff had to adapt it. Now I’ll tell you how.
My Backlog Assessment Points
Idea name – a short name and the essence of what you want to do.
Main hypothesis – how the feature will affect users and business. For example: if we add the “Change number in MB” feature, then we will reduce calls to the operator, the time to the target action and the number of touches. That is, the number of steps required by the user to resolve the issue will be reduced: mobile application – bot – first line – back office, etc. Formats for describing the hypothesis are freely available, I will not list them. The main thing is that you and the team understand what you are doing and why.
Expected Result – what should be the result. The description should contain the criteria for accepting the task, and the result should be specific. For example, “after the implementation of the feature, the user can change the contact phone number without contacting the bank.”
MLP / MVP variant. MVP – is already a familiar concept, but another one has recently appeared – MLP: minimal loveable product. I’m sure you’ve used MLP before, you just didn’t call it that. It is important to think about how we will test our hypothesis and get the first insights before spending resources on a targeted solution. It is worth using MLP if the final decision does not depend on the results of the A / B test, it is possible to separately implement the script in MLP and you can already put the main script on production without losing quality.
Criteria for a good backlog
I use the ICE method to prioritize. At ICE, we evaluate ideas on three dimensions: impact, effort, and confidence.
Impact shows how the idea will positively affect the key indicator that we want to improve.
Effort – an estimate of how much effort and resources is required to implement this idea.
Confidence demonstrates how confident we are in the impact assessments and ease of implementation.
More about impact
To determine the impact on the metrics, we are compiling a reference book. I like the approach in which 1% of the value of the main product metrics is taken as a unit of impact. For example, your main product metric is CAC and it equals RUB 1,500. Impact in this case will be 15 (1% of 1500).
It’s good if product metrics will correspond three main criteria:
Show the value of the product to the consumer.
Intersect with company goals.
It is understandable to influence money.
In my case, the main product metric is the proportion of active customers of a mobile application who do not contact support.
The metric is high-level and insensitive. It is not suitable for compiling a reference book, evaluating team performance, and tracking experiment results. Therefore, I decompose it into a more mundane one: the proportion of active customers of a mobile application who do not contact support on a specific topic. For impact 1, I take 10,000. Top hits are not volatile throughout the year in relative terms. Values 1-10 can mean anything – the main thing is that the values are consistent with each other.
When evaluating a hypothesis, I choose the appropriate value. For example, the “number change” feature in a mobile application will reduce calls to the operator on the “data change” topic by N per month. Substitute the appropriate value for impact.
In this case, it is easy to determine this number. We are aware of user behavior. How many people write to the chat to change the number; how many go to the profile screen before contacting; what proportion of those who have applied have a mobile application, etc. But this is not always the case. If you want to dive deeper into how to calculate potential impact, I recommend the book How to Measure Anything by Douglas Hubbard.
Later, each feature goes through an A / B test, where the main metrics for assessing effectiveness are the number / share of hits on the topic, the time to the target action and the number of touches.
I’ll explain more about how to choose the main product metrics in the next article.
More about confidence
We compose a reference for the type above. You can conduct a survey and understand how colleagues trust this or that fact. Logic also works well here. If you need an example of how you can logically define confidence for a hypothesis, ask in the comments.
To make your logic understandable to others and not give rise to a lot of controversy, you can write directory.
More about effort
We get together with those who will participate in the development of the feature, and write the expected development time in hours / days / weeks. The main thing is to use one unit of measurement. The more information you have on the problem by this time, the more accurate the estimate will be.
With ICE, everything is further only a link to a tracker with a task: with a detailed description of the idea, design, link to research, etc.
What is the catch of the above method in the realities of our team? A good product is always one hundred and one compromises and the ability to manage them.
The product of our team is service. Service runs through all business lines, products and processes. It can take a long time to evaluate and build a backlog.
For example, to assess effort, it is desirable to have a detailed description of the task and design, and this is at least a resource of the product, analyst, designer, colleagues from the back-end and other business lines. Their time can be wasted, because it may turn out that the task is not feasible in the next six months or a year. Design and analytics will become obsolete – work has been done in vain.
Because of this, my hypotheses go through some kind of prescoring before getting into the backlog. His task is to quickly find constraints, try to understand where to solve the problem, and send the implemented tasks to the backlog for evaluation, and not the thoughts of the “astrologer.”
Not every team needs all the points below, but you can definitely take some into service. Martin Kogan’s Inspired: Everything a Product Manager Needs to Know contains some of the limitations. Your task is to understand as early as possible what can prevent you from completing your task, write out these reasons-criteria and speed up the feature according to them.
I’ll tell you what my prescoring includes.
Business restrictions. Having fixed something in one place, you can break it in another and end up making it worse than it was. Not all features need to be done. Come to our team to find out how a feature can affect the customer experience: negativity, conversions, reviews, etc.
Technical limitations. You need to make sure that the feature is realizable in the next 3-6 months. When choosing a period, we proceed from the speed of movement of your team. This is necessary in order to correctly prioritize and do what is not blocked by other systems.
Design constraints. It is important that the feature is not duplicated with another team and there will be no redesign on the screen where changes are planned.
Finding stakeholders aka business opportunity. Our tasks have a lot of back-end dependencies, they affect more than 10 commands. These systems have many customers and their own priorities. To get a task into their backlog with the right priority, you need to evaluate it comprehensively. Perhaps your features will positively affect not only the metrics of your team, but also – lo and behold! – on the metrics of other teams. This is very important because it will help you correctly assess the value and enlist the support of other products.
Process expert. I write down everyone who can advise on this or that issue. To understand the scale: there are already more than a hundred people on my plate. This can be done in your head or beautifully arranged in a table – it all depends on the size of the product and the number of dependencies.
An example of prescoring the feature “changing a contact number in a mobile application”
Business restrictions. There are no explicit restrictions, but you need to observe the metrics of other products to find out how we influence them.
Technical limitations. The very ability to change the number in the mobile application is not available to all types of customers, which means there is no point in showing it to those who cannot use it. The back is not suitable for implementing the feature in a mobile application, since the current back was created for call center operators.
Required improvements in the back:
The method for changing the contact phone number in the Person Profile client data change service.
Improvement of resetting caches in the API.
Refactoring multi-step confirmation in the API.
Refinement of the facial biometrics service.
Finalization of the confirmation in the core-component.
Different teams are responsible for each item. You need to talk to each one to understand the timing and plan the task.
Required improvements in the chatbot. For different types of clients, you need your own, special response from the chatbot: we lead some to the screen in the MT, the second to the CC.
Design constraints. It is planned to redesign the screen by another team, where there will be a point of entry into the feature – sync with the team responsible for the screen.
Search for interested parties. To better plan the backlog and more accurately calculate the value of a feature, I have identified four teams.
The origination command: changing the contact number in mp can affect the proportion of customers with an actual phone number, which will affect the processing speed of the application and the conversion from application to product discovery.
Person profile team: the higher the share of actual contacts, the lower the cost of their deduplication.
Tinkoff Mobile team: one of the products of the Tinkoff ecosystem is the mobile operator Tinkoff Mobile. As a rule, the contact phone number in the bank is the main one for the client. The NPV of the main number is higher by X. The “change the contact number in the MP” feature will help to promote, make the Tinkoff Mobile number a contact in the bank, increase the NPV of Mobile and not increase the cost of processing calls.
Ecosystem: with features, it will be possible to make new recommendations. Now there are two cases in the test: the client does not have to enter a new contact number, since we already know it, and the client does not even have to look for the “change number” feature, since we know that he came to change it (online triggers), and we can suggest it before he starts looking for how to do it, or asks a question in the chat.
Speed up your hypotheses before they make it to the backlog. This can save time, especially in a large company with many unknowns.
If your backlog doesn’t answer the questions “What, why, and in what order do we do it?” Is not a backlog. It should be immediately clear to you and your team members why you are doing feature A after feature B.
If the tasks are completed, and the metrics do not change, then you are moving in the wrong direction.
Guides and articles on the Internet are not the ultimate truth. Consider the nature of your product and the environment where you are.
Backlog Template – backlog