How to save your company time, effort and money using a metrics tree

for product managers in Yandex Workshop. My students do great work on projects where they need to work with metrics, but they pay unfairly little attention to the metric tree. Undeservedly – because this tool helps organize the product process so that work with metrics is consistent and transparent.

Less than 50% of products use a metrics tree – 11% in 2022 and 27% in 2023. The percentage is growing, but it is still lower than we would like. In this article I will show you with examples how to use the tool and what will happen if you don’t. Finally, we’ll look at cases of different monetization models.

What is a metric?

You can't manage what you can't measure.

Let’s look at the example of frameworks that products work with and not only – OKR and KPI.

OKRs, Objectives and Key Results

Target (Objectives): Increase the market share of the product.

Key results (Key Results) can be determined by metrics:

KPI, Key Performance Indicators

Each team has its own metrics that help monitor efficiency and identify areas of growth:

• for the marketing team – increased traffic to the site;

• for the product team – increasing conversion to purchase at all steps of the funnel, retaining and developing customers.

All metrics influence the success of a product in different ways, but are interconnected. Metric tree structures them and helps to understand at what stage and level the process or product is located. It is the visualization of metrics that makes it easier to manage indicators and control results.

What is a metric tree

This is a hierarchical composition of metrics. Sometimes a metric tree is called a metric hierarchy. The levels below are decomposed metrics, based on which a business can make decisions.

There is a top-level metric – it is called a key metric, Tsar-metric. The goal of the company and each employee is to work to improve the key metric. There are metrics at a lower level – they make up the key metric. Each subsequent level describes the metrics above.

When we work with metrics at one level, we get metrics at a higher level. For example: Revenue (GMV) = Number of Transactions x Average Check (AOV).

At the top of the tree are high-level metrics, such as revenue or number of active users.

At lower levels there are more detailed metrics that make up high-level indicators. For example, revenue will depend on the number of purchases, average check and conversion rate.

By analyzing the metrics tree, product managers determine what actions are needed to improve results. For example, if conversion rates are low, you can focus on optimizing the checkout process or improving the user experience.

Why else do you need a metric tree?

The metrics tree also helps to apply management decisions — for example, assigning areas of responsibility between teams. The product director does not need to keep in mind which team is responsible for which metric in order to plan tasks for the year, quarter, and so on.

A metric tree helps keep the focus on one key metric at a specific time period.

1. Let's start with the weakest link

Given
The site is visited by 1000 users per day. Conversion to paid order – 0.1%. This means that out of 1000 users, only one will pay for the order, and the rest will leave.

Solution
If we increase the marketing budget by 10 times, we will get 10,000 users on the site. With the same conversion of 0.1%, there will be 10 buyers.

Alternative solution
Let's imagine a funnel. At the top are site visitors who are attracted by marketing. And the very bottom of the funnel are users who have gone through all the stages and paid for the order, that is, turned into a client. In our example, the bottleneck at the bottom of the funnel and in order to get more customers from the same number of users, we are working to expand the bottleneck. That is, from the weakest link: we go to increase conversion. We increase it from 0.1% to 1% – then from the same 1000 users we will receive the same 10 orders.

2. You can’t optimize everything at once

Given
We need to improve the product to increase GMV, the deadlines are running out, we want to increase the indicator by the end of the quarter in order to receive a bonus. One feature affects conversion (let’s add video covers to the product card), the second – on the average check (add blocks with product recommendations – “they buy with this product” at the bottom of the page with the product card). Moreover, exactly how they will influence there is no data, there are only forecasts.

Solution
We are rolling out two improvements at the same time. Two weeks later, we summarize the results of the A/B test – we see that revenue has increased. How to determine whether the first or second feature had an impact? One could have a positive impact – for example, the sale of product sets increased the average bill, and the second could have a negative impact – a block with product recommendations in the cart reduced the conversion to payment. But we are still in the black. With such results, you can forget about controlled long-term development of the product, launching features without understanding which of them have a positive effect on key metrics and which ones worsen them.

Alternative solution
Make one change and see the results. Then move on to the next revision. The goal should not be to meet the quarterly report, but to maintain the purity of experiments to develop the product and achieve top-level goals.

In real practice, product teams have limited resources. Therefore, it will be more effective to conduct research on one metric, work out the pain points and needs of users, prioritize the backlog and make improvements. Doing parallel work on several goals at once is the same as working half-heartedly and spending twice as much time working on several areas.

Metric trees for different industries

Transactional monetization model

Case 1

Let’s imagine a situation: you come to the office in the morning and open a report on the online store’s revenue for the previous day. And you see that revenue decreased by 15% relative to the previous period – this is a strong drop. We urgently need to understand the reason and fix it. Let's look at two solutions:

Without using a metric tree

We are frantically trying to determine what has fallen – we open all the reports, study the site, involve the entire team in finding the problem, and pause the remaining tasks from the sprint. This way you can find the problem, but it will take time and nerves of the team. After such stress, you can immediately go dye your gray hair and go on vacation.

Using a metric tree

We open the metrics tree that has already been worked out with the team and go along it from top to bottom. Revenue decreased by 15%. What does the revenue consist of? These are orders multiplied by the average bill.

We check orders and average bill in reports. The average bill remained the same – 3,000 rubles. We see that the number of orders decreased by 9%.

Returning to the metrics tree, we see that the number of orders is the sum of the number of customers multiplied by the frequency of orders per customer.

We check the reports again: the frequency of orders has not changed, but the number of buyers decreased by 8%.

Let's go back to the metrics tree. Let's look at what the number of buyers (Buyers) is made up of – conversion to buyer (CR) multiplied by the product audience (Users).

We check the reports: the audience has not changed, but conversion decreased by 20% compared to the previous value.

We analyze the tree of metrics that make up the conversion to purchase – these are four indicators: conversion to viewing a product, conversion from viewing to adding to cart, conversion to placing an order, conversion to redeeming a cart. Based on the user traffic funnel in the reports, we find out that there was a problem with conversion to cart redemption.

We have identified the location with the problem. We went through the client's journey, and it turned out that at this step there were problems with acquiring for one of the payment methods. Within 24 hours the problem was resolved, the metrics returned to their previous high levels (1.2% conversion to payment). You are awesome!

Case 2

The product team of an online DIY goods store was tasked with increasing revenue by 15% in the next quarter. Without increasing the marketing budget, which means the number of orders will not change.

Any fool can do it with money, but try without money!
(c) Quote from great directors

The team members gathered for a brainstorm, opened the metrics tree and sequentially analyzed each level. It was necessary to understand which metric could be increased so that it would give an increase in revenue of more than 15%.

The choice fell on the AOV metric – average check. During the brainstorm, several features were identified that needed to be evaluated and prioritized.

We decided to focus on two features and add services:

Usually, only goods are purchased from an online store. By adding services, you can sell them together with goods, which significantly increased the average check – by more than 30%.

The metrics tree allowed us to see what we could start from when looking for ideas to increase revenue. Without a metrics tree, it is also possible, but the process would be more chaotic and extended over time.

Subscription model + Advertising monetization model

Case 3

Quite often, services that operate on a subscription model also use an advertising monetization model for users who use the product for free. The product will make money from those who pay for subscriptions, and from those who do not pay and watch or listen to advertising – advertisers pay for them. As they say, if you use something for free, someone else paid for it.

Let's consider how Spotify can use the metrics tree to set goals and achieve key results.

Key Metric – time spent listening to music. To increase it, you need to increase the level metrics below:

1. increase retention — returning users back to the service:

   1. add notifications about the connection of new artists;

   2. update and improve recommendation systems;

2. increase average session duration:

   1. create more playlists;

   2. notify about the appearance of new songs – for example, through charts.

I wonder how Spotify introduced a new feature for listening to podcasts and audiobooks. And here there are advantages in several respects at once:

• a new audience of users who are interested in podcasts and audiobooks;

• the average duration of a session listening to a podcast or audiobook is significantly longer than listening to music;

• retention is higher, since a podcast is a show, you want to continue watching it, but you want to listen to an audiobook.

You can imagine what the search for a feature might look like

A team of 10 people gathered in a meeting room, ordered pizza, displayed a tree of metrics on the projector and went to brainstorm. The meeting facilitator asks guiding questions based on the metrics tree:

• We need to increase the time users spend in our product. Why do users come to our product? Listen to music.

• The average length of a song is 3 minutes. You can listen to more music. How else can you increase the session duration? Can we offer anything else other than longer music?

• What do people even listen to?

• …

As a result, the team comes to audiobooks, interviews and podcasts and begins planning their implementation. The metric tree in this case serves as a tool “that is convenient to think about.”

The metrics tree can be used for different purposes, but it absolutely saves the resources of the product team and makes its work more efficient. Reduces stress, helps you work with frameworks and gain time to fix problems. By visualizing the relationships between metrics, products evaluate indicators and find growth areas where they can improve the product and achieve business goals. We teach how to build a metric tree and use it correctly on a course for product managers in Yandex Workshop.