From data-driven to insight-driven. Understanding how to work with product insights
Nowadays, many IT companies make decisions based on data. The data-driven approach helps monitor KPIs and digital product metrics. Understanding key indicators allows you to direct your business strategy in the right direction.
But not every company knows how to move from the stage of awareness of what is happening in the business to the stage of making daily decisions based on valuable information – insights. In this article, I will tell you how to look for them and work with them as effectively as possible.
What is insight and what is it for?
Insight — is a valuable insight gained from data analysis that can significantly impact product development strategy. Insights help identify hidden patterns, trends, and seemingly unnoticeable anomalies.
The guesses that precede insights are called “hypotheses.” They are educated guesses or assertions that need to be tested and confirmed using data and analytical methods. The result of confirming a hypothesis is an insight.
Based on insights, product managers and business owners can make informed decisions to improve user experience, increase engagement, reduce churn, and increase revenue.
Thus, in 2022, the team of one of the largest delivery services in Kazakhstan, Chocofood, noted: revenue falls during the cold season. But it was impossible to understand and fix this while in a data-driven state. Then the company decided to move to the insight-driven stage: to build product analytics processes and implement new tools that would help find insights.
The main tool for product analytics at Chocofood was Amplitude, an event-based product analysis system. The tool was implemented and the first data was obtained. The team analyzed it, confirmed it with statistical methods, and identified a valuable insight: the downward trend is related to search results within the app.
The company did not disclose exact details, but showed figures and developed a new solution – optimization of the products offered to users.
Many users have noticed that the updated app works better and takes into account the needs for specific seasonal products, such as hot drinks and meat dishes. Chocofood representatives themselves, describing this case, state that With the help of insight, we managed to increase seasonal sales by 10%.
Product Analytics – Insight Generator
From the previous case, it is clear that product analytics is needed to generate insights. But in order for it to start generating insights, a special algorithm must be used. Visually, it looks like this:
Let's look at it using a simulated example of a music service.
Step 1: Detect anomalies
The team, either independently or using analytical tools, notices a point of improvement for a specific functionality.
Example. The music service's subscription conversion rate drops on weekends. This could be improved.
Step 2: Build Hypotheses
The team begins to look for the causes of the anomaly. Analyzes databases or analytical systems, determines dependencies and correlations using mathematical algorithms.
Example. Subscription conversion is falling because users spend time with family or friends on weekends. The product analyst made this assumption based on the correlation between two factors: lower activity on weekends and the close geolocation of users.
Step 3. Test hypotheses
It is too early to call the reasons for changes in metrics insights. Hypotheses must undergo causal tests — A/B experiments.
Example. The team is testing pop-up notifications that listening to music together is much more interesting, and offers to subscribe to the new feature.
After conducting an experiment, the team confirmed the hypothesis: subscription conversion increased on weekends. However, if the metrics changed for the worse when testing the hypothesis, the team starts testing a new one.
Step 4: Find insight
After some time, the team is ready to declare that the mathematically correct answer is that in order to increase conversion, it is necessary to implement the ability to listen to music with family and friends.
Step 5: Make a decision
Once the hypothesis is confirmed, the costs of implementing the new function and the benefits it will bring are calculated. At this stage, management must decide whether to give the green light to the insight found or not.
Practical tips for working with insights
Below are some recommendations that will help you better find insights and avoid financial losses when searching for them.
1. Don't forget to maintain analytics systems
It is important to regularly check the accuracy of systems. With each update of a digital product, bugs may appear in analytical systems. Poor quality data from running systems affects the accuracy of insights and the correctness of decisions.
2. Conduct A/B experiments on a sample of users
Test changes on a small subset of users. This will protect you from major negative consequences when experimenting. To calculate the sample size, use special calculators – for example, Evan Miller's calculator.
3. Put on the user mask
Look at your product through the eyes of the customer. It is desirable that you also use the product. This way you will be able to find new hypotheses more often.
4. Think outside the box
Get rid of the framework and don’t be afraid of strange ideas. Sometimes, some insights can change the direction of the entire business. But before implementing radical changes, be sure to conduct tests.
5. Don't neglect manual methods
Nowadays, there are many advanced tools for product analytics, so specialists begin to forget how to process the results of experiments manually. But unlike automatic algorithms, a person has a unique creative thinking. Due to this, an experienced specialist can sometimes come to a completely new conclusion.
7. Don't rush
If the answer to the question “Why?” seems obvious, focus on it. Often in practice, obvious solutions lead to negative results and loss of finance. Check your hypotheses carefully. It is better to spend a few more hours than to lose a large percentage of profit.
8. Use the HADI methodology
It includes the stages of justification, purpose, verification and interpretation of hypotheses. This is a fairly simple method of working with hypotheses, which allows you to describe the processes in a structured manner.
Results
To work correctly with insights and develop a product, you need to:
1. Understand what insight is and how it appears
Insight is valuable information obtained as a result of data analysis, which can significantly influence the outcome of decisions made and the product development strategy.
Insight comes from a hypothesis – an idea that is tested in A/B experiments. Only when the hypothesis is confirmed on certain product metrics and has a positive impact on them, does it become an insight.
2. Use the right product analytics
To search for hypotheses and transform them into insights, you need to have analytical tools for collecting and processing data that are suitable for the digital product, well-established processes for working with product analytics, and an experienced team of specialists – it can not only be formed in-house, but also outsourced.
3. Follow product analytics processes
Even if you have a great team and advanced tools, it is important to process hypotheses and look for insights using the following algorithm: anomaly detection → hypothesis structuring → hypothesis testing → insight description → final decision making. It is important to follow all stages: the accumulated experience of leading global experts will help to avoid unnecessary financial losses both in the work of the product analytics department and in the entire business.
4. Don't be afraid to experiment
Product analytics processes are designed to help professionals be creative in practice. Experienced managers and their employees can find unconventional ideas that algorithmic systems cannot find. But when you are creative in solving a problem, don’t forget about insurance — use A/B tests.
