Case from the life of a growth team
At product conferences, speakers often talk about growth hacking. We hear familiar and dead stories: Instagram and the advent of the iPhone, airbnb and hiring photographers, dropbox and free gigabytes. But their time has already passed.
And the process of growing by 70-80% consists of what didn’t work in any way, and here it’s important to learn not to fall into the traps of dead hypotheses for a long time + to be able to benefit from what didn’t work.
Sometimes at interviews they are asked to tell an interesting case, and preferably a fakap. When they find out that 3 years of work has been dedicated to the growth team in the company freemake (companies with strong processes, rituals and a team of analysts and under the supervision of Yuri Drogan).
People are learning from it. I’ll tell you about how, when I was a growth product, I got stuck on the idea of customization, and it led nowhere. Although the profit from this and I and the team received.
At that time, the growth department had barely started. The team studied and implemented modern grow methodology, processes and best practices. The founders supported us as best they could and believed in us.
We did not question the basic ideas of growhacking, but tested it in practice. One of the principles: a well-formulated value will lead to an increase in metrics at all stages of the funnel: from advertising to “resurrection” of customers. And we decided to conduct an experiment with the value of customization (we address the user not in general, but as if specifically to him).
For India: special edition
The product is global, works for customers in all countries.
There is a value with which we can hit the maximum number of users in each locale.
This value is the locale itself. The closest and most understandable customization is to declare to users: the product is made for you, we know your pains!
Gender, height, weight, ideology, personal values - it’s hard to get on a global product. Highlighting that the product is specifically for this country is a crude idea, but it is quite working. They took it to work.
We will collect data as quickly as possible in advertising – at the very top of the product funnel. Awareness. We run ads in Goggle Ads, where two options participate in the AB test. The usual advertisement for the converter and the same advertisement with the addition “For India: special edition”. According to the classics, we calculate the amount of data on which we can see a reliable increase of 20-30% (if less, then it’s not worth doing it much).
Ad CTR skyrockets by more than 40% Of course, at that time we do not realize that growth is possible stupidly due to the visual increase in ad size.
I, as a product of this hypothesis, fall asleep and wake up with thoughts about customizing everything. Landing page, product, offerwalls, mailing lists – nothing can resist my father’s hypothesis.
As a real Scrooge McDuck, I understand: the philosopher’s stone has been found, grow-hacking works great, I expect money rain and a multiple increase in all metrics. The dream of any product is to find a strong driver of growth and scale.
The logical step when testing a hypothesis at the top of the funnel is to push down what worked. It worked in advertising – we roll it onto the landing page. On software. For offers. Let my brilliant idea cover the whole world.
Pirate Funnel AAARRRR
Sequence in the funnel:
Awareness – Advertising, rating sites or SEO traffic
Acquisition – Install our software
activation – Get an aha-moment, that is, process the first file (add to software and convert)
Revenue – offerwalls, paywalls (payment system)
Retention, Referral, Resurrection – we do not climb further with this idea.
Attempt number one: do it quickly and with little bloodshed
We can drag the idea that worked from the first stage to the next three. Let’s start with the nearest one.
We customize the site.
May the citizens of India forgive me for the cliche, but I did it this way …
We considered the first attempt sufficient. Minimum code, no design, thought reflects.
We launched an AB test in advertising: an old landing against a new version with customization.
No statistically significant changes were seen in the test result …
Delight and anticipation were replaced by bewilderment… How so? Such a cool idea that showed itself perfectly – and failed??? So they did it wrong!
How could I anticipate or resolve errors at this stage:
The right question for yourself and the team is: could only a message about the locale and special transition affect the growth of the metric?
Quickly run hypotheses that test the answers to the first question. For example, lengthen the ad in option B with any other text. In advertising, it is clearly faster to check.
See if there are any stories of such customizations at all.
Grow doesn’t give up
We flogged this question and discussed it with analysts. The idea sounds good. Perhaps the implementation is too weak – so they decided. There is no huge queue in the backlog, you can restart and see. We thought over the design “with recognition, in order to accurately hit the values.” Cliché on cliche, but we are a team of growth: the main thing for us is fast!
We relaunched, energized by dopamine from future success…
Come on, tell me how you amplify and restart hypotheses.
No significant difference stat is visible on the given sample size.
It didn’t get worse or better. The belief that such an implementation would give a result was high. Visually it looks nice. Recognition of places should be.
But the world of growth hypotheses is cruel.
Fuck-ups like a springboard
Normal teams learn from fakes. We are normal guys. What conclusion can be drawn from these two premises? Right.
Our conclusions, or How to continue to reduce the time on the experience of this story:
Maximize the message that the hypothesis carries.
Do not be shy. It is better to spend more time: think over, discuss with the team and strengthen. It doesn’t have to be done well. But – enough good to do right now.
riskiest assumption test – one unsuccessful hypothesis with a strong elaboration will kill not only this hypothesis, but also the whole direction of thought. As hard as possible to check if there is a chance that it will work. If the strongest implementation does not work, then the weak ones disappear immediately.
When selling hypotheses to the team, always ask the question, “Is this really the strongest implementation? How can we strengthen it?” – I must say right away that this question saved us a lot of nerves and saved a lot of time. It is better for one person in the team to think and think for a day longer than the whole team + development + design + time to collect data … Uffff, as a result, it’s obviously “cheaper” for one person to work with their head.
We dug up the idea with customization. I agree, there is a possibility that customization did not help on the site, because. where the conversion limits have been reached. Or the user’s emphasis is not on this at all, but on paywall it could work better …
Thoughts have the right to life, but we rated it as unlikely. Nailed this direction.
And what is the main thing here?
The main thing is to discuss with the team not only successes, but also fakups, and draw conclusions.
It’s just so simple: there was a fakap, a strange result, some event changed your picture of reality – they specially gathered and wrote conclusions. We created an action plan – what to do so that it does not happen again, and who is responsible for not forgetting to do this.
And step by step we set up our devices for an adequate perception of reality.
After spending a couple of weeks on this hypothesis, we drew conclusions that saved us months. Profit!
What do you think? Are we stubborn or did we do everything correctly and logically?
PS: six months later, the sales manager decided that “we implemented it incorrectly, we need to get into the values more clearly” – and wrote down his own version. The result was the same, although the landing page picture is more fun. But that’s another story, which has its own conclusions :))
PPS: Draw your own conclusions, friends!