The (not) usual CustDev. Life hacks for product testing

Hi, Habr. For prospective students course “Product Manager of IT projects” and all those interested have prepared an author’s article. We also invite you to sign up for open webinar on “CustDev for the fastest test of an idea”.


What are problem interviews?

This is just a conversation where you ask the right questions. During such a conversation, it is important to find out the actual past experience of the interlocutor regarding the issues of interest to us. An interlocutor with no actual experience can be useful to you as a link with other people. Ask him to introduce you to his acquaintances who have actual experience on the topic that interests you. It is highly desirable to record the questions and answers of the interlocutor immediately. I do not recommend recording the conversation on a dictaphone in order to record the first impression, to give the respondent a sense of value (you are recording right behind him!).

To conduct a conversation, you need a script

The scenario is based on your hypotheses about consumer needs. A good plan is a hypothesis map. Your task is to make the interlocutor speak and you listen. It is important to immediately find out the attitude of the interlocutor to the problems of interest to you. Ideally, this should be clarified even when making the appointment and the time of the conversation.

To find out the attitude of the interlocutor to the problem, you can use turnkey script or questions like:

  • Have you ever …?

  • Have you been in a situation …?

  • How often does it happen to you …?

  • When was the last time you found yourself in a situation …?

  • Are you worried about …?

  • How does … affect your life?

Examples of extended questions:

  • Can you remember a time when you found yourself in a situation …?

  • Tell us in more detail how … was the last time?

  • Can you tell us what you did then?

  • Could you tell us more about how you solved this problem?

  • Do you always do this?

  • In other similar cases, what did you do?

  • How are you solving this problem now?

  • How difficult is this decision for you?

  • What are you dissatisfied with in the current decision?

  • Why did you do that?

  • What other options did you consider?

  • What results have you previously achieved by solving similar problems?

  • What happened when you didn’t solve this problem?

  • What difficulties did this lead to?

  • What are the costs of this?

  • What did you lose in that situation?

  • How much time or money did you spend solving this problem?

Answers “yes”, “no”, “sometimes” and the like provide very little information. To get more information, ask open-ended questions – those to which the interlocutor will have to give a detailed answer.

A hypothesis map as a conversation plan is much more convenient than a list of questions, a conversation script, or any other linear plan, since the interlocutor, answering your question, may go completely wrong where you expected. If the interlocutor begins to talk about what you have as hypotheses related to other problems, then you can quickly note the connection of information with actual hypotheses and continue to dig up the questions raised. Then, when this direction is exhausted, you can return to the original problem. And do not allow yourself to speak, let the interlocutor speak out. One of the most important results of problem interviews is the found, ascertained value of solving consumer problems, expressed in terms of money or time that is saved or earned in solving the problem. If we find out how much a consumer will earn or save on average by solving a problem, we will be able to build on this when choosing a price for our solution. Ask only about past experiences and avoid opinions, abstract reflections, and speculations about the future.

Of course, the questions should be adapted to the flow of the conversation. If the interlocutor told you that at that time the solution he had chosen was the only one, you should not ask what options he was still considering. That is why the conversation script is less convenient than the hypothesis map, however, without training, you may not immediately be able to construct questions during the conversation. So practice with friends and your co-founders.

Try to “unearth” details and facts from the experience of the interlocutor, finding out specific numbers, names, dates, periods, number of repetitions, places, sources of information, etc., since these facts give us information to assess the degree of awareness of problems.

If a person agrees with you that he has a problem of being overweight and says that he does sports from time to time, then it is very important to find out how often he does and what sports. If he answers that he is swimming in the pool, then it is important to find out how often he visits the pool, which pool, how he chose it, what time he studies, how much he swims at a time, how long it takes him, whether he eats after playing sports and, in general, how he eats. This is important because in these specific facts lies the truth.

After the hypotheses are confirmed, you can proceed to the decision interview: testing the product solution.

Basics of basics

  • Experience is needed, not opinion. We need the respondent to comment on exactly the course of his thoughts and actions while completing the task. But by default people try to come up with ideas on how to improve the system right away. For this reason, videos are most often rejected. Only the visual experience of the first use of the product by a random person is useful, and not an opinion on how to “finish this feature”.

  • It is necessary to set tasks of the “top level”. That is, not “Find a button”, but “Organize a workflow”. And do not try to tell the user how to solve the problem in the formulation of the task, especially if the solution options may be different.

  • This is not a quantitative test, but a qualitative one. You don’t need to hire many people and create a representative picture. The main thing is to understand whether people can cope with the task and whether the product team is happy with how it happens (painful or not). It is enough to order 3-4 tests to be guaranteed to get one meaningful one or pay attention to several episodes from different tests.

  • It is not necessary to look for a tester from your target audience. Almost any confident PC user is suitable, at least for our product. If the product is initially very complex, then the case should be simple. For example, for a complex medical program: “… find where to load the experiment data, start the calculation and get the result, then save it to a file…”.

  • It is necessary to conduct tests on strangers. Casual people are always better than social circles. Friends and acquaintances will quickly run out, and not all of them will want to “hang” for an hour over the interface. And it is the first experience of interaction with the product that is important for us. Testing needs to be done regularly and users with a fresh eye are needed.

Searching for additional sources of revenue and growth points

Large businesses are usually mono-oriented. For example, banks or telecom operators, with a huge customer base, are limited to one backbone direction and at some point run into a growth ceiling. Therefore, they have a question – how to additionally monetize the company’s audience? A logical solution is to offer customers services for different tasks in one window, while entering the market is simplified due to the already existing trust in the brand. In addition, there is an in-depth knowledge of customer needs.

Building an ecosystem for customer retention

All world leaders, be they Facebook or Applehave been moving towards ecosystems for a long time, locking clients inside themselves. In our country, this direction is being successfully developed by Yandex and Sberbank. When, within the framework of one system and one id, one can order food, call a taxi, and watch a movie, it is much more difficult for a client to get off the company’s products. These expanded offerings build an additional competitive advantage. It’s too risky to bet on just one service today.

Increasing business margins

In large corporations, the difference between the cost of a product and its price often decreases from year to year. This can be clearly seen in the example of a telecom – the client pays for the service about the same, and the cost of its maintenance (storage and data protection) only grows. The market is changing, globalizing, more and more companies around the world are fighting for each client. Because of this, you have to look for something other than the main service.

How to work with teams and look for growth hypotheses

When working with internal products in companies, we usually rely on the potential of the product over a period of time. For example, is it possible to get an income of a billion rubles in 3-5 years? Finding and developing growth hypotheses with revenue potential of hundreds of millions of dollars requires several important steps: Collect feedback from potential customers. To move away from market demands rather than company capabilities, it is necessary to identify the presence of value in the product. You can interview clients, analyze reviews, calculate the NPC ratio (the ratio of positive ratings to negatives). Analyze sales. If the sales process is already running, you need to collect data, analyze the sales funnel and product KPIs. Then answer the question – what can be the leverage in the sales process, how to increase them? Multiple growth occurs at moments of promotion with partners, it has built a funnel between channels and launched a new user on board. Developing a strategy for the market. This includes positioning, pricing, sales channels, determining whether the product meets the expectations of the target audience, promotions, adjustments from competitors. Companies, of course, have their own peculiarities. In many cases, a significant competitive advantage for internal products would be to have a parent company that already has customer knowledge, making it easier for audiences, segmentation, and targeting.

Increase competencies, motivate the team. Any work is done by people. Within a company, there are usually no more than 5% of employees with an entrepreneurial mindset who can manufacture and sell new products. But they need to constantly learn new skills in product management, they need to be motivated and given enough freedom to develop.

Evaluate the dynamics of changes in the product. This is an important indicator that reflects how long the team lives in the product, focuses on important growth hypotheses, acquires the missing competencies and how the business is changing qualitatively and quantitatively. Regularly monitor the dynamics, set planned goals, if necessary, send the team the missing experience and resources of the parent company.

Join the Telegram channelto learn more on the topic.


Learn more about the course “Product Manager of IT projects”.

Sign up for an open webinar on the topic “CustDev for the fastest test of an idea”.


Right now, OTUS has the maximum New Year discounts for all courses. You can see the full list of courses at the link below. Also, everyone has a unique opportunity to send to the addressee gift certificate for training at OTUS

By the way, about the “beautiful packaging” of online certificates, we we tell in this article

GET A DISCOUNT

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *