test for searching for the position of Junior Product Manager

My name is Herman, I am the head of the loyalty product line at CSI – we are developing a product for managing customer loyalty in retail called Set Loyalty. We recently decided to expand our product team and I needed to develop a test task to select suitable candidates for the position of Junior Product Manager. I did this for the first time and from scratch. I decided to record this experience and briefly share the principles that guided its preparation. I'll also show you the test itself. Looking ahead, it has already helped us find a new colleague.

What test task did we want to receive?

1. The test task should “provoke” the candidate to demonstrate the required qualities.

Our team includes members with deep expertise in retail. At the same time, we lack colleagues who have real experience in using tools for marketers – advertising, communications, analytics, CRM or CDP platforms. The test task contains several questions, the purpose of which is to force a suitable candidate to demonstrate the required qualities, namely: an understanding of tools for marketers and a willingness to apply their knowledge when designing solutions for the product being developed. Specifically, in task No. 1 there is question 4, and in task No. 2 there are questions 2 and 3 (the entire test is given below; without context, the questions will be incomprehensible).

2. The test task should simulate real tasks that the candidate will have to work with in a team.

For the assignment, I selected tasks that, in my estimation, will take up to 70-90% of the candidate’s time after hiring. I think this approach is fair to the candidate: it immediately becomes clear to him what he will have to do in the team.

3. You must provide all the necessary information to solve the test task.

It is quite difficult to answer the test questions without knowing anything about the product. We have provided information about the product, target market, user, top-level menu section structure and key features. They did not talk about one part of the product because it could immediately lead the user to the correct answer.

4. The test task is based on real problems solved by the team in the past.

Our test consists of two tasks. Both tasks are real issues that needed to be solved at a certain stage of product development. My team and I spent a lot of time discussing possible options. I knew well the pros and cons of all possible solutions. This helped evaluate candidates' answers less subjectively.

5. Reasonable deadlines.

We proceeded from the fact that the test should be either small or paid 🙂 Our test did not seem very large to me. Candidates reported that they spent between 3 and 8 hours completing the task. I consider such labor costs to be quite acceptable for applicants for the position.

What did you get in the end?

We were looking for a candidate for the position of Junior Product Manager. Out of 39 responses to the vacancy, we gradually reached the point of sending a test task to 4 candidates.

Responses were received from all four. The answers were meaningful, with reasoning and constructive argumentation. Three out of 4 answered the key question of the TOR in the same way as the team answered it in real working circumstances (note that the correspondence of the candidate's answer to the team's answer was not a selection criterion). Two out of 4 candidates showed the required qualities in their response: an understanding of tools for marketers and the ability to use this knowledge when designing solutions.

In the end, as I already said, we made an offer. It’s interesting that initially I was quite skeptical about this candidate: the resume did not contain references to the tools in which I needed knowledge, but in the test the person demonstrated this knowledge and suggested a solution based on specific tools.

It is pleasant to note that candidates also spoke positively about the technical specifications during interviews:

Thanks for the test. It was very interesting and consistent. Everything was laid out in detail and painted. It was a pleasure to make. And what’s even more pleasant is that it actually correlates with the activity you are about to do. I've never encountered anything like this before

I am glad that the introductory information about the product, target audience, structure was presented in some detail. It was interesting to perform such a test task

Test

Next you will see the task itself. In the end, I will only say that this experience will definitely be useful to us for the subsequent hiring of products in the department – and, perhaps, to someone else who is now thinking about their technical specifications?

Write what tasks you completed when applying for positions in product teams. Or what tests do you give to your applicants 🙂

Quest texts

Task No. 1

Message from Head of Product Set Loyalty

“Hello!

Earlier I told you about our cash mechanics editor. In the cash mechanics editor, the Set Loyalty user configures the rules for the operation of the promotion made upon purchase. For example:

1. Buy 2 packs of “House in the Village” milk for the price of 1.

2. Buy Cheese “Russian” 200 gr. and Ostankino sausage 300 gr. in the package and get twice as many bonuses as usual on the card – and so on.

So here it is.

Merua Lerlen has just contacted us. They use our product and are currently running one of the similar promotions (similar to promotion number 2 above), configured through this editor. But now they need send a message to the buyer’s phone immediately after he has taken part in this promotion – it does not matter in which channel the buyer made the purchase – online or offline.

The approximate text of the message was sent as follows: “Keep it up, Sergey. You now have 234 bonus points in your account. We are waiting for you to make new purchases in the Merua Lerlen chain of stores.” And so on for each participant.

The scenario, in principle, is product-based – not only Merua Lerlen can be useful. But from my own experience I can say that the task is not as simple as it seems at first glance. I propose to meet with the product team tomorrow and discuss the approach to designing settings to implement such a scenario.

Please prepare your position by answering the following questions in detail:

1. Which path is better for us to take – add a new stage with mailing settings to the promotions editor (cash mechanics) or create a separate editor with mailing settings? Why do you think so?

2. Is the information above enough to design your mailing settings? Do you need to ask any clarifying questions to Merua Lerlen? What questions do you want to ask? Which employees on their side should be asked additional questions? questions (in what positions)?

3. Is there enough information from Merua Lerlen alone to design a quality solution that will add value to our product for all users, and not specifically for Merua Lerlen? Why?

4. What stages of mailing settings need to be highlighted? What settings exactly do you think the user should set at each stage?

But remember, we are not a design company and are not obliged to solve the user’s problem the way he tells us. We have the right to develop an optimal solution that, at a minimum, will not harm our other users, and at a maximum, will be very useful to them.”

Task No. 2

Message from Head of Product Set Loyalty

“Hello.

Marketers from Stixie contacted us. And they need a new type of segment creation in our system. The situation is as follows.

You already know that this is a fairly large client. And large retail chains usually use BI systems for advanced analytics of consumer behavior. Such users pour a very large amount of information into the BI system from a variety of sources: CDP platforms, communication services on the website, website, mobile application, services for collecting feedback and many others. Next, marketers build reports, graphs and tables in the BI system. A large amount of data allows marketers to see a broader picture than the analytical tools of each of the above services separately can provide.

So, “Stixie” now wants to create a list of buyers in this BI system and download a list of phone numbers from it. And then add them to a file, upload it to our system, create a segment and use it when creating conditions for promotions, communications and for other tasks in our system.

This is the first such request to us. And personally, it seems kind of ridiculous to me. I haven’t seen such functionality anywhere and I’m not sure that it can be of value to marketers.

But nevertheless, it needs to be discussed within the team. If we recognize it as valuable for all our marketer users, we will make it free in our product. And if we don’t recognize it, we’ll ask Styxie to compensate for the costs of implementing this functionality. In general, we need to discuss it at tomorrow's meeting.

Please prepare your position on this issue by answering the following questions:

1. Do you think there is value in this feature for marketers (all our users, not just Styxie)? Why?

2. What settings, in your opinion, should the user perform from the moment he initiates the segment creation script in this way until he clicks on the “Save” button?

3. Draw draft window layouts containing the settings you wrote about in the previous paragraph. Don’t worry about the quality of the drawing – just draw in general terms (without details) with a pen on paper and post the photo here.

4. Do you think there should be any requirements for the downloaded file? For what?”

PS If you want to join the CSI team – develop the best products for the retail market in Russia and the CIS – welcome!