A comprehensive guide to assessing the competencies of product designers. Part 1

Hello! I Vanya Solovyovhead of product design at Magnit. Sooner or later, the question arises in a company: how to evaluate the contributions and skills of product designers. Then those in charge begin to search for information on the Internet, collecting scraps of knowledge here and there. This usually leads to inaccurate results: evaluation systems and methods are different, and combining them does not always have a positive effect on the final result.

In this three-part series of articles, I will talk about all the important stages of competency assessment that will help designers understand their strengths and weaknesses, determine the direction of development for the next six months, and help companies gain a clear understanding of designers and their value to the business. Instead of a “sightseeing tour”, I will describe in detail each stage, steps and details that are often missed, but affect the final quality of the assessment.

I have 30 product designers on my team, and we conduct competency assessments every six months. This approach is equally applicable to both small teams of up to 10 people and large teams of 60 or more. Only the scale and degree of control change, but not the approach itself.

What does a competency assessment consist of?

To begin, I will briefly describe the main stages of assessment and then discuss each of them in detail.

Communication for the team

Many people skip this step, thinking it's obvious. But because the estimate varies from company to company, the process can be confusing for many designers. The less uncertainty, the more willing people will follow it.

Common space for assessment

The comfort of completing the assessment and completing all stages on time depend on the organization of the process. When the steps are scattered across different documents or tools, it is difficult to keep in mind what and where to look. This problem can be solved using a single dashboard, where all stages will be collected and quick access to them will be provided.

Self Review

Helps the employee talk about his achievements over the past six months and highlight what he could not do and why. This is an excellent reflection step for both the designer and his manager.

360° feedback

This is a classic 360° assessment, where colleagues with whom the designer has worked closely for the past six months evaluate his personal contribution and teamwork skills. This helps to understand his interaction with the team and identify points of growth so that joint work is even more fruitful.

Skill Assessment

A key step for defining the designer's skills in detail. It is based on a matrix of hard and soft skills with a 5-point rating scale. We have improved this matrix by adding a balancing system so that it takes into account the characteristics of each grade and gives a more accurate idea of ​​the current level of the designer. I’ll tell you more about it in the next article.

Skill Assessment Calibration

To ensure that the assessment of skills was objective, we introduced calibration from design leads for each designer. This helps to get a more accurate picture of the employee's current skills.

Team design check-up

If we consider a detailed analysis to be an assessment of skills, then a design check-up is an overview of its impact on the product and business value. Thus, we have a complete picture of the team and the designers in it.

Final calibration

Another step to add more objectivity to the assessment. The design lead prepares a small summary for each designer using a template so that other design leads can agree with the final opinion or add their own feedback.

Personal development plans

We collect all the feedback from previous stages into a clear and structured document: where the employee is doing well, what he needs to improve, what are the expectations from him for the next six months and related literature, courses and lectures. This is the document that the designer and his lead will return to to track progress over the next six months.

Communication for the team

In order for the assessment to go smoothly, it is worth preparing the team for it in advance. We've created a series of posts about what designers can expect during an assessment. We described in detail the difficult moments, in our opinion, so that the guys were ready for them and knew what was required of them.

We have also prepared a clear assessment roadmap to explain the timeline and show what we will do each week, step by step. This way, the team understood the deadlines, and every week we informed them about the upcoming tasks.

As the designers later reported, this pre-assessment communication helped them set aside time in advance to go through the stages, and for some, even mentally prepare for the upcoming process.

Common space for assessment

What tools did we use to conduct the assessment?

• Notion for completing self-assessments and creating individual development plans. After Notion left, we switched to Ynote. It’s not as convenient as before, but we believe that the guys will be able to make a decent product.

• Google Forms for collecting 360° feedback.

• Google Sheets for creating skill assessments and design checks.

As a result, we ended up with quite a lot of different links for one designer, and we have thirty of them! To make life easier for designers and ourselves, we created a dashboard in Notion by command, where we included important evaluation stages and links to them.

If everyone could see the 360° feedback form, then only the designer and his manager should have seen the Self Review, the skill assessment table and individual development plans. To do this, we used access settings to differentiate between public and private. Thus, at the level of the general dashboard, the designer could see all the teams and all the designers, but could not follow the links that required privacy.

As a result, for each designer and his manager, everything came down to one single link to a common dashboard, which significantly simplified the assessment and control by design leads.

For the next skills assessment, we plan to switch to an internal tool for conducting such events. But this is a completely different story that awaits us in the future 🙂

Self Review

Give each designer a blank slate and have each one write something about themselves, their projects and experiences over the past six months. Whether this data will be of high quality depends on the designer and his ability to express thoughts. To mitigate this factor, Self Review usually has pre-prepared questions that the designer needs to answer.

The quality of the Self Review directly depends on the questions that will be posed to the designer. Therefore, we carefully studied the experience of colleagues from other companies and settled on the following questions.

— What goals have you achieved? What were you able to accomplish that was valuable in terms of your goals and beyond them?

There can be both personal goals and goals that the designer set with the manager.

— What goals were you unable to achieve? What prevented you from achieving your goal? What was missing?

What the designer is dissatisfied with and what he would like to correct and improve in the future. This can include both failures of specific goals set for him or the team, as well as his personal mistakes.

— Assess your contribution to the team’s results in accordance with your role.

Select one of the options:

• Significantly above expectations

• Above Expectations

• Meets expectations

• Below Expectations

• Significantly below expectations

– Describe in detail why you chose this answer.

In this question, the designer can write in a free format everything that you would like and that was not included in the questions or answers earlier.

By having a common template for self-assessment, design leads can more quickly process what the designers on their team have written about themselves and translate it into future development plans.

360° feedback

Like Self Review, the quality of feedback directly depends on the questions that will be answered by the designer’s colleagues with whom he has worked closely for the last six months.

We took the following questions for ourselves.

— Who gives feedback?

— Show feedback to the designer?

• Yes, show in full

• Yes, show without full name

• No, don't show it at all

This question is important because some people avoid giving negative feedback. This becomes a problem when the designer only receives positive feedback. To make things easier for a colleague, he can write whatever he wants while remaining anonymous to the designer.

— What positive and/or neutral can you say about working together?

— What would you like to improve in your work, so that the work becomes more coordinated and the result is even better?

— Contribution to the team’s results in accordance with the role occupied

• Significantly above expectations

• Above Expectations

• Meets expectations

• Below Expectations

• Significantly below expectations

A little technical stuff to make the process easier

If you, like us, decide to conduct assessments using Google Forms and Google Sheets, I recommend doing the following. First create a skill matrix spreadsheet for each designer, then create a feedback form for each designer and link it to the spreadsheet. This way you will have two data streams in one place, making it easier to work with.

The linked form appears in the table as an additional sheet. To prevent the designer from seeing the feedback ahead of time, which is still being collected, I recommend hiding the feedback sheet until all colleagues have responded and you have processed the information received.

This concludes the first part of the Competency Assessment Guide. In the next part, I'll talk about skill assessment and explain why our system is so effective.

Thanks to my design leads: Sasha Bukin, Artyom Flein, Zhenya Mikhailov and Vitaly Simanov for their help in developing and conducting competency assessments ❤️