My name is Ilona, I am Senior Experience Designer at EPAM. Work for me coincides with a hobby – at EPAM I design interfaces for foreign customers, give lectures for employees and students of the lab, mentor designers. In my spare time I teach interface design at ITMO University Master’s program and lead Telegram channel about UX design…
Evaluation is the process of evaluating work before it is completed. This is one of the most important and painful topics for any specialist, be it a programmer, tester, manager or designer. You are always asked for the most accurate estimates possible in order to plan the work. But the estimate is just a hypothesis that may diverge from reality. And the less clear the task, the greater the discrepancy.
In this article, I want to share the specifics of evaluating design work and the approach that helped our design team in one of the EPAM projects.
Project and design team
The project we are working on is a large e-commerce platform with a large team (80+ developers, managers, analysts, testers) and rapidly changing requirements. On such a large project, a design team of 4 roles was logically formed:
UX designer, whose task is to find out the requirements and generate the most user-friendly solution.
UI designer (EPAM also refers to this role as “Visual Designer”), who is responsible for ensuring that the interface is beautiful, aesthetically pleasing, looks good at different resolutions, matches the brand concept and development capabilities.
Usability specialistwho regularly tests the design with real users of the system, interprets the results and makes a list of recommendations for improvements.
Team Leadwho establishes processes and solves organizational issues, provides the team with a comfortable working environment and also deals with UX design.
The processes in the project are built around Scrum: sprint planning with task assessment, two-week sprint, review and retrospective. The team evaluated the tasks in hours: how much time we manage – so much we take in the next sprint. Sounds straightforward, but estimating in hours was a big challenge for the design team.
In general, design tasks are difficult to evaluate for several reasons:
Design requirements are usually the most informal and come in the form of “make it good.” This is normal, but it is difficult to estimate in hours the time spent searching and testing a solution. And you still need to have time to draw it and prepare it for development.
In the course of work, you have to spend time discussing and brainstorming some tasks within the project or with customers. People have different availability, timing, and even time zones. The process can be postponed, delayed, and the task – “hang”.
Each idea needs to be checked and approved with the customer. And it’s not easy to understand how long it will take and how the result of the discussion will affect the designer’s work.
The design must be verified by the development team. If the proposed solution is technically impossible to implement on time, the design needs to be finalized. It is difficult to predict in advance whether this will happen and how long it may take.
Here are some quotes from retrospectives that we conducted within the design team:
I decided to find a new approach to assessment that would help the design team to cope with the difficulties. And then she came to the rescue relative difficulty score work.
Relative difficulty score
Sounds pretty … complicated. I will try not to go deeply into the principles of Scrum, but will only illustrate the approach using the example of our team’s work.
According to Scrum, an abstract unit is used to evaluate tasks. Story point. In one team, it can measure hours, on another day, in the third something else. As it is convenient for anyone.
In our team, 1 Story point was equal to 4 hours. And for the reasons listed earlier, it was extremely difficult for us to evaluate the work in such Story points. Therefore, I decided to endow the Story point another measure – complexity…
Complexity – The amount of effort required to complete a task.
In our case, the assessment of difficulty in Story points turned out to be much more effective than the assessment of hours, because:
Assessing difficulty helps you focus on creating value rather than chasing time. We stopped trying to “get at least something in time”.
It’s easier to include discussions, affirmations, and brainstorms in the final grade.
The client will perceive better: “the task is very difficult” than “I need twice as long to complete the task.”
Discussions in the format “I would do this job faster” are excluded.
We usually spend a lot of time arguing about how long it takes to complete a task, because time is a very personal measure. I need 5 hours, but my colleague may need 8. The assessment of difficulty is not so personal and people more easily come to a single assessment.
Perhaps these reasons will apply to your work as well.
Complexity – a complex measure, and it’s hard to tell right off the bat what it means. For the convenience of the design team, I have divided the complexity into 4 parameters *:
Addiction to other people
The amount of communication within the task
Consistency of the new design solution with the existing ones
*although the parameters describe the work of the designer, I think they will be relevant for other professionals as well.
For each parameter, I selected scales from 1 to 4. For example, the parameter estimate Dependence (Dependency) looks like that:
one (no difficulty) – no one except the designer is involved in the work, there is no dependence on other people;
2 (low difficulty) – 1 person involved and a bit of outside work;
3 (medium difficulty) – 2-3 people and / or a lot of outside work;
4 (high difficulty) – more than 3 people and / or it is impossible to determine the amount of work from outside.
Detailed scales for all four difficulty parameters can be found in my report for the Design Z Day conference:
How to understand what is difficult and what is not
Everything is relative. ©
If the time can be specified specifically (for example, “two hours”), then problems arise with the estimate of complexity. What do you mean “difficult”? Where is the line between “difficult” and “very difficult”?
Secret – relative score… Problems are not evaluated in a vacuum, but relative to each other, in comparison.
To measure the difficulty of a problem in abstract Story Points, you need to compare two problems and understand how much more difficult one is than the other.
For example, task # 1 is to redesign the site login form. Task number 2 – rename the buttons on the registration page. Task # 1 is clearly more difficult than task # 2, and we don’t have to decide how long it will take to complete each of them.
The comparison process is always based on the experience of the specific team. Therefore, the ratings of each team are unique.
How our team started using relative difficulty score
It was difficult for us to change from an absolute watch estimate to a relative difficulty estimate.
To begin with, I described all the parameters of complexity in the internal documentation, gave examples of assessment. We discussed this with the team. Next, we did the following exercise:
From the last sprint, which we evaluated in hours, we took 3 problems. We have already completed them, so we knew everything about them. For each task, we consistently went through 4 difficulty parameters (dependencies, skills, communication, consistency) and gave new estimates in relative Story Points. It was not easy, we discussed each assessment for a long time and came to a single figure. Here’s what we got:
As you can see, a particularly strong discrepancy happened in the last problem. We came to the conclusion that, estimating in hours, we took a lot “with a margin”, because we knew that we would need to communicate with one colleague from the marketing team, and were afraid that we would wait a long time for an answer from him or the discussion would cause problems. Assessing the complexity of the same task, we realized that communication will actually be small and the complexity will be low. As a result, the complexity assessment turned out to be different.
Later we repeated this exercise for all tasks, calibrated our grades and studied. I asked all team members to remember to compare tasks during assessment sessions and to focus on difficulty rather than time. Gradually, comparing tasks became a habit and it became easy for us to assess the complexity of the proposed job.
Results of the transition to the relative estimate of complexity
I want to share the team’s feedback on the first sprints with a new approach – a relative estimate of complexity.
Work has become more predictable and comfortable. We spend less time on assessments, and it gets easier and easier with each assessment session. Gone are the problems of assessing the cost of communication, disputes and the personal factor of the speed of work. The team focused on the value that the task brings, and stopped trying to “do at least something.”
Important results for our design team:
transparency of tasks;
more accurate estimates with less effort;
the design team understands “what we need to know before completing a task”;
better decomposition of tasks;
all work is taken into account, including approvals, generation of ideas and discussions;
we constantly compare tasks and see the big picture.
As a result, there is less stress, overwork and unpleasant surprises.
You can read more about designing interfaces and UX in my telegram channel “Explain for UX“.