How we assessed any customer requirement in two hours
It's great when a company already has a mechanism for assessing requirements, or assessment is done by architects / PMs / POs and other people who are not analysts. But our company did not have an established assessment model, and each team spent a lot of time assessing customer requests. Then we had to develop our own requirements assessment scheme, spending a lot of time on it – first gaining experience and testing it, then implementing and explaining the methodology to employees.
Let me give you some background on the subject area right away – we had a ready-made system with bottlenecks in implementation, and a lot of different consulting around features for refinement (i.e. most of the features required a methodological approach and it was possible to either implement it “head-on as written” or offer the client to use ready-made things in the system, but not as his requirement sounded).
It is also necessary to separately stipulate that in our Company, requests were mainly handled by analysts, not because they know how long it takes to implement each feature, but because technical specialists (developers and architects) wanted a detailed description of what would need to be developed. Since the features were mostly more or less similar (to refine a card, to make a procedure, etc.), the analyst could, based on his experience and our tips for evaluation, enter the required number of development hours. For complex and new things, of course, analysts turned to architects for help.
Below I will describe the framework we began to use to evaluate requirements.
We define the types of requests
We divided all presales (here and below we mean contact with a new client) or requests for the implementation of functionality (additional requirements from a current client), which were sent to the Company for evaluation, into three types depending on the goal pursued by the company sending the request:
Type 1: Evaluation of system capability. At this stage, the customer does not ask for an estimate of the requirements in terms of labor costs in hours, but wants to understand whether the solution is suitable for him.
Type 2: Evaluation of requirements for feasibility and description of the implementation option for the functionality. Here, too, they do not ask to estimate labor hours, the customer wants to understand whether it is possible to do what he wants or not and how.
Type 3: Estimate in hours according to requirements. It is clear from the name.
Here and below I am talking only about the estimate in hours, the cost is usually calculated not by the analyst – depending on the rate or some other way, as is accepted in the company. The analyst sent the “raw” estimate by hours to the project manager.
Recommended Efforts for Presales Evaluation
Since the evaluation of labor costs is not a paid job (there is no contract yet), we made a strong-willed decision to spend no more than 2 hours on evaluating the requirements. The only exceptions were those presales that were considered most likely to result in a contract being concluded. To do this, before the evaluation, the presales manager would assess its probability by expert means, or the analyst would quickly review the requirements and could conclude whether this was our customer or not. This happened quite often, since our solution was not clearly highly specialized and its scope of application could be interpreted in different ways.
We also limited the evaluation time because clients value response speed – long silence and too much detail are worse than quick feedback and a general assessment.
Presale type: 1. Initial assessment of the system's capabilities
This type of presales is typical for the very first request from the customer, when he is looking for suitable systems/products/platforms on the market.
Usually, in such requests, the customer's goal was to justify the choice of a particular system/product/platform to their management, and for this purpose, a questionnaire on requirements is created and sent to the development companies. For this, a version of the point system is also often used, where each requirement is asked to provide an assessment of feasibility and a comment.
Therefore, we honestly gave requirements that could not be implemented “zero points”, and the remaining scores were given with a slight increase than it seemed at first glance, because subconsciously we, as experts in our field, put more meaning and complexity into the requirement than the customer. Usually, when discussing the requirements later, the customer chose a simpler implementation option so as not to delay the development in terms of deadlines.
The result of the assessment was the sum of points for all requirements and the customer could choose between the systems the one with more points. Also, in addition to this, the “weak points” of each of the systems or, in principle, requirements that are not covered by any system on the market were immediately visible.
Presale type: 2. Feasibility assessment
The customer asks you a throwaway question: “Can you do this for us?”
This is very similar to an assessment of capabilities, but most often such a request is very specific and aimed at solving some problem of the customer. In this case, it was necessary to orient the version of the requirement implementation towards solving the problem, and not towards the specificity of its formulation. Since usually this version of the request already assumes that the customer has “thought about” and figured out how to implement it himself.
Presale type: 3. Estimates in hours
The most specific request is that the customer wants to understand how much it will cost him.
—-
General rules of evaluation
The initial estimate was made without reference to the financial capabilities of the customer (whether they can afford it or not), because the Customer sent a formulated request and wants to receive an answer to it. Usually, after the customer received the estimate, a discussion of implementation options began, crossing out items and adjusting estimates. That is, our initial answer on cost is an approximate cost, and this was immediately indicated in the cover letter.
Sometimes it was clear in advance that the customer has some financial constraints, and he asks to estimate the requirements flexibly, i.e. expects us to offer an implementation option that either already exists in the System, or that requires minimal costs – the main thing is that it performs the same business function that the Customer includes in the requirement. That is, it was necessary to conduct an analysis in advance and describe the implementation option in the estimate. At the same time, if there was a requirement that required extensive development, and we could not provide it “out of the box”, the estimate was still estimated adequately to the request, even if in the sum of all the requirements we went beyond the designated financial limit. The Customer could look at the result and refuse this requirement in order to meet the constraints, or even assign it to the second stage of implementation.
Estimate in hours for a new customer
A new client is always a pig in a poke, and not only for you, but also for them. Therefore, we decomposed the on-demand estimate into clear actions and clear values. For example, it could be creating a new card in 3 days. We considered a week of development to be a clear value of the estimate. For some large and unclear things (for example, integrations with new systems), which require preliminary research, we derived a pattern based on previous contracts and gave the customer a range of implementation options and estimates, taking into account that the other system is already ready to integrate and give everything that is needed.
What risks could there be for the new customer:
many stakeholders for decision making (long requirements gathering and long acceptance),
own documentation requirements,
possible personal visits for delivery/clarification of requirements, presence of employees on-site during trial operation,
safety requirements, etc.
A certain percentage of risks (from 20% to 40%) was included in all of this, taking into account that after the initial assessment there would be a discussion and removal of some of the risks.
Estimated requirements (in hours) for the current customer
The current customer is clear risks, a clear pace of the team for us and, in general, simplicity and beauty. In this case, the assessment was carried out by the same team that will develop the functionality. Things were a little more complicated with old customers (the company), but with new people on their side and ours. In this case, we simply applied the rule “this is a new customer” and assessed accordingly.
For the current customer, we additionally did not forget about the specific features of each, for example, the customer has a tendency to test all requirements in detail, so we need to budget for both analytics and testing. In addition, add hours for meetings and incident handling (if any).
The estimate for incident processing was calculated as = total number of hours spent on all incidents (from the customer) / number of requests (from the customer). This will give the average number of hours per incident. Then we estimated the approximate number of requests from the customer on previous projects with him.
Thus, in the assessment of the current customer, we also included all of its features.
Role Ratings
The customer wants to understand “Why so many?” (and this is not the first iteration of developing estimates).
The answer was divided into two parts:
A specific description of what will be done is sufficiently detailed and understandable for the business.
Breakdown by roles, who will spend how much on the task.
When it is clear which role will do which part of the overall “pie”, sometimes the customer is more lenient in accepting estimates for the task.
We derived a pattern of our labor costs for each role (analyst, development, testing, management) and for any task we could easily calculate the contribution of different roles. In principle, you don’t have to do this, and each time for each task you evaluate separately, but if such tasks become more than one Excel screen, we spent an inordinate amount of time.
I will not provide estimates here; each company, business and system will have its own.
Estimation by duration
If everything was completely unclear, we could start from the team's workload for the year. In a large project with unclear requirements, frequent releases or with work on Time&Material, the team, for example, we calculated like this:
RP – 0.3 people
Analyst – 1 person
Developer – 2.5 people
Architect – 0.7 people
Tester – 0.8 people
Total = 5.3 people.
(Not whole values - this is partial involvement of the employee, somewhere he will work 100%, and somewhere at 0%).
For example, on average a project is planned to last 1 year, usually including (very roughly):
1.5 months – development of requirements
8.5 months – development
2 months – trial operation, implementation
That is, according to the most rough calculations, it might turn out to be:
= 5.3 people * 12 months * 8 hours * 23 days = 11,702 hours.
More accurate estimates could have been obtained if the stages of project development had been taken into account:
1.5 months – development of requirementsusually the RP (0.4 people), analyst (1 person), architect (0.2 people) participate = 1.6 people * 1.5 months * 23 * 8 = 441 hours;
8.5 months – development (the whole team, but there are 0.2 RPs) = 5.2 people * 8.5 months * 23 days * 8 hours = 8132 hours;
2 months – pilot operation, implementation (The RP is usually more involved (0.5 people); 1 analyst, the specified tester and part of the development team for bug fixes (for example, 1 person), architect (0.5 people)) = 3.8 people * 2 * 23 * 8 hours = 1398 hours.
Total in this case: 9,971 hours.
In addition, it was necessary to take into account the format of work. The simplest format of work for assessing requirements was described above – waterfall. With flexible approaches, the formula is more complex, taking into account the constant involvement of an architect to collect packages, an analyst and a tester for demonstrations/deliveries/feedback from users, etc.
Variation of min and max estimates
Sometimes the customer asked to estimate the upper and lower limits in the implementation options. That is, for example, you can develop a service for loading data from one system to another (max), or you can estimate the labor intensity of unloading into Excel from another system and semi-manual data entry into ours (min).
Then we described two implementation options and assessed them as separate cases. Perhaps, in some minimal implementation option, development might not be required at all. Everything depended on the requirement and our imagination for its implementation.
Dealing with an unclear requirement
Sometimes we found ourselves in the unpleasant reality that the requirement was unclear, and there was no one to ask what was meant. Then we came up with a rule for ourselves – to offer two implementation options: a simple scenario convenient for us and a complex scenario. Then we assessed both scenarios and derived a score using the following formula: (3 * simple scenario score + 2 * complex scenario score) / 5.
What else should have been taken into account?
Requirements for documentation. Sometimes there was work for a month of development, but we wrote all three of the documents. This was immediately apparent either in the technical specifications that were sent to us for evaluation, or already during the discussion of the requirements.
Method of delivering the result. Yes, we had customers without the requirement to deliver the results iteratively. We still worked with releases, but the customer wanted to see the overall system only at the end. There were no overhead costs for testing, maintenance, assembly of releases, demonstrations, and the estimate in this part was lower. With flexible approaches, the estimate would have been different.
Customer activity and implementation size. It is one thing when 10 people work in the system. Another thing is if the system is implemented in 10 subsidiaries with 100+ users in each. This is a different level of support, including during trial operation.
These are just a few examples from a long list of those aggravating factors of evaluation that we have identified for ourselves.
Iterative approach after requirements assessment
The estimate given for the requirements is not final. Many forgot this when they first faced the need to estimate the requirement. After the estimate was prepared, it was sent to the Project Manager (if the estimate was given by an analyst/architect), and with additional input, it could be adjusted (up or down) according to how the Project Manager saw it.
After the estimate was sent to the Customer, there were various meetings and discussions of the estimates, so we always had to be ready to justify the estimate and the implementation option – that is why in most cases the analyst was involved in the estimates.
The process was repeated until both parties were satisfied with the assessment results and the format for their implementation.
—
But how exactly did we evaluate the requirements?
At the beginning of the article, I said that we set a 2-hour limit for evaluating any requests from the customer. We could only allow ourselves such a limitation after collecting analytics for our analytics and development department for several years. We concluded that for every hour of development, there is a certain “overhead” coefficient in the team – analyst, architect, RP and tester. The coefficient also varied depending on the customer (in our case, it was the current customer / new customer, their field of activity), and the scale of improvements (more than a certain limit of development hours / less). Therefore, all that was left for us to do was to determine how many net hours the development would spend and what coefficient to apply.
This did not always work perfectly; in some projects we missed the mark in different directions, but over the course of one year, across all department projects, we managed to level out within our own coefficients.
