How prototyping helps digital product development

For ourselves, we distinguish between a prototype and an MVP version of a product, where the second is implemented at least on the company’s target stack and meets a number of criteria for digital products. In the case of a prototype, we focus on the speed of testing the business hypothesis despite the implementation stack/tools.

The main goals of prototyping for the company

• minimizing risks for the company: first testing the hypothesis, and only then implementing the target product;

• reducing the time for drawing up technical specifications, developing the target type of solution, reducing the number of adjustments during the subsequent development of the product;

• improving the understanding of the project (required tool) by the business customer: adjusting the required functionality, type, operating principles, etc.;

• search for new ideas: solving the problem through proposed alternatives, both technical and functional;

• reducing the labor costs of developers during the further implementation of the target product, identifying errors at an early stage (for example, in the business logic of the solution);

• and most importantly checking the business hypothesis: how much does the business really need this solution (survivability, demand, viability, sufficiency and correctness of solving the tasks);

• conducting a pilot launch: obtaining/confirming a hypothesis in certain metrics, indicators.

The prototype allows you to test the selected solution option without significant financial and time resources; get an answer about the need for further investment in this direction. Over the entire period of existence of the practice, we have tested 50+ different hypotheses, varying degrees of complexity and uncertainty of the result.

Perhaps we can highlight a prototype for automating the process of damage compensation and dismantling of wagons. The primary task was to actually test the hypothesis to what extent this approach is generally possible while minimizing manual labor. Based on the results of the pilot launch, the prototype showed the high efficiency of the solution and helped colleagues in identifying cars with signs of damage. It was decided to expand the functionality and implement a digital product, which to this day brings financial benefits to the company.

At the same time, we were approached with requests for prototyping very large-scale projects in the future. We implemented prototypes, clarifying the requirements together with customers to fully understand the issues. In some cases, a simpler solution (for example, adding functionality to existing systems) was sufficient to obtain results for a number of operational indicators.

In our practice, there are examples when a pilot launch of a prototype revealed the inappropriateness of further development. In one case, it turned out to be too expensive and time-consuming to implement the target solution, which certainly would not have brought the expected financial results; in another, it turned out that the calculation method was incorrect and it was necessary to refine the business logic of the process.

Such cases perfectly show how prototyping at an early stage gives results and allows the company to save financial and time resources.

Initiative Launch Process

Prototyping stages: how it works here

• Kick-off meeting: the customer schedules a meeting with the team and at this meeting outlines the essence of the hypothesis that needs to be tested or the prototype that needs to be implemented.

• Elaboration: After carrying out the testing, the prototyping team takes a short break, where they think through possible implementation options and return with feedback within a week.

• Requirements meeting: Based on the results of the development, a meeting is scheduled where the prototyping team collects requirements for the prototype – interface, functionality, success criteria (measured metrics), etc.

• Documentation: The project manager, together with the analyst, prescribes a roadmap with an assessment and implementation deadlines, after which the analyst creates functional requirements and technical specifications and sends them to the customer for approval.

• Development: When the implementation deadlines and terms of reference are agreed upon, the prototyping team begins working on the roadmap. Depending on the complexity of the project, we agree on intermediate results and the intensity of meetings with the customer.

• Acceptance: Upon completion of development, we demonstrate the results of the work and collect a list of customer wishes for the implementation stage of the target product.

• Prototype transfer: We prepare design documentation and transfer the final version of the prototype to the customer, and organize a pilot launch.

• Target solution: the customer contacts the project office to initiate the development/implementation of a solution in the target system/product. If necessary, the customer goes through the stages of the architectural committee and the IT committee, and prepares the necessary documentation.

• Revision: in some cases there is a need to introduce
  improvements/edits to the prototype to obtain a more accurate pilot result
  launch; We agree with the customer on the estimate/deadlines and make adjustments to the roadmap
  project map.

Tools for implementation

Python, JavaScript, SQL, Airflow, Excel, BI tools and Figma. The team also has no/low-code solutions in its arsenal, such as Bubble.io, Tilda, Adalo, Glide and others. Moreover, the emergence of new tools and frameworks that allow you to quickly assemble digital products is constantly monitored, the team tests them and puts them into operation.

Metrics

We view the prototyping team as a separate practice and have implemented a number of metrics to track the effectiveness of our activities. The slide shows the metrics tree we use.

Key metrics include: prototyping requests, idea funnel, return on investment, labor costs and customer satisfaction. Below I will tell you in more detail about each of them, why and what they are needed for.

1) Requests for prototyping.

There is a total number of requests for prototyping. We constantly publish articles, take part in events and activities on the corporate portal, talking about prototyping, specifics and business opportunities. Based on the results, we monitor the dynamics of the number of requests for prototyping.

We also regularly communicate with customers (“word of mouth”) who have already implemented projects with us about the possibility of working on new initiatives and/or with related business units where testing hypotheses through prototyping is needed. This approach also contributes to the number of requests.

2) Funnel of ideas.

We regularly record the number of ideas received (requests for prototyping). Some ideas are not implemented for various reasons: from colleagues’ lack of awareness that similar functionality has already been implemented in other products/systems, to the impossibility/expensiveness/inexpediency of implementing both a prototype and a future target solution.

The number of prototypes in work can be compared with the metric at the moment, that is, the real number of prototypes that can be taken into work at a given time, as a kind of “throughput” of the team. Some prototypes are in the queue (team backlog, prototypes on which work will begin as resources are released).

We simultaneously have several prototypes in the works at different stages: from initial acquaintance with the customer’s team and studying business requirements, to the final stages of acceptance of the result of the work performed by the customer and the formation of a backlog for subsequent implementation in the target product.

One of my main tasks is to plan all stages of projects in progress and systematically distribute them to team members without overload and with the opportunity to completely immerse themselves in a certain stage, while maximizing the number of implemented prototypes.

The classification of implemented prototypes is divided into steps:

A. The hypothesis was confirmed, the decision to create the target product/system.

b. The hypothesis was confirmed, the decision to transfer the developments to an existing product/system.

V. The hypothesis was confirmed, the prototype is self-sufficient and answered a number of business questions, no development is required.

d. The hypothesis was not confirmed, development is not required.

3) Return on investment.

Price:

There is a general payroll for the team and an additional cost for the team’s activities, which makes it possible to calculate the actual cost of a department specialist’s man-hour. We keep statistics on labor costs for each project: minimum, maximum and average values ​​of labor costs per project in man-hours. From these data, accordingly, we obtain the cost of prototypes for the company.

Effects:

Each project is assessed for the presence of effects for the company (financial, operational, cultural) from implementing a solution based on the implemented prototype. In the case of financial effects, we consider the effectiveness of the team (over the last 3 years, the team showed an average annual result of x2.5 of the cost).

4) Labor costs.

We record the duration of each stage of the project: from responding to a prototyping request to making a decision on the implemented prototype. We collect statistics on the types of work on the project: analytics, development, testing, project management.

5) Customer satisfaction.

We collect on a five-point scale for various points of interaction with the customer with comments on each point.

As a result, over more than three years of introducing the practice of prototyping, we have implemented more than 50 prototypes. The conversion rate into successful cases was 33%, which became a separate digital product or prototype developments were implemented into existing products.

Examples of projects

1. Prompt receipt of car characteristics (Telegram bot) – a tool for quickly searching for the necessary information about a car by its number without resorting to specialized systems (dislocation, equipment, repairs, invoice, etc.) when away from the office. Reducing the downtime of cars by speeding up the inspection of cars by increasing the accessibility and convenience of obtaining information on the characteristics of cars from a mobile phone.

2. Optimization of travel routes – a service for comparing historical travel options in the direction between selected stations to identify the most optimal (profitable) travel routes, assessing the capacity of sections, providing recommendations for accelerating the turnover of cars and reducing costs for travel tariffs.

3. Compensation for damage – a service for identifying and registering all facts of dismantling of equipment during repairs of rolling stock, repairs of parts, facts of substitution (theft) and damage to the axles of wheel sets. Reduces damage compensation time and reduces labor costs.