Choosing the appropriate bug tracking

I communicated with dozens of QA engineers from different companies and each of them talked about the fact that they use different systems and tools for bug tracking. We also tried several of them and I decided to share the solution to which we came.

Intro

I will be banal. Errors appear and are detected at various stages of the development process. Therefore, it is possible to divide bugs into categories, depending on the time of their detection:

• Deficiencies. These are the mistakes that the developers made while they sawed the new functionality. Such errors are found during research or acceptance testing of new features on developer team stands.
• Recourse bugs. These are defects that find manual regression tests or automatic UI and API tests on the bench for code integration.
• Bugs with sell. These are problems that were found by employees or customers and turned to technical support.

Where did we start, or Jira

Two years ago, we had a dedicated team of testers who manually tested the product after integrating the code of all the teams. Up to this point, the code was checked by developers at developer stands. Errors that testers found were backed up in Jira. Bugs were stored in the general backlog and moved from sprint to sprint with other tasks. Each sprint two or three bugs got and repaired, but the majority remained at the bottom of the backlog:

1. One of the reasons why bugs are accumulating in the backlog is that they do not interfere with users. Such bugs have low priority and will not be repaired.
2. Also, if the company does not have clear and understandable rules for creating bugs, testers can add the same problem several times, because they could not find in the list the already added bug report.
3. Another reason may be that inexperienced testers are involved in the project. Error novice testers, to bring in bug tracking all the bugs found during operation. Inexperienced testers consider testing to be a bug, not providing information about the quality of the product and preventing the appearance of defects.

I will give a simple example. When constructing reports in the date entry fields, today's date is substituted by default. If you change the date in the pop-up, you can again select the current day and the date entry field will be cleared.

I have suspicions that for all the time of our network, no one, except testers, did not reproduce this error. Such errors make up the majority of bugs that are not fixed.

With this approach, when all the found bugs are recorded, some of them are duplicated and most of these bugs are not fixed problems:

• Testers are demotivated, as the errors that they find are not corrected by the developers. One gets the feeling that the work does not make sense.
• The owner of the product is difficult to manage backlog with a lot of bugs.

Goodbye Jira, long live Kaiten

In the spring of 2018, we abandoned Jira and moved to Kaiten. The change of tools was caused by the reasons about which Andrei Arefyev wrote in the article “Why Dodo Pizza began to use Kaiten instead of the bundle Trello and Jira”. After switching to Kaiten, our approach to working with bugs has not changed:

• The shortcomings were entered on the command boards and the developers decided to repair them themselves or not.
• The bugs that were found in the regress (it was performed by a dedicated team of testers) were repaired in the release branch and not released, until all the problems were fixed. We decided that it was more logical to keep and collect information about these problems in the testers channel in Slack. Testers wrote a message that contained a legend, a list of bugs with logs and the names of the developers who took the task to work. With the help of emodzhi, the status was changed, and in the trades they discussed, applied screenshots, synchronized. Testers arranged this format. Some developers didn’t like this method, because in the chat there was a parallel correspondence and this message went up and was not visible. We secured it, but it didn’t greatly simplify life.

• The bugs that were found on the prod were entered into the backlog, the Product Owner, set priorities and chose the ones that we would fix.

Experiment time or not

We decided to experiment with the formats and created a separate board in Kaiten, on which we stored bugs and changed statuses. We have simplified the establishment of a bug report to spend less time. When adding a card to Kaiten, testers tagged developers. This notification was sent to them by mail. We brought this board to the monitor, which was hanging in the aisle next to our workplace, so that the developers could see the progress and the testing process became transparent. This practice also did not take root, because the main channel of communication is Slack. Our developers do not check mail frequently. Therefore, this solution quickly stopped working and we returned to Slack again.

Return ants

After an unsuccessful experiment with a board in Kaiten, some developers were still against the format with a message in Slack. And we began to think how to track the bugs in the slug and solve the problems that prevented the developers. As a result, the search came across an application for Slack, Workast, which helps to organize work with tudushki directly in the messenger. We thought that this application would allow managing the process of working with bugs more flexibly. This application had its advantages: the change of statuses and the assignment to the developers at the touch of a button, completed tasks were hidden and the message did not grow to enormous size.

This was the way the solved tasks looked in the Todo application and the requests to return the “ants”. After the end of the trial period of the Workast application, we decided to refuse it. Because using this application, we came to the same thing that was during the time when we used Jira. There remained some bugs that wandered in this list from regress to regress. And with each iteration they became more. It might have been worthwhile to refine the process of working with this extension, buy it and use it, but we did not.

Our perfect way to work with bugs now

At the end of 2018 – the beginning of 2019 a number of changes occurred in our company, which influenced the process of working with bugs.

First, we did not have a dedicated team of testers. All testers went into teams of developers to strengthen the competence of testing teams. Because of this, we began to find errors earlier, before the integration of the command code.

Secondly, we abandoned manual regression testing in favor of automatic.

Third, we adopted the “zero bug policy”. In the article "#zerobugpolicy or how we fix bugs," Anton Bevzyuk tells in detail how we select bugs that we fix.

Today, the process of working with bugs is as follows:

• Deficiencies. Testers report a problem to analysts or product managers. They go with their feet, show, reproduce, explain how it will affect customers and decide whether it needs to be repaired before the release or can be repaired later, or maybe even not worth repairing it at all.
• Recourse bugs, which found autotests, repair the team that touched this part of the system and we will not release this code until we solve the problem. Testers fix these bugs in any format in the release channel in Slack.
• Bugs with sell. Such bugs go directly to task owners. Analysts drive errors according to the Priority Matrix of the bug and add to the backlog, or fix it in themselves, accumulating statistics on complaints on this issue.

Results

In short, we basically refused the bug-tracking system.. With this approach to working with bugs, we decided a few pains:

• Testers are not upset due to the fact that the errors that they find and lead to bug tracking are not corrected.
• Testers do not spend time on the institution and a complete description of the bugs that even no one will read.
• PO is easier to manage backlog, in which there is no dead weight.

I do not want to say that tracking bugs is useless. The bugs that we take to work are tracked like any other task. But the bug-tracking system is not a mandatory attribute of testing. It does not need to be used only because most companies use it and so it is accepted in the industry. You need to “think with your head” and try on tools for your processes and needs. For us, it is ideal to work without a bug-tracking system now. For half a year of such work, we never thought about going back to it and getting all the bugs there again.

And how does the process of working with bugs work in your company?