I communicated with dozens of QA-engineers from different companies and each of them talked about the fact that they use different systems and tools for bug tracking. We also tried several of them and I decided to share the solution to which we came.
I will be banal. Errors appear and are detected at various stages of the development process. Therefore, it is possible to divide bugs into categories, depending on the time of their detection:
- Deficiencies . These are the mistakes that the developers made while they sawed the new functionality. Such errors are found in research or acceptance testing of new features on developer team stands.
- Regression bugs . These are defects that find manual regression tests or automatic UI and API tests on the bench for code integration.
- Sales issues . These are problems that were found by employees or customers and turned to technical support.
How we started, or Jira
Two years ago, we had a dedicated team of testers who manually tested the product after integrating the code of all the teams. Up to this point, the code was checked by developers at developer stands. Errors that testers found were backed up in Jira. Bugs were stored in the general backlog and moved from sprint to sprint with other tasks. Each sprint two or three bugs got and repaired, but the majority remained at the bottom of the backlog:
- One of the reasons that bugs are accumulating in the backlog is that they do not interfere with users. Such bugs have low priority and will not be repaired.
- Also, if the company does not have clear and understandable rules for creating bugs, testers can add the same problem several times because they could not find in the list the already added bug report.
- Another reason is that inexperienced testers are involved in the project. Error novice testers, to bring in bug tracking all the bugs found during operation. Inexperienced testers consider testing to be a bug, not providing information about the quality of the product and preventing the appearance of defects.
I will give a simple example. When constructing reports in the date entry fields, today's date is substituted by default. If you change the date in the pop-up, you can again select the current day and the date entry field will be cleared.
I have suspicions that for all the time of our network, no one, except testers, did not reproduce this error. This kind of error makes up the majority of bugs that are not fixed.
With this approach, when all the found bugs are logged, some of them are duplicated and most of these bugs are not fixed problems:
- Testers are demotivated, because the errors that they find are not corrected by the developers. It seems that the work does not make sense.
- It’s difficult for a product owner to manage backups with a lot of bugs.
Goodbye Jira, long live Kaiten
In the spring of 2018, we abandoned Jira and moved to Kaiten. The change of tools was caused by the reasons about which Andrey Arefyev wrote in the article " Why Dodo Pizza began to use Kaiten instead of a bunch of Trello and Jira ".
After switching to Kaiten, our approach to working with bugs did not change:
- Deficiencies were put on the command boards and the developers decided on their own whether to repair them or not.
- The bugs that were found in regress (it was carried out by a dedicated team of testers) were repaired in the release branch and were not released until all problems were fixed.We decided that it was more logical to keep and collect information about these problems in the testers channel in Slack. Testers wrote a message that contained a legend, a list of bugs with logs and the names of the developers who took the task to work. With the help of emodzhi, the status was changed, and in the trades they discussed, applied screenshots, synchronized. Testers arranged this format. Some developers didn’t like this method, because in the chat there was a parallel correspondence and this message went up and was not visible. We secured it, but it didn’t simplify life much.
- The bugs that were found on the prod were put into the backlog, the Product Owner, set priorities and chose the ones we’ll fix.
Time of experiments, or not
We decided to experiment with the formats and created a separate board in Kaiten, on which we stored bugs and changed statuses. We have simplified the establishment of a bug report to spend less time. When adding a card to Kaiten, testers tagged developers. This notification was sent to them by mail. We brought this board to the monitor, which was hanging in the aisle next to our workplace, so that the developers could see the progress and the testing process became transparent. This practice also did not take root, because the main channel of communication is Slack. Our developers do not check mail often. Therefore, this solution quickly stopped working and we again returned to Slack.
After an unsuccessful experiment with a board in Kaiten, some developers were still against the format with a message in Slack. And we began to think how to track the bugs in the slug and solve the problems that prevented the developers. As a result, the search came across an application for Slack, Workast, which helps to organize work with tudushki directly in the messenger. We thought that this application would allow managing the process of working with bugs more flexibly. This application had its advantages: the change of statuses and the assignment to the developers at the touch of a button, completed tasks were hidden and the message did not grow to an enormous size.
This was the way the solved tasks looked in the Todo application and the requests to return the “ants”. After the end of the trial period of the Workast application, we decided to refuse it. Because using this application, we came to the same thing that was during the time when we used Jira. There remained some bugs that wandered in this list from regress to regress. And with each iteration they became more. It might have been worthwhile to refine the process of working with this extension, to buy it and use it, but we did not do that.
Our ideal way to work with bugs now
At the end of 2018 - the beginning of 2019 a number of changes occurred in our company, which influenced the process of working with bugs.
First, we did not have a dedicated team of testers. All testers went into teams of developers, to enhance the competence of testing teams. Because of this, we began to find errors earlier, before the integration of the command code.
Secondly, we abandoned manual regression testing in favor of automatic.
Third, we adopted the “zero bug policy”. In the article " # zerobugpolicy or how we fix bugs
tells in detail how we choose bugs that are fixed.
Today, the process of working with bugs is as follows:
- Deficiencies .Testers report a problem to analysts or product managers. They go with their feet, show, reproduce, explain how it will affect customers and decide whether it needs to be repaired before the release or can be repaired later, or maybe even not worth repairing it at all.
- Regression bugs that found auto tests are being repaired by the team that touched this part of the system and we will not release this code until we solve the problem. Testers fix these bugs in any format in the release channel on Slack.
- Sales issues . Such bugs go directly to task owners. Analysts drive errors according to the Priority Matrix of the bug and add to the backlog or fix it in their own, accumulating statistics on complaints on this issue.
In short, we basically refused the bug-tracking system
. With this approach to working with bugs, we decided a few pains:
- Testers do not get upset because errors that they find and get bug-tracking are not corrected.
- Testers do not waste time on the institution and a complete description of the bugs that even no one will read.
- PO is easier to manage backlog, in which there is no dead weight.
I do not want to say that tracking bugs is useless. The bugs that we take to work are tracked like any other task. But the bug-tracking system is not a mandatory attribute of testing. It does not need to be used only because most companies use it and so it is accepted in the industry. You need to “think with your head” and try on tools for your processes and needs. For us, it is ideal to work without a bug-tracking system now. For half a year of such work, we never thought about going back to it and getting all the bugs there again.
And how does the process of working with bugs work in your company?