Test case management systems. Which one to choose for immediate work?
Secondly: in the system, most likely you will not work alone, you have a leader (team lead), maybe several testers (junes), how will you build the processes of interaction with the team? You need a system that can help keep track of who, when, and what did or edited, and, preferably, a system for providing visual reports – what is the state of checks at the current time? And also do not forget about testing planning (wound test).
Well, the third is the price. Usually it consists of the amount of bucks for 1 person-month.
Not fit 100%
TestLink. Pros of open source software. Disadvantages: morally and physically outdated; during installation, difficulties may arise, complex or impossible integration with Jira.
TXT, DOC, XLSX. Pros: Free, you can immediately plan work, you can create draft test cases, use it as an archive or templates. Cons: no integration, no control and time tracking.
Testia Tarantula. Pros: Free. Cons: poor reporting, no integration, no customization.
Qtest. Trial version only on request.
TestLodge. There will be a separate paragraph, see below
YouTube and article favorites: TestRail, Zephyr for Jira, QASE
After preliminary selection, the choice was made on these three programs. Personal opinion: at first glance, because I initially installed TestRail, it seems to me “more familiar”, but given the further development and the need for integration capabilities, Zephyr for Jira may be more profitable, you also cannot ignore such a free tool as QASE, free (up to 3 users), intuitive interface , detailed report, milestones, and generally in no way inferior to the above systems.
Now let’s take a closer look at each one.
Zephyr for Jira
To begin with, Marshmallow is not a standalone program, it is a plugin for Jira. Therefore, if you are not using Jira, it probably won’t work for you right away.
Separately from the minuses, I note that watching a review on YouTube without knowing English will not work, so I watched tutorial on this channel, and he is in “Indian” English.
Well, a lot of things need to be done to make something work. Good integration, beautiful colorful reports, but not made for people, while you fill in all the required fields to create a test case, and then the same number, or rather, even more fields, to run the test runs, it takes a lot of time, and the fields are required.
In short, I watched 90 minutes of the video tutorial, in which the wound test was never started. Complicated test case writing system. – this is both good and bad at the same time.
Because the tool is very powerful, and it requires a lot of time and attention. The approach to reviews takes into account all the time spent on writing a test case and includes: Review Story, Write Test Case, Execute Test Case, everything happens in the form of tusks and jugglers on the Scrum panel in sprint format.
Honestly, I idolized this system for a very long time. reviews are everywhere positive, so they are enthusiastically compared with Zephyr For Jira on many resources, incl. in articles on HABR. Well, since it is so much praised, it means you need to take a test period and try to fill the repository, and assign yourself test runs. But only when compiling test cases, I immediately found an irreparable minus, which changed my view of TestRail – you cannot mark the passed step. Another disadvantage is the test period, it seems to me very short. For about a month now I have been choosing a tool for storing cases – and to be honest, this is a rather difficult task, since you need to try all or almost all systems and give an objective answer why you made such a choice.
In general, TestRail has a fairly large list of integrations: Slack, Jira, GitHub, etc. The tool is powerful enough, but, unfortunately, it did not suit me.
And what didn’t TestLodge please?
Well, with Lodge, it’s very interesting, after considering TestRail and QASE – Lodge seems to be very unfinished, moreover, I will tell you that there may not be more functionality in NotePad, but Excel can surpass 100% of Lodge, because … in the cases I have compiled in Excel – it is possible to set the resolution in front of each step and whether each step passes or fails, for example, I implemented this feature by filling in different colors – Green – passed, Red – Failed, Purple – Blocked, etc. To. the number of colors is not limited.
Let’s list the cons of TestLodge:
Entering Steps in Solid Text
Naturally, we cannot leave the resolution at every step whether it has been passed or passed, or even here the BAG
Only one expected result is specified when going through all the steps
The resolution for passing the case is described by only three scenarios – PASS, FAIL, SKIP
Suddenly, after a long wandering through the resources of Internet video articles on YouTube, I accidentally found an inconspicuous QASE system in one of the articles on the 8th place. It is surprising that such a good (in my opinion) system was underestimated and there is so little information about it. The biggest plus, in addition to its freeness, is the ability to leave resolutions with comments at each step of the test case. Let’s arrange the pluses of the system in the form of a list, so it will be clearer.
For each step there is a field for input data, as well as the expected result, which allows us to make a wider coverage of test cases (include micro-scenarios for previously entered errors), as well as describe in more detail each step, and most importantly, that we are from it expecting.
For each step, you can select one of 4 states (Screen 1): Pass, Fail, Skip, Block – this allows the test case to be more flexible, it is possible to include “exotic scenarios” (without fanaticism) based on and identified in the background earlier mistakes made. From my experience in Yandex as an assessor-tester, I can say that this is a terribly convenient functionality, since the step at which the discrepancy is revealed is immediately entered into the bug report, as well as the environment, the stand, and the steps previously taken, you just have to come up with and write the correct title.
For each step, you can write a comment.
Each test case can have 5 states (Screen 2): Passed, Failed, Blocked, Skiped, Invalid (this is rare, but it happens, When there is a test case, but it is broken or not up to date).
The program is free (up to 3 users).
There is a possibility of customization.
There are quite a lot of parameters for each test case (for behavior, for flakes, for automation), you can also add your own parameter that will create an additional run (for example: we check on MacOS, the second on Android, the third on Win).
Search for keywords that you yourself will enter earlier.
1. Independent system overview
2. 10 best test case management systems in 2019
3. Reading a long article is difficult … Testing a complex product is easy
4. Best Test Management Systems 2019
5. Testing. Test management tools
6. TestRail is a test management system. Review, practice | Online test case management tool
7. Qase Test Management System Overview
eight. Video 11. How to create a test suite (Test Suite). How to create a test case (Test Case). TestLink program
nine. Video 27. How to install Testlink.
ten. Test Day Atlassian 2016 – Masha Shinkareva (Yandex.Money) – Pros and cons of Zephyr for JIRA
eleven. Jira – Zephyr Plugin Tutorial Part 1
12. QASE is a test management system. Review, practice | Online test case management tool