This blog is all about thinking about the past, present, and future in testing. As much as I want to see everything clearly, my crystal ball is rather dim. However, learning is necessary, and here is my tool for that.
Difference between test idea and test case?
Year after year, changing organization one after another, I come to a new venture with the anticipation that I will not have to see test cases other than those that are automated, and through continuous execution keep constant updating.
And yet, year after year, as I move from one organization to another, I learn that people are still writing test cases. These are the test cases that contain the title and the steps to be taken. Those that define the sequence of actions in the application that need to be checked and the steps that you may or may not follow because you are not a robot.
From my point of view, ideas are of no value, and we care little how well they are documented. Ideas written on pieces of paper are often difficult for me to decipher after a month, but these are critical notes and I really need them when I return to the pre-structured information that I previously documented. Test cases are what we might want to leave for later, they are more than ideas. They have a structure that keeps them running, even if they look like a checklist. They often include steps and ideas for ordering. It is better to consider test cases as the result of testing (moreover, automated!), And not as initial data for testing.
The point is that some of our failed experiences will never be published because we cannot talk about them while we are still in the process. Inspired by today’s conversations, I return to an experience that I could not tell when it happened, but I can today, with examples.
I worked for a product / consulting company and the consulting side I represented worked pretty well – so good that it was difficult to make room for any of our testing experts to test our own product. Consulting paid much better. Management pulled out one of us consultants to test Release 1, another to test Release 2, and so on. In the end, I was next in line.
Thinking back 20 years later, it’s funny to realize that I already introduced myself as a testing expert. By testing the product, I set out to do it well – as we all do. I asked what those before me were doing and was pointed out to the test management tool by proud colleagues who made sure to document their testing.
Here are real-world examples of what I got.
They were considered test cases. They’re far worse than some of the best test cases I’ve seen over the years, but even the best ones have the general stigma of being useless for good testing.
These test cases were terrible. They are terrible even now. Time hasn’t made them better, only worse. They were step-by-step instructions in which a lot of effort went into simulating testing by carefully describing the same steps only so that the tester using them knows that he, for example, did box testing from top to bottom, right- left. Some magical meanings have been defined in them that completely miss the point of why these values were chosen, but I can only guess that the choice of data in them reflected names in accordance with the concept, and not names for the likelihood of finding problems or even indicating ideas. where there might be problems.
During my work, the first thing I did was give up all work. These were the work of my esteemed colleagues, and they did the best they could think of in that situation. I could only hope that they reflected the test cases and not the testing that was done.
I transferred testing to the research level. No more test cases. The most we could get was a checklist of features and ideas after all of this was explored and structured through numerous rounds of rehearsals through actual testing.
While what I encounter in organizations these days is usually not so bad, it is not much better either. I have argued over and over that dropping test cases improves testing. A controlled experiment over four weeks in which we used pre-designed cases on two occasions, finding zero problems, and using free exploratory testing with prepared test data on two other cases, finding all problems was made in order to report the results of that user testing. This was essentially the deletion of 39 pages with 46 “test cases”, where I did not even know 3 pieces of information, having joined the new company in the first week. In other cases, when I did not make public presentations, after years I can find the numbers that I have shared.
I wish the world was ready for quality testing, but this is still not the case. Automation and working on better and better testing ideas seems to be our main hope. And I’m glad test automation is just a sane way of documenting testing.
Translation of the article prepared as part of the course “QA Engineer”.
Join the webinar “Development Methodologies”where we will look at development methodologies scrum, kanban, waterfall and explore the role of testing in the development process.