Adding pairwise (pairwise testing) to your QA engineer’s arsenal

A QA engineer has an impressive arsenal of test design techniques: equivalence classes, bounds, pairwise testing, state and transition diagrams, decision tables, use cases and alternative scenarios, combinations, decision trees…umm, that’s all I remember if I forgot something, correct it.

So, use cases and alternative scenarios, we usually get from analysts from the specification .. few people draw tables, trees and diagrams, since it takes a lot of time (with a shortage of resources). As a rule, two popular techniques are in use: equivalence classes and boundary values, and only a few smarties use pairwise (pairwise testing).

To be honest, I was not able to accept this technique right away. At that time, I was not yet completely filled with knowledge, and due to misunderstanding, there was distrust in the cases generated by the tool.

Of course, at social services, everyone bounces off their teeth: “all possible separate combinations of each pair of input parameters are created” or “in 97 percent of cases, bugs are on a combination of two parameters”, but let’s see why pairwise is difficult to apply with handles and understand how to score and apply it successfully in your work.

Pairwise testing with handles

Imagine that we are testing a site that sells notebooks. The user can assemble the notebook himself, through the form, after filling which we see the image of the assembled notebook. Form:

This is what our notebook designer looks like. Do not look at the adequacy and design, the essence is important =)

This is what our notebook designer looks like. Do not look at the adequacy and design, the essence is important =)

First, let’s write out our parameters with their values:

If we use the usual combinatorics, without optimization, we will get 4*8*5*5*2*4*11*2 = 140800 cases to cover all the functionality 100%. I got this figure by multiplying the number of values ​​of each parameter. I have not yet calculated the colors of the pages when selecting the “Multi-colored pages” checkbox … to be honest, I’m not sure here: in theory, there is a multiple choice, which means that each color is not a value, but a parameter, and it would be more correct to make such a sign:

Adjusted, it turns out… erm, 4*8*5*5*2*4*11*2*2^11 = 288,358,400 cases. A lot is expected, we optimize:

  • Format
    From the point of view of analytics, A3 and A6 are rarely ordered formats, let’s check the most popular ones: A4 and A5;

  • Quantity pages
    We apply our favorite equivalence classes and boundary conditions: 30, 50, 500, 1000;

  • Density paper
    Yes .. there are also numbers, but we will divide into classes according to technical information: 40 g / m2 and 70 g / m2 are included in one class, 80 g / m2 and 100 g / m2 in another, and 120 g / m2 in the third . Result: 40, 80, 120;

  • markup And Upholstery
    The values ​​are unique, there’s nothing you can do about it..

  • Color pages
    And here is an interesting point. Nothing depends on the color of the pages, they are not unique. Of course, by combining colored paper into one equivalence class, we will take, for example: “Black”, “Red”. And “White” will be singled out separately, since this is a special value determined by the subject area;

  • multicolored pages
    Eh .. here I feel you are throwing a plop on me in the comments. Basically my logic is this:
    Since we have determined that a color is not something unique, and in this context, even “White” is not a special value (we checked it earlier), I will simply operate on the values ​​of the possible chosen colors, putting it in one equivalence class. As a result, we get: 1.2, 10, 11;

Here’s how our sign has changed:

Now, the number of possible combinations is: 2*4*3*5*2*4*3*2*4 = 23040, which is already less than 140800 cases. And now we apply pairwise testing with handles:

Step 1: We arrange the parameters from the largest number of values ​​to the smallest and fill them with values, trying to ensure that pairs of values ​​from different parameters occur at least once:

Table start

Table start

Table continuation

Table continuation

Step 2: Cells marked in yellow are any parameter values ​​- you can insert any values ​​here:

Table start

Table start

Table continuation

Table continuation

We got 36 checks. I will not claim that I did everything without errors, but it seems like I found most of the pairs of all values ​​…

Step 3: But you must admit that in real cases there is no such thing that there would be no business logic.

  1. If the number of pages >= 200, then we cannot use paper with a weight >= 70 g/m2

  2. If the number of pages >= 500, then we cannot use paper with a weight >= 40 g/m2

  3. If the number of pages >= 500 and <= 50, then we cannot use "Wood" upholstery

  4. If pagination is enabled, then we cannot use the paper color “Black”

  5. If the format is A4, then the number of pages cannot be >= 200

  6. If the number of pages <= 100 or the format >= A4, then we cannot use the “Colorful Pages” option

  7. If the “Colorful Pages” option is enabled, then the “Page Color” selector is not available.

  8. If the “Multi-colored pages” option is disabled, then the choice of colors in “Multi-color” is not available.

Taking into account the business requirements listed above, all our cases turned out to be irrelevant (( Honestly, I went through each condition and filtered out the cases from the table:

Table start

Table start

Table continuation

Table continuation

Actually, that’s all =) At this stage, we figured out why pairwise is difficult to apply with pens: it is difficult, long and ineffectual in most cases.

But like I said, fuck it! We stomp here and download PICT (pict.exe) for Windows, and if you have Unix, then read here .

Leveraging the power of PICT

Step 1: Open a command prompt in the directory where pict.exe is located and run it:

Step 2: Create a text file with the content:

# Параметры и их значения
#
Page format: A5, A4, A3, A6
Page count: 30, 50, 500, 1000
Paper density: 40, 80, 120
Page markup: without, incline, cell, line, music
Page numeric: yes, no
Page color: white, black|red|blue|green|yellow|salad|violet|pink|dark blue, NotUsed
Different color pages: yes, no
Different color pages combinations: 1, 2, 3|4|5|6|7|8|9, 10, 11 , NotUsed
Note cover: skin, wood, dermantin, textile 

# Условия
#
# Если включена опция "Разноцветные страницы", то селект "Цвет страниц" не доступен
# Если выключена опция "Разноцветные страницы", то выбор цветов в "Разноцветность" не доступна
if [Different color pages] = "yes" then [Page color] = "NotUsed";
if [Different color pages] = "no" then [Page color] <> "NotUsed";
if [Different color pages] = "no" then [Different color pages combinations] = "NotUsed";
if [Different color pages] = "yes" then [Different color pages combinations] <> "NotUsed";

# Если нумерация страниц включена, то мы не можем использовать цвет бумаги "Черный"
if [Page numeric] = "yes" then [Page color] <> "black";

# Если формат А4, то количество страниц не может быть >= 200
if [Page format] = "A4" then [Page count] <= 200;

# Если количество страниц >= 200, то мы не можем использовать бумагу плотностью >= 70 г/м2
if [Page count] >= 200 then [Paper density] >= 70;

# Если количество страниц >= 500, то мы не можем использовать бумагу плотностью >= 40 г/м2
if [Page count] >= 500 then [Paper density] >= 40;

# Если количество страниц >= 500 и <= 50, то мы не можем использовать обивку "Дерево"
if ( [Page count] >= 500 and [Page count] <= 50 ) then [Note cover] <> "wood";

# Если количество страниц <= 100 или формат >= A4, то мы не можем использовать опцию "Разноцветные страницы"
if ( [Page count] <= 100 or [Page format] in {"A4", "A3"} ) then [Different color pages] = "no";

Before feeding the PICT file, let’s discuss the nuances. Pict syntax is intuitive, but I want to clarify some points:

  1. This form of entry is: black|red|blue|green|yellow|salad|violet|pink|dark blue denotes that this is an equivalence class with enumerated values. Instead of using one value from the equivalence class in each test, as we did originally, PICT will randomly put down different values ​​from the given equivalence class in the tests. So we will make our tests more interesting, more diverse and, in addition, we will cover unnecessary values ​​.. that’s cool)

  2. Keyword NotUsed necessary in order to implement the opposite conditions in tests. The nuance is that it is always necessary to explicitly prescribe the conditions when the parameter is equal to (=) and not equal to (<>) to the value NotUsed.

  3. Also, PICT supports the condition of entering the range of values: in {"A4", "A3"}

  4. Other PICT features are well described. here.

Step 3: Let’s feed our PICT file and get the generated list of tests:

Generating tests with the PICT tool

Generating tests with the PICT tool

For more convenient work with the resulting cases, it is better to direct the output of the tool to an .xls file: pict note_tests.txt > note_tests.xls .

As a result, we get a beautiful list of checks:

Beginning of file

Beginning of file

File continuation

File continuation

PICT generated 43 checks for us. And I spent literally 30 minutes compiling a text file, while with pens I made a table for 2 hours .. if not longer.

Of course, there are much more complex and confusing conditions in the work, the description of which will take more time, but I think the capabilities of PICT will be enough for you to cover 90% of the tasks.

A few words about negative scenarios

Yes, PICT can help you with this as well. To specify a negative value for the parameter, use the prefix "~"

Step 1: In our file with the description of parameters and conditions, add negative values ​​to the “Parameters and their values” block (Leave the conditions unchanged):

# Параметры и их значения
#
Page format: A5, A4, A3, A6, ~NotSelected
Page count: 30, 50, 500, 1000, ~0
Paper density: 40, 80, 120, ~0
Page markup: without, incline, cell, line, music, ~NotSelected
Page numeric: yes, no
Page color: white, black|red|blue|green|yellow|salad|violet|pink|dark blue, NotUsed
Different color pages: yes, no
Different color pages combinations: 1, 2, 3|4|5|6|7|8|9, 10, 11 , NotUsed, ~0
Note cover: skin, wood, dermantin, textile, ~NotSelected

Where there are string parameters, I think it is logical to designate a negative value as ~NotSelectedand where numeric ~0

Step 2: We give our file for processing a pict: pict with_negative_note_tests.txt > with_negative_note_tests.xls and get the result:

Beginning of file

Beginning of file

File continuation

File continuation

End of file

End of file

In total we have 93 tests with positive and negative scenarios! Yes, it’s still too much, but this is where prioritization comes into play =)

I also want to point out a couple of interesting points when working with PICT:

Regression with PICT (Seeding)

Each time PICT generates new combinations of scenarios and, accordingly, a new set of tests. Sometimes this is not convenient for us, since we want to regress according to previously created scenarios, simply by adding some new condition or value.

In order for PICT to understand our wishlist and work with the old set of cases, you need to point it to a file with previously generated tests, on the basis of which a new set will be generated:
pict with_negative_note_tests.txt > with_negative_note_tests.xls /e:with_negative_note_tests_seed.txt

Where /e:with_negative_note_tests_seed.txt is an indication of a file with basic scripts, which can be done by converting an .xls file with tests to .txt format.

PICT will try to reuse old test cases as much as possible with new changes!

You can read more about seeding here.

Reducing the number of tests with PICT

While Pict has cut our testing time short, some may not find it enough. For such greedy people, there is an opportunity to further reduce the number of cases, not much to the detriment of coverage.

Earlier I said that PICT generates new test combinations every time and the generation process is highly dependent on the initial conditions. Nevertheless, each created set is guaranteed to cover all the necessary combinations, but Pict forms some combinations more efficiently.

Knowing this information, we can use the option /r , which will allow Pict to minimize the number of tests without losing test coverage. Run the tool with this flag several times and select the one with the minimum number of tests.

I ran for a test set with negative and positive tests, where there were originally 93 tests:

pict with_negative_note_tests.txt > with_negative_note_tests.xls /r

The result of the attempts is as follows: 93, 97, 99, 94, 98, 95 … It turns out that in my case, the very first attempt was optimal.

But when I ran a seed for a test set with positive cases, where there were originally 43 tests:

pict note_tests.txt > note_tests.xls /r

That result was the following: 42 42 43 42. Thus, we reduced our set by 1 case =) Well…also the result.

You can read about other usefulness of the PICT tool in PICT User’s Guide And microsoft/pict/doc .

And remember, PICT is not afraid of a lot of parameters, it is afraid of a lot of values! So be sure to optimize this point before using pairwise testing.

I hope you, like me, were interested in analyzing this technique in more detail and you will definitely include it in your QA engineer’s arsenal.

This article is the first for me) It was written as part of my personal blog about testing and QA: https://t.me/qanva_blog

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *