Test design in test automation

Author of the article: Pavel Balakhanov, over 13 years in testing automation

In this article, I want to touch on an important topic — test design in test automation. Why is this topic important? Because the quality of autotests and test automation tools directly depends on the test design that will be embedded in your test automation tool. But before moving on to the design of autotests and test automation tools, I suggest considering the main indicators of autotest quality:

1. Stability

2. Speed

3. Price

Speed

In my opinion, this is a very important indicator of high-quality automated tests, which many teams simply neglect. Most often, automated tests are launched in test stand assembly pipelines, and when the stand is assembled at night, you want to get the results of testing the deployed release in the morning at the beginning of the working day. I think few people want to launch automated tests and wait for them to pass and report the result after 1-2 working days. Therefore, it is extremely important to have fast tests. How is this achieved? In fact, the most popular ways to get fast test runs are the following:

• Reducing the pool with autotests that is launched after the stand assembly. In this method, we can identify the main scenarios with which it will be possible to conduct smoke testing of all the main components of the deployed software product and run only these scenarios on a regular basis. On the one hand, we will speed up the time of running autotests by reducing the pool, but on the other hand, we will not be able to guarantee the completeness of testing that can be achieved when running the pool with all autotests.

• Test parallelization. In my opinion, one of the main ways to speed up autotests. There are different options for parallelizing autotests: thread side, server side, ci/cd side. But by building an effective parallelization model, you can achieve fantastic results and speed up autotest runs many times over.

• Profiling the autotest code. This is also the main way to speed up autotests. It involves reviewing the autotest code and removing code that can obviously slow down the autotest run.

Stability

It is also an important indicator of the quality of automated tests and shows how stable the automated tests are from run to run on the same configuration of the product deployed on the stand.

Price

This indicator is related to the stability of automated tests and shows how many resources are needed to cover scenarios with automated tests and adapt automated tests to changes in the product.

These indicators of the quality of automated tests are influenced by the test design of automation testing tools, which we will consider below.

In testing automation, there are 2 main test designs:

Two-level

Today, this type of test design is often used and is the main one when designing testing tools and automated tests. It looks like this:

In this image, two levels can be distinguished:

• Level of interaction. At this level, the resources for interaction with SUT are described depending on the type of interaction and types of autotests.

• Test level. At this level are scenarios in which interaction resource objects are created via DI. At this level are located only tests, which are methods with a set of calls to interaction resource methods.

This design is simple, cheap and allows you to build automated tests on any project of any complexity, which will have the basic quality criteria described above.

Three-level

This type of test design is also popular in certain automated testing teams. It looks like this:

Unlike the previous test design, a third layer appears here, which is a kind of adapter between the interaction resource layer and the test layer. This test design is often used in BDD. At first glance, it seems that we can describe scenarios at the test level using calls to business methods that are described on the steps layer – and autotests will become more readable by non-technical specialists. But on the other hand, the steps layer makes automation tools and autotests more expensive. And the expensiveness is that this layer must be written and constantly supported for changes in the product, which can significantly increase the cost of automation as a whole.

To sum it up: in this article we have considered the main types of test designs, as well as the pros and cons of one or another solution. In fact, the choice of test design is greatly influenced by the project, the agreements in the team and the processes that have been created and integrated or are planned to be created and integrated. But the right choice of test design in the future will allow you to write high-quality automation tools and high-quality autotests.

Finally, I invite testers to the upcoming open lessons on automation:

• August 13: API Test Automation with Rest Assured. Let's look at a popular tool for API testing automation, Rest Assured, and methods for writing API autotests on it. Sign up via the link

• August 20: Stream API and Functional Interfaces in Java. Let's look at processing collection elements using the Stream API and functional interfaces when writing automated tests. Sign up via the link