Automated Testing with Pytest

We are living in an era when software is moving very quickly to the market. Because of this, the development process becomes very stressful. High rates of software implementation and fast delivery look like a good part of the business model, but the question arises here about how to deliver software of the right quality.

Why do we need automated tests

There are many advantages to automated testing, here are three main ones:
Reuse: There is no need to write new scripts every time, even when a new version of the operating system is released, unless there is an urgent need for it.
Reliability: People tend to make mistakes, and cars make them less likely. And they work faster when performing repeated steps / tests that need to be performed continuously.
24/7 work: You can start testing at any time, even remotely. If you start testing at night, it will run even while you sleep.

Developed full-featured pytest test tool in Python

Currently, there are many frameworks and tools for testing. There are different types of frameworks, for example, data-driven, keyword-driven, hybrid, BDD, etc. You can choose the one that best suits your requirements.

I must say that Python and pytest occupy a huge niche in this matter. Python and its related tools are widely used, probably because they are more accessible to people with little programming experience compared to other languages.

Framework pytest makes it easy to write small tests, but also scales to support sophisticated functional testing of applications and libraries.

A few key features pytest:

• Automatic detection of test modules and functions;
• Effective CLI to improve control over what you want to run or skip;
• Large third-party ecosystem of plugins;
• Fixtures – different types, different applications;
• Work with the traditional unit testing framework.

Automatic and configurable test detection

Default pytest expects to find tests in those Python modules whose names begin with test_ or end on _test.py. Also by default, it expects test function names to begin with a prefix test_. However, this test detection protocol can be changed by adding your own configuration to one of the configuration files. pytest.

# content of pytest.ini
# Example 1: have pytest look for "check" instead of "test"
# can also be defined in tox.ini or setup.cfg file, although the section
# name in setup.cfg files should be "tool: pytest"
[pytest]
python_files = check _ *. py
python_classes = Check
python_functions = * _check

Let's look at a very simple test function:

class CheckClass (object):
    def one_check (self):
        x = "this"
        assert 'h' in x

    def two_check (self):
        x = "hello"
        assert hasattr (x, 'check')

Have you noticed anything? There are no assertEqual or assertDictEqualsimply accessible and understandable assert. There is no need to import these functions to simply compare two objects. Assert is what Python already has and there is no need to reinvent the wheel.

Template code? Do not worry, fixtures are in a hurry to help!

Look at the test functions that test the basic operations in the Wallet program:

// test_wallet.py
from wallet import Wallet
def test_default_initial_amount ():
   wallet = Wallet ()
   assert wallet.balance == 0
   wallet.close ()
def test_setting_initial_amount ():
   wallet = Wallet (initial_amount = 100)
   assert wallet.balance == 100
   wallet.close ()
def test_wallet_add_cash ():
   wallet = Wallet (initial_amount = 10)
   wallet.add_cash (amount = 90)
   assert wallet.balance == 100
   wallet.close ()
def test_wallet_spend_cash ():
   wallet = Wallet (initial_amount = 20)
   wallet.spend_cash (amount = 10)
   assert wallet.balance == 10
   wallet.close ()

Ahem, interesting! Have you noticed? There is a lot of boilerplate code. Another thing worth noting is that this test does something else besides testing the functional part, for example, creating a Wallet and closing it with wallet.close ().

Now let's look at how you can get rid of boilerplate code using fixtures pytest.

import pytest
from _pytest.fixtures import SubRequest
from wallet import wallet
# ===================== fixtures
@ pytest.fixture
def wallet (request: SubRequest):
   param = getattr (request, ‘param’, None)
   if param:
     prepared_wallet = Wallet (initial_amount = param[0])
   else:
     prepared_wallet = Wallet ()
   yield prepared_wallet
   prepared_wallet.close ()
# ===================== tests
def test_default_initial_amount (wallet):
   assert wallet.balance == 0
@ pytest.mark.parametrize (‘wallet’, [(100,)], indirect = True)
def test_setting_initial_amount (wallet):
   assert wallet.balance == 100
@ pytest.mark.parametrize (‘wallet’, [(10,)], indirect = True)
def test_wallet_add_cash (wallet):
   wallet.add_cash (amount = 90)
   assert wallet.balance == 100
@ pytest.mark.parametrize (‘wallet’, [(20,)], indirect = True)
def test_wallet_spend_cash (wallet):
   wallet.spend_cash (amount = 10)
   assert wallet.balance == 10

Nice, isn't it? Test functions are now compact and do exactly what they should do. Wallet is configured, installed, and closed using fixtures wallet. Fixtures not only help write reusable code, but also add the concept of data sharing. If you look closely, then the amount in wallet – This is part of the test data provided externally by the test logic, and not rigidly fixed inside the function.

@ pytest.mark.parametrize (‘wallet’, [(10,)], indirect = True)

In a more controlled environment, you may have a file with test data, for example test-data.ini in your repository or shell that can read it, while your test function can call various shells to read test data.

It is recommended, however, to put all your fixtures in a special file. conftest.py. This is a special file in pytest that allows the test to detect global fixtures.

But I have test cases that I want to run on different datasets!

Don't worry u pytest There is a cool feature to parameterize your fixture. Let's look at an example.

Suppose your product has a CLI that is locally managed. In addition, your product has many default parameters that are set at startup, and you want to check all the values ​​of these parameters.

You might consider writing a separate test case for each of these parameters, but with pytest everything is much simpler!

@ pytest.mark.parametrize (“setting_name, setting_value”, [(‘qdb_mem_usage’, ‘low’),
(‘report_crashes’, ‘yes’),
(‘stop_download_on_hang’, ‘no’),
(‘stop_download_on_disconnect’, ‘no’),
(‘reduce_connections_on_congestion’, ‘no’),
(‘global.max_web_users’, ‘1024’),
(‘global.max_downloads’, ‘5’),
(‘use_kernel_congestion_detection’, ‘no’),
(‘log_type’, ‘normal’),
(‘no_signature_check’, ‘no’),
(‘disable_xmlrpc’, ‘no’),
(‘disable_ntp’, ‘yes’),
(‘ssl_mode’, ‘tls_1_2’),]) def test_settings_defaults (self, setting_name, setting_value):
   assert product_shell.run_command (setting_name) == 
     self. ”The current value for ’ {0}  ’is ’ {1}  ’.”. format (setting_name, setting_value), 
 ‘The {} default should be {}’. Format (preference_name, preference_value)

Cool, isn't it? You just wrote 13 test cases (each sets a different value setting_value), and in the future, if you add a new parameter to your product, then all you need to do is add another tuple.

How does pytest integrate with user interface testing with Selenium and API tests?

Well, your product may have several interfaces. CLI – as we said above. Similar to the GUI and API. Before deploying your software product, it is important to test all of them. In enterprise software, where several components are interconnected and dependent on each other, a change in one part can affect all the others.

Remember that pytest – This is just a framework to facilitate testing, not a specific type of testing. That is, you can create tests for the GUI using Selenium or, for example, tests for the API with the library requests from Python and run them with pytest.

For example, at a high level, this may be a check of the structure of the repository.

As you see in the image above, it gives a good opportunity to separate the components:

apiobjects: A good place to create wrappers for calling API endpoints. You may have Baseapiobject and a derived class that meets your requirements.

helpers: You can add your helper methods here.

lib: library files that can be used by various components, for example, your fixtures in conftest, pageobjects etc.

pageobjects: architecture pattern PageObjects can be used to create classes on various GUI pages. We use Webium, which is a library of implementations of Page Object templates for Python.

suites: you can write your sets of pylint checks for code, they will help you gain more confidence in the quality of your code.

tests: You can catalog tests based on your preferences. This will make it easy to manage and review your tests.

I brought it just for reference, the structure of the repository and dependencies can be organized according to your personal needs.

I have many test cases and I want them to run in parallel

You can have many test cases in your set, and it happens that you need to run them in parallel and reduce the overall test execution time.

Pytest offers an awesome parallel test run plugin called pytest-xdist, which adds several unique runtimes to the base pytest. Install this plugin using pip.

pip install pytest-xdist

Let's see how it works with an example.

I have a CloudApp automated testing repository for my Selenium GUI tests. In addition, it is constantly growing and updated with new tests and now it has hundreds of tests. What I want to do is run them in parallel and reduce the overall test execution time.

In terminal just type pytest in the project root folder / test folder. This will allow you to run all the tests.

pytest -s -v -n = 2

pytest-xdist will run all tests in parallel!

In this way, you can also run multiple browsers in parallel.

Reports

Pytest comes with built-in support for creating test results files that can be opened using Jenkins, Bamboo, or other continuous integration servers. Use the following:

pytest test / file / path - junitxml = path

This will help generate a great XML file that can be opened with many parsers.

Conclusion

The popularity of Pytest is growing every year. In addition, it has powerful community support, which allows you to access many extensions, for example pytest-djangowhich will help write tests for web applications in Django. Remember that pytest supports test cases. unittestso if you use unittest, pytest should be considered in more detail.

Sources

• docs.pytest.org/en/latest
• pythontesting.net/framework/pytest/pytest-introduction

That's all. See you on the course!