Backend Client as a universal tool for UI autotests

Hi all!

In this article, I would like to share the tool that we at Okko use for UI autotests for Android platform client applications.

Usually, when companies start thinking about UI tests, the first milestone of such automation is pure e2e tests. This is already a good result, but unfortunately, as soon as our coverage starts to go beyond what can only be done through the client, big problems begin.

Also, as coverage expands, we begin to face the problem of test execution time, which can easily be several hours, since for each case we have to re-click on the client all the preconditions for any of the tests.

And accordingly, the more we do this, the higher the probability of falling UI tests, since in the ideal world of interaction with the client there should be exactly as much as necessary to complete the test, but we get the opposite. Therefore, after such tests have already been written, everyone starts to think about whether to start applying something from the backend to the front-end tests. We are facing the same issue.

In the article I will tell you how you can get out of this situation, and this experience can be applied preventively, even before your first tests are written.

Backend Client

We came up with a very universal thing, and without really bothering about its name, we called it Backend Client.

Previously, this tool already existed in the company in a limited form as a library for backend testers, it had internal dependencies and other troubles. We took this library, pulled it out into a separate entity, added the functionality necessary for the front, and turned it into such a contraption that everyone can now use, including us – client application automators.

Multi-platform structure

Multi-platform structure

We use this tool for UI autotests on three platforms: Web, Android, and iOS. The tests for the first two platforms are written in python – the backend client is available as a python library on them, so you just need to import it into the project. Autotests for iOS are written in Swift (XCUITest framework), and for this platform it was necessary to write a CLI handler through which work with the backend takes place.

Import, initialization, calls

Import, initialization, calls

In Python autotests, we initialize our client through the pytest fixture, and thus access it from anywhere in the code. On Swift, everything is a little more complicated – here an external call to a shell script is used, in which the necessary tricks are already being turned.

Now we are using our toolkit on a PRE-PROD environment (that is, a test one), which in theory allows us to manipulate any data without the possibility of breaking PROD. I will write about the latter below, since integration in production of this kind is included in the roadmap for further development.

Functionality

Functionality

For clarity, I grouped all possible functions into groups and divided them into three large sections:

  • The first group contains everything related to the users of the service. Now we can perform any operations with accounts – create, delete, edit information, merge accounts (the so-called merge) and do all this before or at the time of the tests.

  • The second group includes everything related to content. We can play with the so-called user’s bag – change the content that he added to favorites, recommendations and browsing history, checking a bunch of scenarios at once. We can even change the saved view time for a single piece of content as a test precondition – this is necessary because the logic of the application’s behavior is different depending on whether the user has started browsing or not yet.

  • The third group is perhaps the most important, because it is directly related to the profit of our cinema. You can check the availability and correct operation of each monetization model, as well as manage the balance and linking bank cards.

What is it for

Creating preconditions

Creating preconditions

Preparation of a test environment that is inaccessible to the client has become possible. Now what could not be done from the UI of the application, and required manual preparation of preconditions, can be done as an autotest precondition. Manual “fitting” of test data is no longer required.

On the picture above we abstractly create a user from Kazakhstan in automatic mode, before the test we assign the necessary parameters to him in order to check how the application behaves when buying content with these settings.

Reducing Autotest Time

Reducing Autotest Time

The state of the application, which was previously set through the UI, can now be set in a couple of commands either as a test precondition or directly during its execution.

On the picture above shows an example in which we need to perform some kind of test on a user with a specific subscription. If we used only the UI, the preparation of the environment would take three more steps. Below we will demonstrate a clear example of a similar test case from practice, how many times we have accelerated autotests thanks to our Backend Client.

Reducing the number of flaky tests

Reducing the number of flaky tests

Moving most of the action to the backend increases the stability of UI autotests, since there are always factors in the latter that can suddenly fail. Applying this practice with transfer, we get:

  • fewer actions with the client application (i.e. the device is less likely to miss a command if there are fewer commands themselves)

  • fewer interactions with the client reduces the number of potential errors of the client-side automation framework

  • if we are talking about testing on real devices, then if the conditions for the test are prepared from the backend side, then we do not use the wireless network in which the device is located, excluding potential network losses.

On the picture above here is a real before and after example from our Android TV app. Previously, for PIN-code authorization, it was required to launch an external browser on the machine, slip the desired user into it, and only then log in. Now we do it through the back, and everything works much more stable.

Working with backend logs

Working with backend logs

Also, our tool contains various auxiliary modules and utilities, including those for finding errors in the logs that are sent by backend microservices during the tests. In other words, in parallel with checking that the application UI correctly processes the script, we can also monitor if something suspicious happened on the back. And if it happened, then we can get this information from the local Elasticsearch and attach it to the fallen test in order to make it easier for QA to further the so-called. research, and go looking for the root of the problem in the right direction.

Manual testing tool

Manual testing tool

Manual QA, instead of manually creating an environment for tests, can use the CLI utility and do it by executing a single script. This thing can ease the routine and reduce the same type of actions with the backend to a minimum, so that a specialist can do direct testing without wasting time on long precondition settings.

About the benefits – clearly

You can clearly see how much we managed to improve individual tests. First, let’s talk about the increase in speed. Let’s take a real test case where we need to check that each of the purchased subscriptions opens access to certain content:

Videos with tests will be accelerated by 10 times. In order to get by with only the UI tests that we see on the left, we need:

  • register a new user

  • link a test card to your account

  • purchase the required subscription

  • and only then check if the required content has opened for viewing.

On the right, we see how the same case is executed using the Backend Client, where there is a much smaller number of UI actions that need to be performed on the end device. Before each iteration of the test, we simply remove the old subscription from the account and add a new one, without even having to re-authorize on the client.

On a dry run-time basis, we get seven and a half minutes for the left scenario, and less than a minute for the right one. The increase in speed is very significant.

Also, for statistics, we ran both cases as much as a hundred times each, and received the following stability results – due to the peculiarities of the payment gateway in the test environment, the payment card was not always linked, so about two times out of ten the tests ended with an error.

Using the Backend Client, we can bypass this factor and directly “attach” a subscription to the right user, bypassing the payment gateway. We get a stable result of the test – 0 falls out of 100 possible.

Development plans

Tests for PROD

The logical decision regarding further development is the output of both tests and the described toolkit to the production environment.

The advantages are obvious – by running autotests on production, we can definitely be sure that the end user will be protected as much as possible from possible critical bugs that block user scenarios.

Statistics and analytics

In large projects, such a thing as sending statistics is a business-critical function – after all, the presence of all the feedback that allows you to understand in which direction to further develop the product depends on it. So far, unfortunately, it looks like manual work, but in the plans we are going to use the Backend client for automatic checks of the data sent by the application. Let’s start with the most critical – sending analytics when viewing content in the player, gradually moving towards expansion.

Implement similar things in your projects, and stable autotests for you!

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *