UI testing automation

We are a team of a mobile project FL.ru (Alina Smirnova, Mobile QA Engineer and Viktor Kotolevsky, Flutter Mobile Development Lead). In our article, we want to introduce you to the Flutter Driver and talk about automating UI testing of mobile applications using this tool. Longread consists of two parts: about automation and about expanding the interaction of the Flutter Driver with the application.

Let’s start.

Our service FL.ru is the largest Russian-language freelance exchange. Like all of us, we are faced with certain difficulties in testing and try to quickly solve the assigned tasks. With the scaling of the FL.ru application, it became necessary to write autotests on the project. Let’s get to know the Flutter Driver.

The Flutter Driver is a tool specially designed for testing applications built on Flutter. It is very similar to projects such as Selenium driver, Protractor and Google Espresso, can be used to test various elements of the user interface and helps to write tests for integration testing in the Dart programming language. We decided to stick with the Flutter Driver as it is a set of useful methods for testing the work with the application interface right out of the box.

Setting up the environment

In order to start writing tests, you need to add a package depending on the project under test flutter driver to file pubspec.yaml:

flutter_driver:
    sdk: flutter
  test: any

The integration test suites do not run in the same process as the application under test, so you need to create a folder in the project test driver with two files inside: app.dart and app_test.dart

The first file contains an application equipped with tools for controlling the application through test cases, which are described in the second file.

File name app.dart can be anything. File app_test.dart must match the name of the first file, but with the ending _test. Thus, the following structure will appear:

my_app/
  lib/
    main.dart
  test_driver/
    app.dart
    app_test.dart

Importing the extension flutter_driver to file app.dart and activate its work in the main function:

import 'package:flutter_driver/driver_extension.dart';
import 'package:counter_app/main.dart' as app;

void main() {
  enableFlutterDriverExtension();
  app.main();
}

Now you can start writing tests in a file app_test.dart. According to the official documentation, the libraries need to be imported flutter_driver and test. The first provides an API for testing flutter applications and running on emulators and real devices, and the second provides a standard way to write and run tests in Dart. An example of a test case structure:

import 'package:flutter_driver/flutter_driver.dart';
import 'package:test/test.dart';
    // Тесты можно сгруппировать с помощью функции group ()
void main() {
  group(My App', () {    
    FlutterDriver driver;
    // Перед запуском тестов вызываем функцию соединения с flutter driver
    setUpAll(() async {
      driver = await FlutterDriver.connect();
    });

    // Завершаем соединение с flutter driver после прогона тестов
    tearDownAll(() async {
      if (driver != null) {
        driver.close();
      }
    });

   // Описываем тест, в данном случае тест ищет текст, содержащий "0"

    test('starts at 0', () async {
   // Объявляем локаторы для поиска элементов в интерфейсе 
      final counterTextFinder = find.byValueKey('counter');
   // Описываем действия с элементами интерфейса
      expect(await driver.getText(counterTextFinder), "0");
    });
  });
}

Thus, having created locators for certain widgets, we can write tests that start after connecting to the application through the function setUpAll (), go through the necessary scripts and end by disconnecting from the application through the function tearDownAll ()

Finding and defining locators

What are locators? Locator is a command that tells Flutter Driver which GUI elements (e.g. text field, buttons, checkboxes, etc.) to use to simulate user experience (on clicking, swiping, swiping, etc.).

Flutter has several ways to find and label locators: bySemanticsLabel, byTooltip, byType, byValueKey.

To write a UI test, we need to know how to create a “link” to a UI element to access its properties. If the element does not have a locator, the easiest and most reliable way to achieve this is to “reference” the key (key) of the UI element and find that locator using the findbyValueKey () method.

Key (key) Is an identifier that points to the searched element in the interface structure. Let’s say we need to create a key for the button to go to the user profile. Accordingly, you need to make sure that the user interface component (Flat Button) has a locator or add it yourself:

return FlatButton(
                      key: Key('profileButton'),
                      child: Text('Профиль')            
)

If you don’t know where a specific interface element is in the code, Android Studio tools come to the rescue: Flutter Inspector or DevTools. For this:

  • Launching the application

  • Open Flutter Inspector or DevTools

  • Go to the view mode of interface elements Select Widget Mode

After activating the Select Widget Mode function, any touch on the interface element will allow you to move in the project code to the desired widget.

You can enable Flutter Inspector through View> Tool Windows or set the setting for the start of the tool with the launch of Android Studio through File> Settings> Languages ​​and Frameworks> Open Flutter Inspector on launch

  1. Accessing the Flutter Inspector via the menu

2. Enabling Flutter Inspector when launching Android Studio

3. An example of the Select Widget Mode function from Flutter Inspector

Across DevTools – a special set of tools for debugging for Flutter, the same functionality is also available, except that DevTools are opened in a browser.

4. DevTools is available from the Run tab

5. An example of the Select Widget Mode function from DevTools

After the element is found, you need to create a “link” to the user interface component in the test file:

final userProfile = find.byValueKey('profileButton');

And describe the actions in the body of the test that must be performed with the element, for example, press or define the text on the button:

await driver.tap(userProfile);
expect(await driver.getText(userProfile), "Профиль");

To make tests more readable and easier to maintain, they can be structured using a DRY approach, that is, putting all the locators in one place or breaking them up into classes based on the functionality of the application. If the UI changes with the use of some element, then we will change the locator in only one place, for example, the locator class for the profile page:

abstract class ProfileIds {
 static const profileAvatar = "profileAvatar"; 
 static const buttonFeedback = "buttonFeedback";
 static const editProfileButton = "editProfileButton";
 static const portfolioButton = "portfolioButton";
 }

Flutter Driver Methods

The main methods of interacting with the interface of the application under test are the following list:

  • tap () – plays a tap in the center of the widget;

  • screenshot () – creates a screenshot;

  • enterText () – enters text in the input field on which the focus is;

  • getText () – returns the text from the widget;

  • scroll () – returns the action of scrolling the screen;

  • scrollIntoView () – scrolls the element with the Scrollable property in which the desired element is located;

  • scrollUntilVisible () – repeats the scrolling of the widget with the Scrollable property until the desired element appears on the screen.

Running test scripts

flutter drive --target=test_driver/app.dart

This command creates a build of the application and installs it on the device / emulator, launches this application and runs test scripts from the app_test.dart file.

For convenience, you can split tests into groups of files by functionality and run them separately (or all using the first command if you import them into app_test.dart):

flutter drive --driver=test_driver/ui/login/login_test.dart --target=test_driver/ui/app.dart

Examples of how Flutter Driver methods work

Click on the checkbox after it appears on the screen:

final checkbox = find.byValueKey('checkboxItem');

await driver.waitFor(checkbox ).then((_) => driver.tap(checkbox));

Scrolling through the list of orders until the first publication:

final feedList = find.byValueKey(FeedIds.newFeedList);
final feedItem = find.byValueKey(feedProjectCard + "1");

await driver.scrollUntilVisible(feedList, feedItem, dyScroll: -300);

We answer in the chat:

final fieldMessage = find.byValueKey(MessageIds.fieldMessage);
final sendMessage = find.byValueKey(MessageIds.buttonSendMessage);

await driver.tap(fieldMessage);
await driver.enterText("Здравствуйте");
await driver.waitFor(sendMessage).then((_) => driver.tap(sendMessage));
await driver.waitFor(find.text("Здравствуйте"));

Writing UI tests has many benefits. They are one of the most reliable types of tests in the testing pyramid because they are closest to the end-user experience and will give you the confidence that your application is performing as expected.

At the end of this part, I want to share the problems that we encountered when automating testing with the Flutter Driver:

1. slow passing of tests – to speed up, we removed authorization for each test;

2. poorly readable structure with an increase in the number of tests – to correct this situation, we divided them into functional groups;

3. dependence of tests on the work of the back-end – we solve by using mocks in tests.

Extending the interaction of Flutter Driver with the application

Flutter Driver has enough functionality to interact with the application at the interface level, namely tapes, text input, scrolling, searching for items by key, etc. But sometimes it is required to implement more complex interaction schemes to ensure the necessary conditions for passing the tests. There may be a widget on the screen whose state cannot be determined with a simple driver.getText. Or, to pass the test, you need to simulate a response from the server. To do this, there is the enableFlutterDriverExtension method, which must be called before running the tests in the main main method.

The enableFlutterDriverExtension method has three parameters: DataHandler handler, bool silenceErrors, List finders. Let’s consider these parameters in more detail.

The DataHandler handler parameter is a function:

Future<String> Function(String message) 

It handles messages passed through the driver.requrestData method. Below is an example of using this method.

void main() {
 enableFlutterDriverExtension(handler: (string) => UITestMessageHandler.handle(string));

 /// Запуск driver тестов
 /// runApp();
}

abstract class UITestMessageHandler {
 static final _messageHandlers = <UITestMessageHandler>[
   GetStringMessageHandler(),
 ];

 static Future<String> handle(String message) async {
   try {
     final json = jsonDecode(message);
     final type = json['type'];
     final data = json['data'];
     return _messageHandlers.firstWhere((handler) => handler.canHandle(type), orElse: () => null)?.handleMessage(data) ?? null;
   } catch (e) {
     return null;
   }
 }

 bool canHandle(String type);

 Future<String> handleMessage(data);
}

class GetStringMessage extends UITestMessage {
 static const String typeId = 'GetStringMessage';

 GetStringMessage(String stringKey) : super(stringKey);

 @override
 String get type => typeId;
}

class GetStringMessageHandler extends UITestMessageHandler {
 @override
 bool canHandle(String type) => type == GetStringMessage.typeId;

 @override
 Future<String> handleMessage(data) async {
   if (data != null && data is String) {
     final s = await S.load(Locale('ru', ''));
     switch (data) {
       case StringKey.messageOk:
         return s.message_ok;
       case StringKey.messageCancel:
         return s.message_cancel;
     }
   }
   return null;
 }
}

class StringKey {
 static const messageOk = "messageOk";
 static const messageCancel = "messageCancel";
}

In this implementation, all messages are sent as a json object with the required type and data fields. The message type (type) helps determine the class that will process messages. In our case, the GetStringMessage message is handled by the GetStringMessageHandler class. The handleMessage method will be called already inside the application, thus we can get localized strings by the key from the StringKey class, it is enough to make a call inside the test:

static Future<void> assertString(FlutterDriver driver, String stringKey, {String keyId}) async {
 final stringValue = await driver.requestData(GetStringMessage(stringKey).toString());
 expect(await driver.getText(find.text(stringValue), stringValue);
}

The following example provides an implementation of authorization without having to contact the server:

class AuthorizeMessageHandler extends UITestMessageHandler {
 @override
 bool canHandle(String type) => type == AuthorizeMessage.typeId;

 @override
 Future<String> handleMessage(data) async {
   if (data != null && data is Map<String, dynamic>) {
     final user = User.fromJson(data['user']);

     return DiProvider.get<LoginService>()
.statuses.add(user).then((value) => 'ok');
   }
   return null;
 }
}

class AuthorizeMessage extends UITestMessage {
 static const String typeId = 'AuthorizeMessage';

 AuthorizeMessage(User user) : super({
   'user': user.toJson(),
 });

 @override
 String get type => typeId;
}

The LoginService class, which provides an API for authorization, has a PublishSubject statuses field. In a simple case, the user enters a username / password, after which these data are transmitted to the server, where authorization is already taking place. Next, the server returns a User object, which has already been added to the statuses stream. Now, with the help of AuthorizeMessageHandler, we do not need to go through the login / password entry screens in order to log in to the application while the integration tests are running.

Next, we’ll look at the List finders parameter. This list contains custom classes for finding and validating widgets. Below is an example of a FinderExtension implementation for finding a MyWidget widget by the value of the param parameter.

class MyWidget extends StatelessWidget {
 final String param;
 const MyWidget(this.param, {Key key}) : super(key: key);

 @override
 Widget build(BuildContext context) {
   /// build MyWidget
 }
}

class MyWidgetFinder extends SerializableFinder {
 const MyWidgetFinder(this.param);

 final String param;

 @override
 String get finderType => 'MyWidget';

 @override
 Map<String, String> serialize() => super.serialize()
   ..addAll({'param': param});
}

class MyWidgetFinderExtension extends FinderExtension {

 String get finderType => 'MyWidget';

 SerializableFinder deserialize(Map<String, String> params, DeserializeFinderFactory finderFactory) {
   return MyWidgetFinder(params['param']);
 }

 Finder createFinder(SerializableFinder finder) {
   MyWidgetFinder myWidgetFinder = finder as MyWidgetFinder;

   return myWidgetFinder.byElementPredicate((Element element) {
     final Widget widget = element.widget;
     if (element.widget is MyWidget) {
       return element.widget.param == myWidgetFinder.param;
     }
     return false;
   });
 }

The silenceErrors parameter speaks for itself: if the value is true, then exceptions that occur during message processing will not be logged.

Thus, Flutter Driver allows developers and test automation engineers to add their own unique functionality to organize complex integration tests. As part of our FL.ru project, we were able to easily connect tests to CI, which helped us speed up the testing process and improve quality: now failures are detected faster and, as such, can be eliminated faster, which leads to more productive flow work.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *