Introduction to testing methods

Hello, Amigos! This is Pavel Gershevich, Mobile Team Lead of the Amiga product development agency. My team and I have prepared a translation for you articles about testing in Flutter. We will consider Unit testing, Widget testing, Golden tests and integration testing. Enjoy reading!

This article provides many examples and common mistakes that professionals encounter when writing tests. It also shows how to write code to make testing easier and how to use AI tools such as ChatGPT or GitHub Copilot to increase the speed of writing tests.

And before I get to the article itself, I want to introduce you to our telegram channel Flutter.Many. We are running it with the whole team of Amiga mobile developers and we talk about personal experience, share useful plugins/libraries, article translations and cases. Join us!

Why do you need to write tests?

There are 3 main reasons why writing tests is necessary:

1. Quality assurance.
   Writing tests helps to find bugs in time, which ensures the correct operation of the application in various scenarios on different operating systems, their versions and even different devices.

   Additionally, some test cases may be too complex to test manually, such as calling more than 1000 API requests at once, but tests can be easily written for such scenarios.

2. Help with refactoring.
   Refactoring code is the most risky because it is easy to introduce bugs unintentionally. However, when running tests after refactoring, bugs will be detected if they exist.

3. Saving time and money.
   While writing tests adds costs at the initial stages of a project, in the long run it saves time and money.

Testing methods

There are 4 main testing methods in Flutter: Unit tests, Widget tests, Golden tests, and Integration tests. They differ in purpose, scale, and execution time.

Unit tests

Unit tests are used to test functions and methods. For example, for static functions, high-level functions or methods separately.

The purpose of unit tests is to check the correct operation of a function or method with different input conditions.

For example, let's imagine we have 3 functions: saveToken, getToken, and login:

bool saveToken( String token) { 
  return sharedPreferences.saveToken(token); 
} 

String  get token => secureStorage.token; // как получить код 

bool login( String email, String password) { 
  final token = apiClient.login(email, password); 
  return saveToken(token); 
}

With unit testing, you need to write tests for each function separately.

For example, for the saveToken function, when a token arrives, the function's response should be true or false depending on the test scenario. When we call the token getter, the function should return the token stored in SecureStorage. For the login function, we need to return true or false depending on the email and password passed to the input.

In addition to testing the response depending on the input, you can also test whether the apiClient.login function was called or how many times it was called. If it was not called or was called more than once, then there is most likely a bug in the code.

Writing unit tests is just a necessary step to ensure that your application works correctly.

In the example above, a deliberate mistake was made, where they saved it in SharedPreferences and tried to get the token from SecureStorage, from which it is definitely impossible to get the right one.

Why can't unit tests catch this error? They are focused on testing each function individually. The saveToken function is used to save the token to SharedPreferences, and it doesn't know if it will be retrieved from there.

Similarly, the token getter is used only to get the token from SecureStorage and does not care where it was saved. When each function is executed correctly, the unit tests also pass. But when the application is launched, these 2 functions will throw an error, then integration tests come to the rescue.

Integration tests

Integration tests are used to test how individual classes and functions work together, or to test the performance of an application running on a real device.

There are also end-to-end (or e2e) tests, which are similar to integration tests, but check not only the internal logic of the application, but also all interactions with various APIs.

For example, we need to test the login functionality. When the user enters the correct email and password, we get to the Home screen.

When running integration tests, the application will run on a real device or emulator and will automatically work as if the tester is testing the application. Thus, the application will work more accurately than when using only Unit tests. But there is a disadvantage – the execution time of integration tests is much longer than that of Unit tests. In addition, when a bug is encountered, it can be very difficult to track down which function exactly it is in.

Unit tests and integration tests are mainly used to check the logic of the application. If you need to test the user interface, for example, “the color of the button matches the design”, “the button is on or off” or “the button is visible”, then you need to use Widget tests and Golden tests.

Widget tests

The purpose of Widget tests is to check that the user interface matches the design and that interactions with it work correctly.

Just like Unit tests, Widget tests do not require the application to be running on a real device or emulator.

It is worth noting that we will not be able to test all widgets using Widget tests.

Golden tests

Golden tests are essentially the same as Widget tests, but they also allow you to check the correct placement of the widget on the screen.

For example, in addition to matching the button color to the design, you can check whether it is enabled, disabled, or visible to the user.

Golden tests also check that the button is positioned on the screen according to the design. This is done by generating an image of the widget's user interface, known as Golden Images, and comparing them to the current ones. If both images match, the test will pass. Golden images can be generated on multiple devices of different sizes, such as phones and tablets.

For example, here are golden images that show the expected user interface in the initial state and after a single press on the Floating Action Button on two different devices: a smartphone and a tablet in landscape orientation.

If you change the color of the Floating Action Button to red and move 2 Text widgets up:

then Golden tests will find these differences and report errors when comparing images, as shown below:

Golden tests save a lot of time and money because they check the correctness of the user interface by comparing images, which is something that regular Widget tests cannot do.

Conclusion

In this article, we looked at a brief overview of the four main testing methods in Flutter. In the next part, we will take a closer look at Unit testing.

More interesting and useful information about Flutter is in the telegram channel Flutter. A lot..

Cases, personal experience, review of plugins and libraries. Join us!