How to test remotely and do without tons of different devices. Surf experience

November 2021, COVID-19 is back in our lives. We, like the rest of the country, already worked from home a year ago. Now the city has again declared “non-working days”, and all employees have switched to remote work.

My name is Pavel Zhdanov, I work as a tester at Surf… We develop mobile applications, and QA needs a lot of mobile devices to work. What to do when you are transferred to a remote location? Carrying ten cell phones to your home?

In this article I will tell you how to organize QA work on a remote location and show you how it works in Surf.

Office or remote location? Where can I get so many devices at home, huh?

Important Notice: Our QA team is based in the same city

The Surf QA department consists of twenty people. We test projects in the banking environment, e-commerce and others. The testing process itself at home and in the office looks the same. But working from home has one big drawback. Absolutely all the testers with whom I talked said about it. And this minus:

few devices

It’s all. Indeed, a mobile tester needs to have iPhone 5 through 13 with different combinations of iOS versions. Also, for some projects or before the first release, an iOS tablet will come in handy. Plus dozens of Android phones from the fourth to the twelfth version.

We come to an ambiguous situation. The office has devices for every taste: old, new. And at home what?

All right, let’s go to work in the office and not a step home during working hours !!! Haha, especially true when you were forcibly sent home 🙁

No! You can also at home. The tester must be able to adapt to different conditions.

Okay, but if I’m from another city, how can I work?

After reading the article, you will understand that all solutions are valid for guys from anywhere in the world.

Now I’ll tell you how we built QA workflows remotely.

But first the question: “What Android and iOS devices do I need to take home?”

The first factor for choosing a device is the feature of a particular project. For example:

  • the project is being developed for only one platform (Android or IOS);

  • the project only supports certain OS versions;

  • the project is being developed for phones based on Huawei OS;

  • the project is developed as a demo, and the device is not important;

  • the project is being developed only for tablets.

Depending on this, you can roughly understand what smartphones are needed.

Second factor: analytics of OS usage. For example, here are statistics for one of our apps, collected over the past three months using AppMetrica:

Apple with a policy of updating even very old devices helps testers. Most users have upgraded to iOS 14 or the recently released iOS 15. With these numbers, support for iOS 10-13 can be diminished – but not completely ruled out. We check the application first on new operating systems, then on old ones.

With Android, things are not so simple. Almost all versions fall into the mandatory scan zone. But, as with Apple smartphones, we first check on more recent devices of 9-11 versions, then on Android 6-8. Android 12 users are few and far between. Android 5 is also lagging behind: you can think about excluding it from support.

The third factor is the device park in the company… We don’t have many Android 5 devices at Surf anymore: support for them is coming to an end. New iPhone 13 devices are being bought in addition, a number of devices have been transferred to iOS 15. Android 12 is now available only for Pixel. The rest have not yet submitted updates for their devices, and it is not advisable to buy phones with Android 12 yet.

The fourth factor is how many testers are on the project. If there are two or more, you can easily cover the project with all OSes.

Having your own device also helps. Remotely, some QA’s use their own smartphone to check builds. It is clear that not everyone uses personal devices for work purposes: this is taken into account when distributing devices.

Total:

  • Find out the specifics of the project and what devices you need to use for verification.

  • Take into account the analytics of the use of devices on the project.

  • View the fleet of devices and how many necessary devices are in stock.

  • If there are several QAs on the project, correctly distribute the devices between them.

  • Remember the ability to use a personal device for testing.

There are device-specific bugs. And also the devices will need to be replaced – for example, at the end of the iteration. And it is also necessary to check the roll-forward on everything. We have come up with two types of solutions for such cases:

Physical solution

  • Contact the office manager who will transfer the device by taxi. This option works because we are all in the same city.

Virtual solution

  • A special channel in the slack, where everyone can write a message about the device: take, look, check something in the assembly, check a bug, and so on.

❗️If there is a nondisclosure agreement (NDA) for the project, you will have to use only your own fleet of devices and use emulators and simulators (about them will be a little below).

How the site works

First, the user is authorized. Then he gets to the page with all the devices of the company.

Each line consists of: “Device-User-SlackID-Bind / Unbind”.

The user attaches the device with the Bind key. Unbind if the device is already given away.

For greater convenience, we have made a device search bar and a “My devices” button so that the user can see what devices are recorded on him.

Why do we need this site if the chat in Slack is enough?

The site was created to minimize chat flooding. The device search algorithm looks like this:

  1. Go to the site and find who has the device.

  2. Write to a person and ask for a device for tests.

  3. If he doesn’t have the device, go to the chat and write the cherished “@here” and the phrase “who has the iPhone 7 iOS 12 device?”

Summarizing these small innovations during a pandemic, we got an easy opportunity to “test on all devices” while at home.

How software makes life easier

Modern software greatly simplifies the life of a remote tester. This is primarily about emulators and simulators.

Emulation is the reproduction of the operation of a program or system while maintaining the key properties and principles of operation. Copies the essence (process, object) of the work.

Simulation is the reproduction of the program’s operation, simulating the execution of the code. Copies the surrounding reality (shell, object properties).

We use both emulators and simulators. From the survey of our employees, it became clear that the main tools for work are Xcode (iOS) and Android Studio. At the same time, emulation is possible both for the developer of native applications and for Flutter developersthat make one application for all platforms.

In addition to emulation with official tools, simple Android emulators are used. You can find a lot of suggestions on Google.

And why do we need all this hemorrhoids with devices, when there are already so many emulators and simulators?

Yes, we at Surf, of course, use emulators and simulators: this allows us to more extensively check the layout and check the assembly quickly, without using a physical device. However, they cannot yet completely replace real devices. Using the emulator, you cannot check the features of smartphones: whether the camera, mobile Internet, voice recorder, and so on are working correctly.

At the same time, simulators do not take into account the hardware part of the devices at all: the behavior can sometimes differ from the behavior on a real smartphone. Because of this, it happens that the application crashes on the emulator or simulator and works fine on the device – and vice versa. Therefore, the key to success is in balance: for full-fledged work, we use real smartphones, emulators and simulators.

Let’s sum up the pros and cons of working with such software.

pros

  • Different versions of Android and iOS, which can be installed separately.

  • Inexpensive solution.

  • Changing the screen resolution and type of device (tablet, mobile) makes it easy to check the layout.

Minuses:

  • Advertising in some emulators from the Internet.

  • Do not allow simulating incoming SMS and calls.

  • They cannot imitate the work associated with the features of smartphones: with a camera, a mobile network, in economy mode.

  • If you run multiple emulators, they will slow down on older PCs.

  • Not all OS versions are available on emulators.

Or you can buy a farm of mobile devices …

One of the new solutions for remote testing is testing using cloud technologies, or “mobile device farms”. These tools or web services allow you to remotely connect to a real device and fully test the application.

It turns out that there is access to the functionality of a real phone, only physically the device is “somewhere out there, far away”.

We use our own remote testing solution. This is our personal farm of mobile devices: it consists of devices that are connected to a PC in the office.

Remote debugging is also convenient for developers: they can fix bugs reproducible on some “special” device.

We have configured remote access to devices using the utility scrcpyinstalled on the node. It allows you to see the screen of a device connected via usb and control it. For this you need:

  • Have a VPN configured on your PC.

  • Connect to the node server (the address will be given by the system administrator), and then enter the username and password.

  • Download the project repository.

  • Ask someone in the office to connect the device to the node for remote access.

  • To start screen replay in a terminal, run the command scrcpy.

If you cannot organize such a stand physically or financially, but you need to test it on devices, you can use cloud farms. I will not go into details about the services, there are a lot of articles on the Internet. Briefly about the most popular:

A mobile device farm – office or cloud – allows you to test applications without having devices at hand. It’s convenient: it’s not a problem if the office runs out of devices and someone doesn’t get the phone they need. And a tester from anywhere in the world can become a team member.

Mobile tester can work remotely!

Going remote for mobile QA in the first lockdown seemed a daunting and daunting task. We were able to customize the workflow. Not everything was perfect, but we minimized the loss in quality and efficiency: the indicators did not drop.

The usual workflows are exactly the same whether at home or in the office. Yes, there are some nuances with devices. But this can be solved:

  • An integrated approach of physical and virtual solutions minimizes the device problem.

  • Emulators and simulators help with testing.

  • You can use cloud technology – a “farm of mobile devices”, which will give access to hundreds of devices.

  • You can build a “farm of mobile devices” in your office, and testers can remotely connect to it. Bonus: programmers will also say “thank you”, because the farm will be useful for them too.

Similar Posts

Leave a Reply