Debugging Applications in the Huawei Ecosystem: Cloud Debugging Platform, A / B and Open Testing Services
Hello, Habr! The Huawei mobile ecosystem has several tools for debugging and validating applications: you can run automated tests in the cloud or remotely on Huawei devices, as well as work with groups of users. On the DigiX Lab cloud platform, developers can check stability, performance, power consumption and compatibility of their applications with our brand devices in emulator mode. A / B and open test services help you understand audience reactions and get feedback. Under the cut I will tell you about the capabilities of these services and how to start working in them.
What the DigiX Lab consists of
DigiX Lab – laboratory in the Huawei Developer environment. It includes 2 services: testing and debugging.
Cloud testing service allows you to test the compatibility, stability, performance and power consumption of your application using device emulators. After testing, the developer receives the results in the form of a detailed report. Thanks to this, it can detect any problems with the application: crashes, unexpected closings, increased power consumption, errors, etc.
Cloud debugging (debugging) service allows you to identify problems with the operation of applications on specific devices. After confirming your identity, you can work for 24 hours for free with any real devices – up to 2 hours of work is provided for one model. When the 24 hours have expired, you can apply for an extension of 4 or 8 hours. The number of applications is not limited.
How do I get started with DigiX Lab?
Both corporate and individual developers can register in the DigiX Lab service. An individual developer will need to upload an identity document and wait for it to be verified. To verify the data, a corporate developer will need to provide a business license, number DUNS (nine-digit internationally recognized company identifier), company address.
Enterprise developers gain access to more sections, such as Payment Service, Huawei Ability Gallery, AppTouch, Paid Promotion and Gifts, Comment Management, AppAdvisor, HiAI, HMS Core Ads Kit. Individual developers will not have access to these sections.
Types of cloud tests
In DigiX Lab, you can run 4 types of tests: compatibility, stability, performance and power consumption.
Compatibility testing will take about 60 minutes. The test will provide information about the first installation and reinstallation of the application, its uninstallation and the resulting crashes and errors, such as no response (ANR) or unexpected application closure.
This mode allows you to filter devices by model and operating system version. You can choose one or several models.
Stability testing Allows you to track application failures such as hang-up – no response (ANR) incidents, platform failures – native crash – and resource leaks. The system will generate a report from the data on all incidents. If the number of failures is more than 10, then the testing was unsuccessful. You can adjust the duration of the test at your discretion.
When testing stability, you can select the duration of testing, as well as select the required smartphone models and operating system versions. Only one model can be tested at a time. To test another model, you need to create a new test. At the end of testing, you can download the report or view its short version. The summary report contains data on pass rates, error distributions and testing details.
The stability test report contains Logcat logs, screenshots of the test progress, recorded performance data, including startup duration, CPU load, memory usage, traffic and power consumption.
Performance testing will take about 60 minutes. The benchmark provides application performance data such as CPU usage, memory usage, power consumption, traffic usage, and more, as well as in-depth analysis to identify application performance problems. You can also select one or more devices for performance testing.
Power consumption testing will take about 100 minutes. The test provides power consumption data and demonstrates resource usage such as wakelock duration, screen usage, Wi-Fi and audio usage, and behavior patterns.
During testing, the smart bypass algorithm takes screenshots to make it easier to detect problems. Some metrics from the test may differ depending on the application category.
How to run tests and see their results
After confirming your account, you will have access to cloud testing services. More than 2,000 device models are available in DigiX Lab, and this library is constantly growing. Automatic testing is carried out on Huawei mobile phone emulators.
To conduct testing, you need to download the APK, select the device models and the version of the operating system. The number of devices to run the test is unlimited, but we recommend choosing up to 9 models. Otherwise, you will have to wait too long for your turn. Remember that only one device can be selected for the stability test.
Sometimes waiting for a task can take too long. This means that too many developers are using one of the selected models at the same time. The system can also block the test, which means that the selected device has been removed. In this case, you need to select another device and start the test again.
A short version of the report can be viewed without downloading. This report contains data on pass rates, error distributions, and testing details.
Localizing errors using a debugging service
Debugging service allows you to select suitable gadget models for testing the application in real time. You get access to real brand devices and can test the operation of your application on any of them.
This service allows you to select the device on which the testing will be carried out, and the version of the operating system, EMUI and the duration of debugging: 30 minutes, 1 hour or 2 hours. Debugging from one Huawei ID can be performed on 2 real devices at the same time. At any time during debugging, you can view detailed information about the device on which you are debugging.
You are given 2 hours to debug one device. In total, each user gets 24 hours of free use of the service. When a user has less than 2 hours left, they can apply for an extension of 4 or 8 hours. The number of such applications is not limited. Log reports can be filtered by type, and data can be viewed online or exported.
Testing audience reaction with A / B tests
A / B test service allows you to run tests on 2 different groups of users who receive different versions of the tested product and note their reaction. It can be used to run multiple tests for different audiences or compare user reactions to design options, content, functionality.
In order to run A / B tests, you must first integrate Huawei Analytics Kit, then in AppGallery Connect find the project you want and create an experiment on the page with A / B test configurations. In it, you must specify the conditions, duration and key indicators of testing and provide the service with access to the services (remote configuration, Push Kit and Analytics Kit) necessary for its implementation.
Experiment data and report content depend on the tracking indicators you choose. Usually, the report data can be viewed the next day after the publication of the experiment. However, if the tracking indicator is set to “daily interaction duration” or “retention rate after 15 days or more,” it will take longer to generate reports.
Open testing on user groups
This service is still working in beta mode. It allows you to give access to your application to the first test group of users and get feedback. To launch the “open testing”, an email must be sent to agconnect@huawei.com for a claim.
After your application is approved, you can create a test user list in AppGallery Connect, configure it, download the application package, and send out invitations to selected users. After the user agrees to take part in open testing, he will be able to upload a test version of your application to AppGallery. The user will then be able to directly inform you about the problems found in the application.
That’s all for now. If you have questions about testing and debugging applications on the HMS platform, ask in the comments.
