We speed up the passage of iOS UI tests. Part 2: Test Parallelization

This is the second article in a series of articles on iOS automation in which I will talk about how to speed up the passage of UI tests.

This article will be useful for experienced iOS automators or developers.

In this article, we will cover the following steps:

  • why parallelize tests?

  • parallelize tests locally;

  • parallelize tests on CI.

Why parallelize tests?

Parallel testing is your ticket to faster testing. When testing applications, it is important to remember that time is an important factor. Testing an application 100% is a noble cause, but it’s not always easy to achieve the desired test coverage. Parallel testing allows you to run more tests in a shorter amount of time.

Imagine the situation: you have 500 ui tests that run in 4 hours. Four hours to run is a lot, but if you divide these 500 tests into 8 simulations, the run will take 30 minutes!

Parallelizing tests locally

You will need:

  • Xcode version ≥ 11.7;

  • Test plan containing ≥ 2 classes with tests.

In the test plan, you need to select “Execute in parallel”

After that, you can simply run all the tests in your test target and Xcode will automatically create multiple clones of the selected simulator. It’s important to note that Xcode divides tests into classes, not an equal number of simulators. For example: you have 3 classes with tests:

  • The first contains – 3;

  • The second contains – 2;

  • The third contains – 10.

In this case, xcode will create three simulators and run them on each class with tests. Xcode will not split them equally for each simulator.

Parallelizing tests on CI

As an example, consider the implementation of multi_scan is a plugin for fastlane.

Imagine a situation that we have 40 tests, and we want to run them on 4 simulators.

multi_scan(
  project: 'yourProjectPath',
  scheme: 'SwiftRadioUITests',
  fail_build: false,
	device: 'iPhone 8'
  batch_count: 4,
  parallel_testrun_count: 4
)

We are interested in the parameters:

  • batch_count – into how many parts it is worth dividing the total number of tests to run on the simulator. For example, you have 40 tests, and we specify a value of 4, then 10 tests will run on each simulator.

  • parallel_testrun_count is the number of simulators to run tests in parallel. There are no limits on the number, but I advise you to use common sense and test how many simulators your build agent can withstand.

Specify the same numbers in these two parameters.

Now let’s imagine another example in which we need to explicitly specify tests for each simulator.

multi_scan(
  workspace: 'yourProjectPath',
  scheme: 'SwiftRadioUITests',
  fail_build: false,
	destination: 'platform=iOS Simulator,name=iPhone 8,OS=13.1'
  parallel_testrun_count: 2,
  batches: [
    [
      'SwiftRadioUITests/RegressionTests/testExample1',
      'SwiftRadioUITests/RegressionTests/testExample2',
      'SwiftRadioUITests/RegressionTests/testExample3'
    ],
    [
      'SwiftRadioUITests/RegressionTests/testExample4',
      'SwiftRadioUITests/RegressionTests/testExample5',
      'SwiftRadioUITests/RegressionTests/testExampl'
    ]
  ]
)

Let’s pay attention to the parameters:

  • batches is an array with your tests. It should be used if you want to explicitly specify which tests should be run on each simulator. In the array, you must specify the path to your test according to the following scheme: Test target name/ Class name with tests/ Test name.

  • destination is an argument where you can specify a specific one: simulator, physical device, mac.

Let’s analyze in more detail destination:

Key

Description

Example values

platform

Supported platform

iOS Simulator, tvOS Simulator, watchOS Simulator

name

The full name of the simulator, which can be found in “Devices and Simulators”

iPhone 8, Apple TV, Apple Watch SE – 40mm

id

UDID of your device. UDID can be obtained using the simctl utility

6DC4A7BA-EA7F-40D6-A327-A0A9DF82F7F6

OS

iOS version, tvOS, watchOS. Specify the version supported by Xcode

OS=13.1

A few examples to understand destination:

  1. We indicate the iPhone X simulator on OS version 13.1.

    -destination 'platform=iOS Simulator,name=iPhone X,OS=13.1'
    
  2. Specify the simulator using the simulator’s UDID.

    -destination 'platform=iOS Simulator,id=6DC4A7BA-EA7F-40D6-A327-A0A9DF82F7F6'
    

The most important

  • Parallelization of tests allows you to speed up the passage of your tests several times!

  • You can do this locally through xcode, but it will divide them by class, not by an equal number.

  • To parallelize tests on ci, use the multi_scan plugin for fastlane. The implementation is very simple, and there is a lot of profit.

Test parallelization is a simple and very useful tool for speeding up your tests, which I recommend everyone to use.

Useful articles on this topic:


Interested in automation on iOS? Subscribe to my telegram channelin which I publish materials that will be useful to both beginners and experienced iOS automators.

Similar Posts

Leave a Reply