Ultron is the easiest UI test framework for Android

In most cases, when it comes to test automation on android, many people recommend the popular kaspresso framework. Are there any alternatives?

Today we will talk about such a framework as Ultron for android ui testing, something different from kaspresso with its own approach. On the Internet, few people know about this framework. The framework is based on Espresso, UI Automator and Compose UI testing framework, written by Alexey Tyurin, who has spoken on Heisenbug many times. Therefore, it would be nice to highlight Ultron on Habré.

What are the benefits of using a framework?

  • Simple Syntax

  • Full control over any action or assertion

  • Architectural Approach to UI Test Development

  • Mechanism for connecting preconditions and postconditions

Let’s take a closer look at this framework.

Approach to Assertions and Actions

Let’s take a look at examples of actions and assertions on elements:

Espresso

onView(withId(R.id.send_button)).check(isDisplayed()).perform(click())

Ultron

withId(R.id.send_button).isDisplayed().click()

As you can see from the example, the ultron syntax allows you to more concisely describe actions on elements. All Ultron operations have the same names as espresso. Ultron also provides a list of additional operations.

Let’s see what it would look like using the page object pattern:

object SomePage : Page<SomePage>() {
    private val button = withId(R.id.button1)
    private val eventStatus = withId(R.id.last_event_status)

    fun checkEventStatusText(expectedEventText: String) {
         button.click()
         eventStatus.hasText(expectedEventText)
    }
}

Everything looks extremely simple and clear. You just need to specify matchers for your elements and call the necessary actions on them.

Recycler view actions approach

Espresso

onView(withId(R.id.recycler_friends))
    .perform(
        RecyclerViewActions
            .scrollTo<RecyclerView.ViewHolder>(hasDescendant(withText("Janice")))
    )
    .perform(
        RecyclerViewActions
            .actionOnItem<RecyclerView.ViewHolder>(
                hasDescendant(withText("Janice")),
                click()
            )
        )

Ultron

withRecyclerView(R.id.recycler_friends)
    .item(hasDescendant(withText("Janice")))
    .click()

Ultron provides a very comfortable API and all the code fits in a couple of lines concisely.
Let’s take a look at best practice using the page object:

object FriendsListPage : Page<FriendsListPage>() {
    // param loadTimeout в мс указывает время ожидания загрузки элементов RecyclerView
    val recycler = withRecyclerView(R.id.recycler_friends, loadTimeout = 10_000L) 
    fun someStep(){
        recycler.assertEmpty()
        recycler.hasContentDescription("Description")
    }
}

API provided by the framework for working with the recycler:

recycler.item(position = 10, autoScroll = true).click() // найти 10 айтем по позиции и проскроллить до него
recycler.item(matcher = hasDescendant(withText("Janice"))).isDisplayed()
recycler.firstItem().click() // взять первый айтем ресайклера
recycler.lastItem().isCompletelyDisplayed()

// если невозможно указать уникальный матчер для целевого элемента
val matcher = hasDescendant(withText("Friend"))
recycler.itemMatched(matcher, index = 2).click() // вернуть 3 совпадающий по матчеру элемент начиная с 0 индекса
recycler.firstItemMatched(matcher).isDisplayed()
recycler.lastItemMatched(matcher).isDisplayed()
recycler.getItemsAdapterPositionList(matcher) // вернуть все позиции элементов по матчеру

It’s worth noting that you don’t have to worry about scrolling to the element. By default, autoscroll to elements is set to true in every recycler element method.

If you have complex recycler items (have nested views) and need to work with them, then you can use the following approach:

object FriendsListPage : Page<FriendsListPage>() {

    // задаем матчер для ресайклера
    val recycler = withRecyclerView(R.id.recycler_friends)

    // создаем функцию для получения айтема ресайклера
    fun getListItem(contactName: String): FriendRecyclerItem {
        return recycler.getItem(hasDescendant(allOf(withId(R.id.tv_name), withText(contactName))))
    }

    // создаем класс, который мы подразумеваем как айтем ресайклера withRecyclerView(R.id.recycler_friends)
    class FriendRecyclerItem : UltronRecyclerViewItem() {
        val avatar by lazy { getChild(withId(R.id.avatar)) }
        val name by lazy { getChild(withId(R.id.tv_name)) }
        val status by lazy { getChild(withId(R.id.tv_status)) }
    }

    // используем getListItem(name) для доступа к status
    fun assertStatus(name: String, status: String) = apply {
        getListItem(name).status.hasText(status).isDisplayed()
    }
}

With the approach of creating a class like FriendRecyclerItem, we can use this api:

recycler.getItem<FriendRecyclerItem>(position = 10, autoScroll = true).status.hasText("UNAGI")
recycler.getItem<FriendRecyclerItem>(matcher = hasDescendant(withText("Janice"))).status.textContains("Oh. My")
recycler.getFirstItem<FriendRecyclerItem>().avatar.click() // взять первый ресайклер айтем
recycler.getLastItem<FriendRecyclerItem>().isCompletelyDisplayed()

// если невозможно указать уникальный матчер для целевого элемента
val matcher = hasDescendant(withText(containsString("Friend")))
recycler.getItemMatched<FriendRecyclerItem>(matcher, index = 2).name.click() // вернуть 3 совпадающий по матчеру элемент начиная с 0 индекса
recycler.getFirstItemMatched<FriendRecyclerItem>(matcher).name.hasText("Friend1")
recycler.getLastItemMatched<FriendRecyclerItem>(matcher).avatar.isDisplayed()

Espresso WebView operations

Espresso

onWebView()
    .withElement(findElement(Locator.ID, "text_input"))
    .perform(webKeys(newTitle))
    .withElement(findElement(Locator.ID, "button1"))
    .perform(webClick())
    .withElement(findElement(Locator.ID, "title"))
    .check(webMatches(getText(), containsString(newTitle)))

Ultron

id("text_input").webKeys(newTitle)
id("button1").webClick()
id("title").hasText(newTitle)

Since working with webview is not such a frequent case, all the features of the framework can be looked at dock.

UI Automator operations

Espresso

val device = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation())
device
    .findObject(By.res("com.atiurin.sampleapp:id", "button1"))
    .click()

Ultron

byResId(R.id.button1).click()

The functionality of the framework through UI Automator is described here.

Simple compose operations

Compose framework

composeTestRule.onNode(hasTestTag("Continue")).performClick()
composeTestRule.onNodeWithText("Welcome").assertIsDisplayed()

Ultron

hasTestTag("Continue").click()
hasText("Welcome").assertIsDisplayed()

Compose interaction is described in wiki.

Compose list operations

Compose framework

val itemMatcher = hasText(contact.name)
composeRule
    .onNodeWithTag(contactsListTestTag)
    .performScrollToNode(itemMatcher)
    .onChildren()
    .filterToOne(itemMatcher)
    .assertTextContains(contact.name)

Ultron

composeList(hasTestTag(contactsListTestTag))
    .item(hasText(contact.name))
    .assertTextContains(contact.name)

Working with LazyColumn/LazyRow is described here.

Framework stability

What determines the stability of tests in the framework?
The framework catches the list of specified exceptions and tries to retry the operation within a certain timeout (5 seconds by default).

UltronWrapperException::class.java,
UltronException::class.java,
PerformException::class.java,
NoMatchingViewException::class.java,
AssertionFailedError::class.java,
RuntimeException::class.java

You can customize the list of handled exceptions, add new ones or remove existing ones via UltronConfig, as well as change the default timeout. You can also specify a custom timeout for any operation – withTimeout(..) (will be discussed below).

Let’s look at what features the framework has:

Custom timeout for any operation

withId(R.id.last_event_status).withTimeout(10_000).isDisplayed()

This feature allows you to set a timeout within which an action or assertion will be performed on an element (5 seconds by default)

There are 2 approaches to using the withTimeout(..) method:
Specify withTimeout(..) at the place of the element declaration and it will be applied to all operations with this element

object SomePage : Page<SomePage>() {
    private val eventStatus = withId(R.id.last_event_status).withTimeout(10_000)
}

Specify withTimeout(..) inside the step to keep the interaction with the element for the specified time. This timeout value will only be applied once per operation.

object SomePage : Page<SomePage>() {
    fun someLongUserStep(expectedEventText: String){
         longRequestButton.click()
         eventStatus.withTimeout(20_000).hasText(expectedEventText)
    }
}

That is, the hasText(expectedEventText) check on the eventStatus element will run for 20 seconds until it succeeds. If the operation fails within 20 seconds, the test will fail with a corresponding error.

Boolean operation result

The framework has the isSuccess method, which allows you to get the result of any operation as a boolean value. If false, the isSuccess check may take too long (5 seconds by default). Therefore, it is reasonable to specify a custom timeout for some operations in conjunction with the isSuccess function.

For example:

val isButtonDisplayed = withId(R.id.button).isSuccess { withTimeout(2_000).isDisplayed() }
if (isButtonDisplayed) {
    // какие-то необходимые проверки и действия
}

Extending the framework with your custom ViewActions and ViewAssertions

Under the hood, all Ultron espresso operations are described in the UltronEspressoInteraction class. to create your custom actions and checks you just need to extend this class using kotlin extension function e.g.

fun <T> UltronEspressoInteraction<T>.appendText(text: String) = apply {
    executeAction(
        operationBlock = getInteractionActionBlock(AppendTextAction(text)),
        name = "Append text '$text' to ${getInteractionMatcher()}",
        description = "${interaction!!::class.simpleName} APPEND_TEXT to ${getInteractionMatcher()} during $timeoutMs ms",
    )

AppendTextAction is a custom ViewAction

class AppendTextAction(private val value: String) : ViewAction {
    override fun getConstraints() = allOf(isDisplayed(), isAssignableFrom(TextView::class.java))
    override fun perform(uiController: UiController, view: View) {
        (view as TextView).apply {
            this.text = "$text$value"
        }
        uiController.loopMainThreadUntilIdle()
    }
    ...
}

To make your custom operation 100% native to the Ultron framework, you need to add 3 more lines.

//support action for all Matcher<View>
fun Matcher<View>.appendText(text: String) = UltronEspressoInteraction(onView(this)).appendText(text)

//support action for all ViewInteractions
fun ViewInteraction.appendText(text: String) = UltronEspressoInteraction(this).appendText(text)

//support action for all DataInteractions
fun DataInteraction.appendText(text: String) =  UltronEspressoInteraction(this).appendText(text)

Finally, we can use our custom appendText operation:

withId(R.id.text_input).appendText("some text to append")

Custom operation assertions

There are cases when the click() operation does not go through the element and the test fails (more precisely, it passes visually, but for some reason the onClick listener does not fire or the click action is not sent) or you need to make some necessary assertion on the click to make sure that the click had an effect. You can use the following solution:

button.withAssertion("Assert smth is displayed") {
    title.isDisplayed()
}.click()

“Assert smth is displayed” is the text you will see if an exception occurs.
But this is optional and can be written like this:

button.withAssertion {
    title.isDisplayed()
}.click()

By default, all Ultron operations inside an assertion block are not logged to logcat, but you will see the reason in case of an exception anyway.

If you want information to be logged to logcat, use the isListened parameter

button.withAssertion(isListened = true) { .. }

There is also one remark to be made here, about the timeouts of the assertion and action operations.

  • withAssertion {..} can double the time before an exception is thrown. This is because the action is executed at least twice. And the assertion block is also executed twice. This is necessary to determine the true exception on which the action falls.

  • If the action succeeds but the assertion fails, then the action is repeated. You can have multiple action and assertion interactions.

  • You can limit the assertion time, for example:

button.withAssertion {
    title.withTimeout(3_000L).isDisplayed()
}.click()

You can also still increase the overall timeout for the entire operation:

button.withTimeout(10_000L).withAssertion {
    title.withTimeout(2_000L).isDisplayed()
}.click()

RuleSequence + SetUps & TearDowns for tests

  • control the fulfillment of pre- and post-conditions of each test

  • control the moment of launching the activity

  • no need to use @Before and @After, you can replace them with lambda expressions of SetUpRule or TearDownRule objects

  • combine your test conditions using annotations

RuleSequence

The RuleSequence rule is an advanced replacement for the JUnit 4 RuleChain. This allows you to control the order in which the rules are executed.

RuleSequence has no problems with class inheritance, and this is its original idea.

The order in which rules are executed depends on the order in which they are added. The RuleSequence contains three lists of rules with their own priority.

first – rules from this list will be executed first
normal – rules will be added to this list by default
last – rules from this list will be executed last

You can read more about the order of execution here.

It is recommended to create RuleSequence in BaseTest. You will be able to add rules to RuleSequence in BaseTest and subclasses of BaseTest.

abstract class BaseTest {
    val setupRule = SetUpRule().add {
            // предусловия для тестов или автологин в приложение
        }

    @get:Rule
    open val ruleSequence = RuleSequence(setupRule)
}

Adding rules is best done in the init block of your test class:

class DemoTest : BaseTest() {
    private val activityRule = ActivityScenarioRule(MainActivity::class.java)

    init {
        ruleSequence.addLast(activityRule)
    }
}

When using RuleSequence (as was the case with RuleChain), you do not need to specify the @get:Rule annotation on other rules.

Full examples can be checked here:

SetUpRule

This rule allows you to specify lambda expressions that must be called before running the test. Moreover, in combination with the RuleSequence setting, lambda expressions can be called before the activity starts.

Preconditions for Tests

We add a lambda to SetUpRule without the string key in parentheses and it will be executed before every test in the class.

open val setupRule = SetUpRule()
    .add {
        Log.info("Login valid user will be executed before any test is started")
        AccountManager(InstrumentationRegistry.getInstrumentation().targetContext).login(
            CURRENT_USER.login, CURRENT_USER.password
        )
    }

Precondition for a specific test

  1. add lambda with string key to (..) in SetUpRule

  2. add @SetUp(string key) annotation to desired test

setupRule.add(FIRST_CONDITION){ 
    Log.info("$FIRST_CONDITION setup, executed for test with annotation @SetUp(FIRST_CONDITION)")  
}

@SetUp(FIRST_CONDITION)
@Test
fun someTest() {
    // степы теста
}

companion object {
    private const val FIRST_CONDITION = "Название предусловия"
}

Don’t forget to add the SetUpRule rule to the Rule Sequence.

ruleSequence.add(setupRule)

TearDownRule

This rule allows you to specify lambda expressions that are sure to be called after the test completes.

Postcondition for all tests

Add a lambda to the TearDownRule without the parenthesized string key and it will be executed after every test in the class.

open val tearDownRule = TearDownRule()
    .add {
        AccountManager(InstrumentationRegistry.getInstrumentation().targetContext).logout()
    }

Postcondition for a specific test

  1. add lambda with string key to (..) in TearDownRule

  2. add @TearDown(string key) annotation to desired test

tearDownRule.add (LAST_CONDITION){ 
    Log.info("$LAST_CONDITION tearDown, executed for test with annotation @TearDown(LAST_CONDITION)")  
}

@TearDown(LAST_CONDITION)
@Test
fun someTest() {
    // степы теста
}

companion object {
    private const val LAST_CONDITION = "Название постусловия"
}

Don’t forget to add the TearDownRule to the Rule Sequence.

ruleSequence.addLast(tearDownRule)

Solving non-trivial Espresso exceptions

One of the magic bugs that occurs during test execution can be Waited for the root of the view hierarchy to have window focus and not request layout for 10 seconds. In this framework, this problem is solved and here is how it looks:

val toolbarTitle = withId(R.id.toolbar_title)

fun assertToolbarTitleWithSuitableRoot(text: String) {
    toolbarTitle.withSuitableRoot().hasText(text)
}

The withSuitableRoot() function looks for the visible root view where your matching matcher view is and assigns it to your ViewInteraction. If one is not found within the Espresso internal timeout, then a native NoMatchingRootException error will be thrown that no root view is suitable for the desired view for your matcher. In this context, this will mean that all visible root views do not have your desired view.

Cons of the framework

  • No adbServer, only shell

  • The framework does not have many authors and is not being developed as actively as we would like

Conclusion

In this article, I tried to highlight the key features of the framework and the base that everyone is interested in at the stage of choosing a ui automation framework for my project. Thank you for reading / scrolling to the end and I hope that in this article you were able to find something interesting for yourself.

Join the group if you have questions Ultron telegram group.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *