Features and pitfalls

In order for the user to be able to fully interact with the interface, we, mobile developers, solve a seemingly simple task every day: the input fields of the form on the screen must always be visible and located above the keyboard.

Android has a flag for this. android:windowSoftInputMode="adjustResize"but it shrinks the application window without taking into account the keyboard animation, so the user sees an empty field for a split second while the keyboard opens:

To fix this behavior, the developers of Jetpack Compose proposed a new way to interact with the keyboard: edge-to-edge mode in combination with Modifier.imePadding().

So, let's apply this modifier to the main one Column our form:

Column(
    Modifier
        .padding(horizontal = 32.dp)
        .imePadding()
        .verticalScroll(rememberScrollState())
) {
    Spacer(Modifier.height(it.calculateTopPadding()))
    32.dp.VerticalSpacer()
    AccountIcon()
    32.dp.VerticalSpacer()
    Text(
        text = stringResource(Res.string.create_your_account),
        style = MaterialTheme.typography.titleLarge,
        fontSize = 32.sp,
        modifier = Modifier.align(Alignment.CenterHorizontally)
    )
    // ...
}

Now the behavior is much better: the interface smoothly adapts to the opening keyboard, and if it overlaps the input field, the selected field smoothly “moves” up exactly as much as necessary:

It would seem that this is what we were trying to achieve. However, if we want to “poke around” our interface on iOS and transfer the application to Compose Multiplatform, a new nuance will arise: in new versions of iOS, the keyboard is translucent with the typical Apple Gaussian blur. But our Modifier.imePadding() – this is precisely the indentation (the library developers are not lying to us in the method naming), so the form interface will not “show through” under the translucent keyboard:

Unfortunately, there are no out-of-the-box methods for solving this problem in Compose. To make our interface look more native, we will write our own wrapper that will bind the scroll state to the keyboard height. I called it ImeAdaptiveColumn. It took quite a bit of experimenting with it, but the result was quite satisfactory.

class FocusedAreaEvent {
    var id: String by mutableStateOf("")
    var rect: Rect? by mutableStateOf(null)
    var spaceFromBottom: Float? by mutableStateOf(null)
}

class FocusedArea {
    var rect: Rect? = null
}

data class History<T>(val previous: T?, val current: T)

// emits null, History(null,1), History(1,2)...
fun <T> Flow<T>.runningHistory(): Flow<History<T>> =
    runningFold(
        initial = null as (History<T>?),
        operation = { accumulator, new -> History(accumulator?.current, new) }
    ).filterNotNull()

data class ClickData(
    val unconsumed: Boolean = true,
    val offset: Offset = Offset.Zero
)

@OptIn(ExperimentalFoundationApi::class)
@Composable
fun ImeAdaptiveColumn(
    scrollState: ScrollState = rememberScrollState(),
    scrollable: Boolean = true,
    modifier: Modifier = Modifier,
    horizontalPadding: Dp = 16.dp,
    content: @Composable ColumnScope.() -> Unit
) {
    val screenHeight = LocalScreenSize.height
    val imeHeight by rememberUpdatedState(imeHeight())

    var clickData by remember { mutableStateOf(ClickData()) }
    val focusedAreaEvent = remember { FocusedAreaEvent() }
    val focusedArea = remember { FocusedArea() }
    LaunchedEffect(
        key1 = focusedAreaEvent.id
    ) {
        if (focusedAreaEvent.id.isNotEmpty()) {
            focusedAreaEvent.spaceFromBottom?.let { capturedBottom ->
                snapshotFlow { imeHeight }
                    .runningHistory()
                    .collectLatest { (prev, height) ->
                        val prevHeight = prev ?: 0
                        if (height > capturedBottom) {
                            if (prevHeight < capturedBottom) {
                                val difference = height - capturedBottom
                                scrollState.scrollBy(difference)
                            } else {
                                val difference = height - prevHeight
                                scrollState.scrollBy(difference.toFloat())
                            }
                        } else {
                            if (prevHeight > capturedBottom) {
                                val difference = prevHeight - capturedBottom
                                scrollState.scrollBy(-difference)
                            }
                        }
                    }
            }
        }
    }

    Column(
        modifier = modifier
            .onFocusedBoundsChanged { coordinates ->
                coordinates?.boundsInWindow()?.let {
                    focusedArea.rect = it
                    if (clickData.unconsumed && clickData.offset in it) {
                        focusedAreaEvent.run {
                            id = uuid()
                            rect = it
                            spaceFromBottom = screenHeight - it.bottom
                        }
                        clickData = clickData.copy(unconsumed = false)
                    }
                }
            }
            .pointerInput(Unit) {
                awaitEachGesture {
                    val event = awaitPointerEvent(PointerEventPass.Main)
                    // If the software keyboard is hidden, register a new focused area.
                    if (event.type == PointerEventType.Press && imeHeight == 0) {
                        val offset = event.changes.firstOrNull()?.position ?: Offset.Zero
                        clickData = ClickData(
                            unconsumed = true,
                            offset = offset
                        )
                    }
                }
            }
            .background(MaterialTheme.colorScheme.surface)
            .padding(horizontal = horizontalPadding)
            .verticalScroll(scrollState, enabled = scrollable),
        content = content
    )
}

@Composable
fun imeHeight() = WindowInsets.ime.getBottom(LocalDensity.current)

What's going on here? My goal was to write a Composable API that would adapt to the keyboard regardless of the content. In other words, it's a “content-agnostic” component, where the user of the code doesn't need to call anything extra on their side, everything happens under the hood.

For example, you can now call ImeAdaptiveColumn Thus:

ImeAdaptiveColumn(horizontalPadding = 32.dp) {
    Spacer(Modifier.height(it.calculateTopPadding()))
    32.dp.VerticalSpacer()
    AccountIcon()
    32.dp.VerticalSpacer()
    Text(
        text = stringResource(Res.string.create_your_account),
        style = MaterialTheme.typography.titleLarge,
        fontSize = 32.sp,
        modifier = Modifier.align(Alignment.CenterHorizontally)
    )
    32.dp.VerticalSpacer()
    SignupFormTextField(
        label = stringResource(Res.string.first_name_title),
        placeholder = stringResource(Res.string.first_name_placeholder)
    )
    // ...
}

This convenience is achieved through two main APIs available in Compose: Modifier.pointerInput() And Modifier.onFocusBoundsChanged().

Let's break down step by step what happens when a user taps on an input field:

1. The click event propagates up the UI component tree. Our modifier method (pointerInput) listens for this event if the keyboard is currently closed and stores the click coordinate in a variable (clickData.offset).

2. The focused area changes, the onFocusBoundsChanged callback is called, which puts the click event into the focusedAreaEvent.

3. For each focusedAreaEvent, a LaunchedEffect is launched, which tracks changes in the keyboard height and scrolls our Column if this height overlaps the coordinates received in the focusedAreaEvent.

So we get this result on iOS (which is best seen if you turn on the dark theme). The first video is the interface with the normal imePaddingthe second – with ImeAdaptiveColumn.

This not-so-complicated trick once again shows how differently technologies designed for one platform (Android) behave when ported to another (iOS). And yet, I think that this one small trick is worth it to bring the user experience in Compose Multiplatform closer to native iOS technologies.