Maestro Flakiness: Source Code Analysis

We analyzed Maestro’s source code. Here’s what “built-in flakiness handling” actually does — and where it falls short.

Maestro markets itself as a test framework that “embraces the instability of mobile applications.” But what does that actually mean in code? I dug into the source to find out.

The Marketing Promise

From Maestro’s documentation:

“UI elements will not always be where you expect them, screen tap will not always go through, etc. Maestro embraces the instability of mobile applications and devices and tries to counter it.”

“No need to pepper your tests with sleep() calls. Maestro knows that it might take time to load the content and automatically waits for it (but no longer than required)."

Sounds great. Let’s see what the code actually does.

1. Element Finding: The Hardcoded 17-Second Timeout

When you write tapOn: "Login", Maestro doesn't look once and fail. It polls continuously. But for how long?

Source: Orchestra.kt

class Orchestra(
    private val maestro: Maestro,
    private val lookupTimeoutMs: Long = 17000L,        // Hardcoded: 17 seconds
    private val optionalLookupTimeoutMs: Long = 7000L, // Hardcoded: 7 seconds for optional
)

What this means:

Every element lookup waits up to 17 seconds by default. Optional elements wait 7 seconds. You cannot change this per-command.

Want to wait 30 seconds for a slow API response? Too bad. Want to fail-fast in 3 seconds for performance testing? Also no.

2. The Polling Mechanism: Simple but Rigid

Source: MaestroTimer.kt

fun <T> withTimeout(timeoutMs: Long, block: () -> T?): T? {
    val endTime = System.currentTimeMillis() + timeoutMs

    do {
        val result = block()
        if (result != null) {
            return result
        }
    } while (System.currentTimeMillis() < endTime)
    return null
}

This is a tight polling loop with no configurable delay between iterations. It just hammers the view hierarchy until it finds the element or times out.

Compare to Appium’s FluentWait where you can set custom timeout, custom polling interval, and exceptions to ignore.

3. Tap Retries: Only 2 Attempts, Non-Configurable

Maestro has a clever feature: if a tap doesn’t change the UI, it retries. But how many times?

Source: Maestro.kt

private fun getNumberOfRetries(retryIfNoChange: Boolean): Int {
    return if (retryIfNoChange) 2 else 1  // That's it. Just 2.
}

Two retries. Hardcoded. Not configurable.

And the explicit retry command in YAML? Also capped:

Source: Orchestra.kt

private const val MAX_RETRIES_ALLOWED = 3

So even if you write retry: 10 in your YAML, you get 3 maximum.

4. Screenshot Diff Threshold: Magic Number 0.5%

When Maestro decides whether a tap “worked,” it compares screenshots pixel-by-pixel:

Source: Maestro.kt

private const val SCREENSHOT_DIFF_THRESHOLD = 0.005  // 0.5% pixel difference

If less than 0.5% of pixels changed, Maestro assumes the tap failed and retries.

Problems:

A small spinner animation might trigger false “success.” A full-screen color change of 0.4% would be considered “no change.” You can’t adjust this threshold.

5. Wait For App To Settle: Fixed 200ms Polling

After interactions, Maestro waits for the UI to stabilize:

Get Om Narayan’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Source: ScreenshotUtils.kt

repeat(10) {  // Poll up to 10 times
    val hierarchyAfter = viewHierarchy(driver)
    if (latestHierarchy == hierarchyAfter) {
        val isLoading = latestHierarchy.root.attributes
            .getOrDefault("is-loading", "false").toBoolean()
        if (!isLoading) {
            return hierarchyAfter
        }
    }
    latestHierarchy = hierarchyAfter

    MaestroTimer.sleep(MaestroTimer.Reason.WAIT_TO_SETTLE, 200)  // Fixed 200ms
}

Hardcoded values:

10 iterations maximum. 200ms between polls. Total maximum wait: 2 seconds.

Fast app? You’re wasting 200ms per poll. Slow app? 2 seconds might not be enough.

6. waitUntilVisible: 10 Seconds, Take It or Leave It

Source: Maestro.kt

private fun waitUntilVisible(element: UiElement): ViewHierarchy {
    var hierarchy = ViewHierarchy(TreeNode())
    repeat(10) {  // 10 attempts
        hierarchy = viewHierarchy()
        if (!hierarchy.isVisible(element.treeNode)) {
            LOGGER.info("Element is not visible yet. Waiting.")
            MaestroTimer.sleep(MaestroTimer.Reason.WAIT_UNTIL_VISIBLE, 1000)  // 1 second
        } else {
            LOGGER.info("Element became visible.")
            return hierarchy
        }
    }
    return hierarchy
}

10 polls × 1 second = 10 seconds maximum. Not configurable.

7. Platform Differences: Android vs iOS

Maestro handles settling differently per platform:

Android (AndroidDriver.kt):

// Checks if window is still updating
val windowUpdating = blockingStubWithTimeout.isWindowUpdating(...)

iOS (IOSDriver.kt):

// Uses screenshot comparison to detect animation end
val didFinishOnTime = waitUntilScreenIsStatic(SCREEN_SETTLE_TIMEOUT_MS)

Different approaches, different reliability characteristics, same lack of configurability.

Summary: The Hardcoded Reality

Parameter Value Configurable? Element lookup timeout 17,000ms No Optional element timeout 7,000ms No Tap retry attempts 2 No Max retry command 3 No Screenshot diff threshold 0.5% No Settle polling interval 200ms No Settle max iterations 10 No waitUntilVisible timeout 10,000ms No

The Verdict

Maestro’s “built-in flakiness handling” is real — it does more than raw XCUITest or Espresso. But it’s a one-size-fits-all solution with hardcoded values.

The code is clean and the approach is sound. The problem is the lack of escape hatches. When the defaults don’t work for your app, you’re stuck.

This isn’t necessarily bad — it’s a trade-off for simplicity. But the marketing implies more intelligence and adaptability than the code delivers.

Key Source Files

If you want to explore further:

• Orchestra.kt — Command execution and timeouts
• Maestro.kt — Tap logic and retries
• MaestroTimer.kt — Polling primitives
• ScreenshotUtils.kt — Screenshot comparison
• Commands.kt — Command definitions

All source files are available on Maestro’s GitHub repository.

We’re Not Just Pointing Out Problems

We love Maestro’s YAML syntax — it’s the best thing to happen to mobile test automation in years. Simple, readable, version-control friendly.

But the execution engine has real limitations. Hardcoded timeouts. No configurability. Platform inconsistencies.

So we’re building something to fix it.

An open-source engine that runs Maestro YAML tests on Appium’s battle-tested infrastructure. Configurable timeouts. Real device support. No magic numbers.

Watch this space.

Learn More →

Originally published at devicelab.dev

This is Part 1 of the Maestro Deep Dive series.