How to Run Espresso Tests on Local and Cloud Android Devices?
I have lost a lot of hours to a test that passes on my Pixel and fails on a Samsung ten minutes later. My first instinct used to be that Samsung had done something strange, and I was usually wrong.
So now I hold the app APK and the Espresso test APK still and change only the device underneath them. One note before the commands, because Espresso runs inside your app process as Android instrumentation and cannot run on an iPhone. You would reach for XCUITest there.
Prepare One Espresso Build for Multiple Devices
If every device gets its own build, I cannot tell whether the device or the build caused a difference. So I build once and reuse the same two files everywhere.
./gradlew :app:assembleDebug :app:assembleDebugAndroidTestOn a standard app module, Gradle drops the outputs in these two paths. Product flavors move them.
app/build/outputs/apk/debug/app-debug.apk
app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apkNone of this works until you declare the runner, so I check that first.
android {
defaultConfig {
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
}
}I write down the versions Gradle resolved rather than the ones the docs print. Android’s Espresso setup page still shows 3.6.1, while the current stable release is 3.7.0 from 30 July 2025.
./gradlew :app:dependencies \
--configuration debugAndroidTestRuntimeClasspath- I turn on Test Orchestrator and accept the slower run: Setting execution ‘ANDROIDX_TEST_ORCHESTRATOR’ with clearPackageData: ‘true’ gives every test its own Instrumentation invocation and clears shared state, which kills the most common reason a suite passes on my desk phone and fails on a freshly provisioned one.
- Check the Orchestrator version too: It has sat at 1.6.1 since 31 June 2024 while the rest of the line shipped in July 2025, so plenty of projects run a mismatched pair.
Run Espresso Tests on a Connected Android Device
I always start with a phone on my desk, because adb gives me a fast debugging loop. Cloud devices win on breadth and lose on the first hour of diagnosis.
Connect and Identify Your Pixel or Samsung Device
Before I run anything I record what each device actually is, because I quote that record when a failure turns out to be environmental.
adb devices -l
adb -s PIXEL_SERIAL shell getprop ro.product.model
adb -s PIXEL_SERIAL shell getprop ro.build.version.sdkThen I collect the four values that explain most visibility failures, on both phones.
adb -s PIXEL_SERIAL shell wm size
adb -s PIXEL_SERIAL shell wm density
adb -s PIXEL_SERIAL shell settings get system font_scale
adb -s PIXEL_SERIAL shell getprop ro.product.localeDensity and font scale decide whether a view sits fully on screen, and that decides whether Espresso’s visibility check passes before your assertion runs.
Install Both APKs and Execute the Test Suite
Installation takes two commands with different flags, and the flags trip people up. Install the app APK, then the test APK, then confirm the runner.
adb -s PIXEL_SERIAL install -r -g app-debug.apk
adb -s PIXEL_SERIAL install -r -t app-debug-androidTest.apk
adb -s PIXEL_SERIAL shell pm list instrumentation- The test APK needs the -t flag: Gradle marks test APKs as testOnly, and skipping the flag fails the install in a way that later looks like a missing runner, which sends you hunting in the wrong place.
- I use -g to grant manifest permissions at install time: It clears out a category of OEM permission dialog, though I drop it when I am testing the permission flow itself.
- An empty pm list instrumentation means the test APK never installed: Fix that before you read your test code for faults.
The run needs -w, because that flag makes the shell wait for instrumentation to finish. When I compare two devices I pin a single method rather than running the whole suite.
adb -s PIXEL_SERIAL shell am instrument -w \
-e class com.example.tests.LoginTest#validLogin \
com.example.app.test/androidx.test.runner.AndroidJUnitRunnerThe option I almost never see anyone use is -e log true. It reports which tests your filters would select and runs none of them, so you can check a filter before it burns device time.
Fun fact: If you pass both package and class, am instrument gives the package filter precedence. A CI script that builds both from config runs the entire package and hands you a green pass for a class it never executed.
Execute the Same Tests on a Second Device
You have already run these commands, so point them at a different serial. I normalise the animation settings first, because two phones with different animation scales are not running the same test at all.
adb -s SAMSUNG_SERIAL shell settings put global window_animation_scale 0
adb -s SAMSUNG_SERIAL shell settings put global transition_animation_scale 0
adb -s SAMSUNG_SERIAL shell settings put global animator_duration_scale 0
adb -s SAMSUNG_SERIAL install -r -g app-debug.apk
adb -s SAMSUNG_SERIAL install -r -t app-debug-androidTest.apk
adb -s SAMSUNG_SERIAL shell am instrument -w \
-e class com.example.tests.LoginTest#validLogin \
com.example.app.test/androidx.test.runner.AndroidJUnitRunnerAndroid’s Espresso documentation names those same three settings under Developer options, so I apply them to the Pixel too. The APKs and the method now match, and only the hardware changed.
Run Espresso Tests Online When Local Devices Are Unavailable
Two phones on a desk works fine. Once I need six models across three Android versions on every pull request, hosted devices start paying for themselves.
Tools You Can Use to Run Espresso Tests Online
Four options cover most teams, and the right one usually follows wherever your CI runs. Here is what each does well and where I hit its limits.
Get Sean Wintermere’s stories in your inbox
Join Medium for free to get updates from this writer.
1. Build-managed Gradle devices
- Best for: Repeatable virtual devices you define in your build script and launch with a task such as ./gradlew pixel2api30DebugAndroidTest.
- Limitation: It manages emulators only, and ATD images run headless, so screenshot tests that need hardware rendering fail.
2. Firebase Test Lab
- Best for: Teams already on Google infrastructure, calling gcloud firebase test android run with — device model=Pixel7,version=33 and — test-targets.
- Limitation: The documentation tells you to run Orchestrator locally first because a misconfiguration eats quota, and the Gradle plugin still sits at 0.0.1-alpha05.
3. AWS Device Farm
- Best for: Teams whose CI already lives in AWS, where Dex Test Parser enumerates your tests so you can preview what the service will find.
- Limitation: That parser and your device pools drive test selection rather than the -e filters you use locally, so the two invocations stop resembling each other.
4. BrowserStack App Automate
- Best for: Moving a local reproduction onto hosted physical hardware unchanged, because the build API takes the same class, package, annotation and size filters you used over adb.
- Limitation: Sharding eats parallel slots multiplicatively, so four shards across three devices block twelve parallels, and the platform queues anything beyond your plan.
I use App Automate for the rest of this because that filter parity keeps the comparison honest.
Run the Espresso Test with BrowserStack App Automate
Three curl calls cover the flow, and the payload uses the same pair of APKs from the start.
curl -u "USERNAME:ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/app" \
-F "file=@app-debug.apk"
curl -u "USERNAME:ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/test-suite" \
-F "file=@app-debug-androidTest.apk"Both return a bs:// reference, which I pass into the build call with the same method I ran locally.
curl -u "USERNAME:ACCESS_KEY" \
-X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
-H "Content-Type: application/json" \
-d '{
"app": "bs://APP_HASH",
"testSuite": "bs://TEST_HASH",
"devices": ["Google Pixel 7-13.0", "Samsung Galaxy S23-13.0"],
"class": ["com.example.tests.LoginTest"],
"deviceLogs": true
}'- Device strings use a name and OS version format: A regular expression such as “Samsung Galaxy .*” grabs the first available match, which shortens queue time and costs reproducibility, so I pin the exact model when I chase a failure.
- Enable the logs you need first: Instrumentation logs and video run by default, while deviceLogs and networkLogs stay off until you ask for them.
Compare Local and Cloud Espresso Results
A comparison only tells me something when both sides ran the same tests and produced comparable artifacts. Gradle writes local results to two fixed paths, and CI consumes the XML.
app/build/reports/androidTests/connected/
app/build/outputs/androidTest-results/connected/For adb runs I add -r and take the raw instrumentation stream, which carries the class, method, status and stack trace in a format that parses the same way wherever it came from.
adb shell am instrument -w -r \
-e class com.example.tests.LoginTest#validLogin \
com.example.app.test/androidx.test.runner.AndroidJUnitRunnerDiffing the INSTRUMENTATION_STATUS lines lets one script check a local run against a cloud session, and teams skip this step more than any other. Keep the selection identical on both sides, because if one environment ran the full suite and the other ran a single method, your own setup created the difference you found.
Troubleshoot Tests That Fail on Only One Device
When a test fails on exactly one phone, I chase the mechanism rather than the manufacturer. Four causes cover nearly everything I have seen.
- Unregistered asynchronous work fails on the slower phone first: Espresso synchronises only with operations that post to the MessageQueue, so background work needs an IdlingResource that you register through IdlingRegistry.getInstance().register() in @Before. The phone that failed is usually the one where the call took longer.
- Leaked application state shows up as an order-dependent failure: My development phone already holds a session and cached data that a fresh cloud device has never seen, so one run with clearPackageData settles it.
- The display environment changes what Espresso can see: Screen size, density, font scale and API level all feed the visibility check, so a view that sits fully visible on one handset may need a scrollTo() on the other.
- Permission dialogs and autofill overlays intercept clicks: Installing with -g clears them when they have nothing to do with the scenario.
One gap caught me out. The Espresso Device API gives you onDevice().setScreenOrientation() and @RequiresDeviceMode, but Android’s documentation says it works only on virtual devices with emulator control enabled. On a physical Pixel or Samsung, including one hosted in a device cloud, you fall back to ActivityScenario.recreate() or UiAutomator.
Espresso Cross-Device Execution Checklist
I work through this before I treat two runs as comparable. Every item names a command to run or a value to write down.
- Build both APKs once and reuse them everywhere.
- Record the Espresso and runner versions Gradle resolved.
- Confirm both devices with adb devices -l.
- Record the API level, wm size, wm density, font_scale and locale.
- Zero all three animation scales on every device you compare.
- Install with -r -g for the app and -r -t for the test APK.
- Validate the filter with -e log true, then run one method on each phone.
- Upload the same APKs and pin the same filter in the cloud payload.
- Pin an exact device and OS version when you reproduce a failure.
- Diff the local XML against the cloud session, then check synchronisation and application state before you blame the hardware.
Final Thoughts
Adding devices multiplies whatever your suite already is. A test with an unregistered IdlingResource across ten hosted handsets gives you ten copies of the same flake and a dashboard that looks like real coverage. One Orchestrator flag fixes that, and nine more phones do not.
Starting narrow has held up for me. I run one test on one known device, write down every environment value, then add a second device with the same build and filter. The cloud matrix comes last, because by then a difference in results means something.
So here is my question for your suite. When it fails on the second device, do you know whether the device differs, or whether the first device covered for an unsynchronised call?
