Making Maestro E2E Part of Delivery: Two Platforms, Version Matching, and Evidence
A mobile E2E test often starts with a simple flow: open the app, tap a few controls, and assert that a screen appears. Integrating that flow into team delivery introduces different questions. Which native binary and JavaScript bundle were tested? Did a locally discovered case actually run in CI? Did a green workflow execute any tests? Can a remote teammate inspect what happened after a failure?
I introduced Maestro into an existing React Native application and connected the app repository, shared GitHub Actions workflows, and a separate E2E repository. The work covered Android and iOS execution, version selection, change-based suite selection, case conventions, and artifact retention.
This article explains those decisions using generic names and examples. It is an implementation case study, not a complete CI template.
Give every run an explicit identity
For an RN application that can update its JavaScript bundle, “the current branch” is not a complete test identity. I separated four inputs:
| Version | What it determines |
|---|---|
| App source and target commit | The changes we intend to validate |
| Native binary | Available modules, native permissions, and native runtime |
| JavaScript bundle | Executed business logic and test selectors |
| E2E code | Test steps, suite selection, and assertions |
The workflow definition has its own version as well. Changing its ref does not necessarily change the downloaded binary. Selecting a test branch does not mean the app contains the selectors added on that branch.
I recorded these values as explicit inputs or resolved outputs. When an existing binary was used as a fallback, each platform's artifact availability and actual source needed recording. Native code or dependency changes required a matching new binary; switching bundles could not replace that build.
The artifact also had to match the device. An iOS Simulator needs a simulator-compatible app, rather than a device IPA. Android checks included the application ID, build environment, and emulator ABI. Performing these checks before business tests made failures easier to classify.
Separate infrastructure from test policy
Technical flow; implementation and validation boundaries are explained below.
I avoided maintaining duplicate business-suite lists in the app repository and shared workflows, using the ownership boundaries shown above.
The E2E repository owned suite definitions, path mappings, ordering, and platform runners. Shared workflows handled invocation and environment preparation. Maintainers could find the test rules together when adding a case.
One important detail: local discovery of a YAML file does not prove CI executes it. An earlier CI implementation listed suites explicitly. Adding a case therefore required checking discovery, registration, and the actual remote execution plan.
Treat unavailable change information differently from an empty list
We mapped business paths to suites in E2E configuration and generated a plan from the changed files. Common preparation ran first; selected business suites followed their configured order.
| Input | Behavior |
|---|---|
| Explicit full run | Run all configured suites |
| Changes match a business path | Run the matching suites and common preparation |
| Changes match a shared-impact path | Run all configured business suites |
| Change retrieval succeeds without business matches | Still run the configured preparation flow |
| Change retrieval is unavailable | Fall back to all configured business suites and record why |
“No relevant changes” and “change information unavailable” are different states. Converting an API failure to [] could silently reduce test scope while producing an apparently normal run.
We also handled empty cross-repository refs explicitly. A caller can pass an empty string, so an input default alone is insufficient. Normalizing it before checkout prevents the E2E code from drifting to an unintended default branch.
Make business state observable
Each case had a small contract: known preconditions, one primary business behavior, cleanup limited to its own records, and a known final screen on success. Failures preserved the scene for diagnosis.
Suites owned shared preparation. Cases did not depend on business results left by earlier cases. Cleanup used exact IDs from the current case, rather than deleting unrelated records from a shared account.
We preferred stable testID selectors and checked them against the actual UI hierarchy. A selector in source code does not prove the installed app or loaded bundle contains it. Additional test-state nodes also needed an explicit test-environment boundary.
After a tap, the flow should wait for an observable result. This simplified navigation fragment uses illustrative selectors; the timeout needs adjustment for the real application:
- tapOn:
id: "example-open-detail"
- extendedWaitUntil:
visible:
id: "example-detail-ready"
timeout: 10000
This verifies screen readiness. A test intended to create an order or complete a payment must assert that business outcome as well. extendedWaitUntil proceeds when its condition is met; the timeout is an upper bound. Maestro wait command
Optional dialogs need state-aware handling. A dialog that may not appear should not become a mandatory assertion. Dismissing a dialog also does not prove the flow succeeded. If several asynchronous branches can appear, the bounded waiting logic must recognize those states and still finish with the intended business assertion.
Retries should be narrow. Observation and navigation can support bounded retries, but operations that create, charge, or delete should not simply be wrapped in a retry of the entire flow. Test retries do not provide business idempotency. Maestro retry documentation
Preserve evidence from passing runs too
Separate test status from evidence delivery
This job fragment distills the implementation principle. The illustrative runner must export diagnostics to runner-output/ and leave them there before returning. An outer always() cannot recover files the runner has already deleted. The script name is an integration contract, not a runner shipped in this repository.
# Fragment inside an existing job after device / App preparation.
- name: Execute tests and preserve the result
id: e2e
shell: bash
run: |
mkdir -p evidence
set +e
bash scripts/run-selected-suites.sh > evidence/console.log 2>&1
test_exit=$?
set -e
printf '%s\n' "$test_exit" > evidence/test-exit-code.txt
copy_exit=0
cp -R runner-output/. evidence/ || copy_exit=$?
printf '%s\n' "$copy_exit" > evidence/evidence-copy-exit-code.txt
if [ "$test_exit" -ne 0 ]; then
exit "$test_exit"
fi
exit "$copy_exit"
- name: Upload evidence on success or failure
if: ${{ always() }}
uses: actions/upload-artifact@v7
with:
name: e2e-${{ runner.os }}-${{ github.run_id }}-${{ github.run_attempt }}
path: evidence/
if-no-files-found: error
A failed test keeps its original exit code. If tests pass but copying evidence fails, delivery also fails, with both statuses recorded. Uploading in a separate step avoids losing that operation when the shell exits. An advisory wrapper must still report the recorded test outcome. Inputs follow upload-artifact.
The earlier implementation uploaded diagnostic directories only on failure and cleaned up successful output. When we wanted engineers to inspect successful runs, changing the upload condition alone was insufficient.
Two changes were necessary: runners copied successful output before cleanup, and workflow upload steps included both outcomes. Otherwise, the upload step could run after its input files had disappeared.
The diagram preserves the critical order: save results and evidence before cleaning temporary output.
Workflow status conditions allowed collection after success or failure. They could not guarantee complete evidence when a runner never started or was forcibly terminated. We still checked upload outcomes and artifact contents. GitHub status conditions, workflow artifacts
The retained evidence included target versions, the execution plan, result files, JUnit, console and debug logs, recordings, and relevant failure screenshots or UI hierarchy. This gave teammates enough context to review a run without attending it.
Keep advisory execution separate from test truth
The rollout used E2E as an advisory regression signal rather than a required merge gate. That was a delivery-policy decision; it did not change whether the test passed.
I kept the runner's real result, the workflow's continuation policy, and the summary or notification distinct. Allowing the workflow to continue should not turn a failed business assertion into a reported pass. A skipped test is not completed validation either.
A green top-level workflow therefore still required inspection: was the run eligible, did both platform jobs execute, did the selected suites run, and did the final assertions pass?
Validate the app's caller chain
Running the E2E repository directly proves that entry point. To demonstrate integration into app delivery, we also had to exercise the app's caller and inspect version resolution, suite selection, and both platform jobs.
The recorded validation covered distinct scenarios:
- The full configured suite sequence passed on Android and iOS.
- An app-triggered integration run selected the expected scope from changed files and executed it on both platforms.
- Runs with business failures still uploaded diagnostic evidence.
- A passing preparation-only run produced downloadable results, reports, logs, and recordings on both platforms.
These scenarios answered different questions. They demonstrated the corresponding execution and artifact paths, not complete business coverage or a permanently flake-free test system.
Make the next case maintainable by the team
To help product engineers extend coverage, I documented the information needed for a new case: objective, preconditions, steps, optional branches, success assertions, and cleanup. Engineers could describe the behavior in writing, with tools helping inspect components, locate selectors, and draft YAML.
Tools reduce implementation effort; the business owner still needs to confirm that an assertion represents real success. Delivery should include the runnable case, suite registration, execution results, and enough documentation to investigate failures.
This work moved E2E from a script on one engineer's machine into a regression capability the team could invoke, inspect, and maintain. Further work could then focus on useful assertions, better failure attribution, and keeping the tested binary and code versions aligned.
ivan works on React Native, Expo, and iOS migrations, troubleshooting, and delivery workflows. Available for part-time remote technical support through written, asynchronous communication.