CI integration

Integrating MobileBoost into your CI pipeline enables automated test execution against your latest app builds.

GitHub Actions

Run your tests from GitHub Actions with the actions in MobileBoostHQ/actions. You reference them directly in your workflow, so there is nothing to install.

upload-build uploads your build and returns its build-id.
run-tests runs your tests by tag or ID, waits for them to finish, and fails the job if a test fails.

Add your credentials

In your repository, go to Settings > Secrets and variables > Actions.

In the Secrets tab, add MOBILEBOOST_API_KEY. Create the key in MobileBoost under Settings > API keys.
In the Variables tab, add MOBILEBOOST_ORG_ID, your organisation ID. Ask your MobileBoost contact if you do not have it.

Run your tests against a build

Upload the build your pipeline produced, then run every test with one of the tags you list:

name: MobileBoost tests

on:
  pull_request:
    branches: [main]

jobs:
  mobile-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      # Your existing build steps

      - uses: MobileBoostHQ/actions/upload-build@v1
        id: upload
        with:
          api-key: ${{ secrets.MOBILEBOOST_API_KEY }}
          organisation-id: ${{ vars.MOBILEBOOST_ORG_ID }}
          build-path: app/build/outputs/apk/release/*.apk

      - uses: MobileBoostHQ/actions/run-tests@v1
        with:
          api-key: ${{ secrets.MOBILEBOOST_API_KEY }}
          organisation-id: ${{ vars.MOBILEBOOST_ORG_ID }}
          build-id: ${{ steps.upload.outputs.build-id }}
          mode: ai-sdet
          tags: smoke,login
          async: false

mode: ai-sdet runs the test code MobileBoost wrote and verified for each of your tests, on MobileBoost devices.
tags runs every test that has at least one of the tags. Use test-ids to run specific tests, or both to run both sets.
async: false makes the step wait for the run. It then writes a result table with links to each recording into the job summary, and fails the job if a test failed or was blocked.

build-path takes an .apk, an .ipa for physical iOS devices, or a .zip or .tar.gz of the .app for iOS simulators. A directory is zipped for you, and a glob uses its first match. See Build requirements.

Run against the app installed on your device

If your organisation tests the app already installed on its reserved device, such as the App Store or Play Store version, you do not need to upload a build. Name the app by its bundle ID (iOS) or package name (Android) and its platform instead:

name: MobileBoost tests

on:
  schedule:
    - cron: "0 3 * * *" # every night at 03:00 UTC
  workflow_dispatch:

jobs:
  mobile-tests:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        include:
          - platform: ios
            bundle-id: com.example.app
          - platform: android
            bundle-id: com.example.app
    steps:
      - uses: MobileBoostHQ/actions/run-tests@v1
        with:
          api-key: ${{ secrets.MOBILEBOOST_API_KEY }}
          organisation-id: ${{ vars.MOBILEBOOST_ORG_ID }}
          bundle-id: ${{ matrix.bundle-id }}
          platform: ${{ matrix.platform }}
          mode: ai-sdet
          tags: smoke
          async: false

The tests run on your physical device, the only kind with your app installed.
They test the version installed on the device, not the code of the commit that started the workflow. That makes this a good fit for scheduled and manual runs. To test a pull request’s changes, upload the build it produced instead.
Running without a build has to be enabled for your organisation. Ask your MobileBoost contact.

Requires version 1.9.0 of the actions. @v1 picks it up automatically.

`run-tests` inputs

Input	Required	Description
`api-key`	Yes	Your API key. Use the repository secret
`organisation-id`	Yes	Your organisation ID
`build-id`	One of `build-id` or `bundle-id`	The `build-id` output of `upload-build`
`bundle-id`		Bundle ID (iOS) or package name (Android) of the app installed on your reserved device
`platform`	With `bundle-id`	`ios` or `android`. A `build-id` run takes it from the build
`mode`	Yes	Set to `ai-sdet` to run your tests’ generated code on MobileBoost devices
`tags`	`tags`, `test-ids`, or both	Run every test with at least one of these comma-separated tags
`test-ids`		Comma-separated IDs of the tests to run
`use-physical-device`	No	`true` for a real handset, `false` for an emulator or simulator. Defaults to your organisation’s setting. See Device configuration
`tunnel-name`	No	A MobileBoost Local tunnel for apps that call internal hosts. See Local testing in CI
`async`	No	`false` to wait for the run and fail the job on failures. Defaults to `true`, which returns as soon as the run has started
`timeout-minutes`	No	How long to wait with `async: false`. Defaults to `180`
`fail-on-test-failure`	No	`false` to report results without failing the job. Defaults to `true`

The step outputs run-id, and with async: false also passed, failed and blocked. Every selected test must have its automation written and verified for the platform; see Step 2: Start a run.

Results on the pull request

When a run comes from a pull request, MobileBoost can post its results as a comment on that pull request: what passed, what failed and why, with links to the reports. Install the MobileBoost GitHub App on the repository and ask your MobileBoost contact to turn the comments on. No permissions: block or GITHUB_TOKEN is needed.

Pin the version

@v1 is recommended. It gets every v1.x.x release, with fixes and no breaking changes.
@v1.9.0 pins an exact version.

Run the tests a pull request describes

The MobileBoost Test Action works differently: it uploads your build and sends the pull request description, which holds the test instructions, to MobileBoost in a single step.

name: MobileBoost tests

on:
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Build project
        run: |
          # Build your project and generate the build file

      - name: Trigger MobileBoost tests
        uses: MobileBoostHQ/mobileboost-test-action@v1
        with:
          organisation-id: ${{ secrets.MOBILEBOOST_ORG_ID }}
          app-path: path/to/your/build.apk
          platform: android

For how to write the instructions, see the format requirements. The action takes organisation-id (store it as the secret MOBILEBOOST_ORG_ID), app-path (the .apk or .ipa) and platform (Android or iOS).

If your app talks to a staging environment that is not reachable from the public internet, open a tunnel in the same job so the devices can reach it. See Local testing in CI.

CI provider snippets

For other CI systems, use the upload API to send your build artifact and capture the returned buildId.

In all examples below, replace <ORG_KEY> with your MobileBoost organisation key, <platform> with ios or android, and the build file path with the actual path to your artifact.

Bitrise
GitLab CI
CircleCI
Jenkins

Add a Script step to your workflow after the build step. This uploads the build and makes the buildId available as an environment variable for subsequent steps.

call=$(curl -i -X POST \
        -H "Content-Type: multipart/form-data" \
        -F "build=@$BITRISE_BUILD_PATH" \
        -F "organisation_key=<ORG_KEY>" \
        -F "platform=<platform>" \
        -F "metadata={}" \
        https://api.mobileboost.io/uploadBuild/)

buildId=$(echo "$call" | awk '/^{/ {print}' | jq -r '.buildId')

envman add --key MOBILEBOOST_BUILD_ID --value $buildId

Link each build to its pull request (recommended). Pass a ci field so MobileBoost knows which pull request produced each build. Results then map back to the right PR, and MobileBoost can select the tests relevant to that PR’s changes. Every value comes from a built-in Bitrise variable, so there is nothing extra to create.

# Assemble the ci payload from Bitrise's built-in PR variables
CI_JSON=$(jq -n \
  --arg repo      "$BITRISEIO_GIT_REPOSITORY_OWNER/$BITRISEIO_GIT_REPOSITORY_SLUG" \
  --arg commitSha "$BITRISE_GIT_COMMIT" \
  --arg branch    "${BITRISEIO_GIT_BRANCH_DEST:-$BITRISE_GIT_BRANCH}" \
  --arg prNumber  "$BITRISE_PULL_REQUEST" \
  '{repo: $repo, commitSha: $commitSha, branch: $branch}
   + (if $prNumber != "" then {prNumber: ($prNumber | tonumber)} else {} end)')

call=$(curl -i -X POST \
        -H "Content-Type: multipart/form-data" \
        -F "build=@$BITRISE_BUILD_PATH" \
        -F "organisation_key=<ORG_KEY>" \
        -F "platform=<platform>" \
        -F "ci=$CI_JSON" \
        https://api.mobileboost.io/uploadBuild/)

buildId=$(echo "$call" | awk '/^{/ {print}' | jq -r '.buildId')

envman add --key MOBILEBOOST_BUILD_ID --value "$buildId"

Each ci field maps to a variable Bitrise sets automatically:

`ci` field	Bitrise variable	Availability
`repo`	`$BITRISEIO_GIT_REPOSITORY_OWNER` / `$BITRISEIO_GIT_REPOSITORY_SLUG`	Always
`commitSha`	`$BITRISE_GIT_COMMIT`	Always
`branch`	`$BITRISEIO_GIT_BRANCH_DEST`, falling back to `$BITRISE_GIT_BRANCH`	PR destination branch on PR builds; current branch otherwise
`prNumber`	`$BITRISE_PULL_REQUEST` (the PR number)	Pull-request-triggered builds only

$BITRISE_PULL_REQUEST and $BITRISEIO_GIT_BRANCH_DEST are only set on pull-request-triggered builds, so make sure your workflow runs on pull requests (App settings > Triggers). On non-PR builds the snippet omits prNumber and links the build by commit instead - no changes needed.

The only value you provide yourself is your organisation key - store it as a Secret (App settings > Secrets) and reference it (for example $MOBILEBOOST_ORG_KEY) instead of hardcoding <ORG_KEY>. The build path is whatever your build step produces ($BITRISE_APK_PATH for Android, $BITRISE_IPA_PATH for iOS).

Add an upload stage to your .gitlab-ci.yml pipeline:

stages:
  - build
  - upload

build_job:
  stage: build
  script:
    - echo "Building the project..."
    # Build your project and generate the build file
    - export BUILD_FILE_PATH=path/to/your/build_file.apk

upload_job:
  stage: upload
  script:
    - echo "Uploading the build..."
    - >
      call=$(curl -i -X POST
                   -H "Content-Type: multipart/form-data"
                   -F "build=@${BUILD_FILE_PATH}"
                   -F "organisation_key=<ORG_KEY>"
                   -F "platform=<platform>"
                   -F "metadata={}"
                   https://api.mobileboost.io/uploadBuild/);
      buildId=$(echo "$call" | awk '/^{/ {print}' | jq -r '.buildId')
      echo "MobileBoost buildId: ${buildId}"

Add an upload-build job to your .circleci/config.yml:

version: 2.1

jobs:
  upload-build:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout

      - run:
          name: Install dependencies
          command: |
            sudo apt-get update
            sudo apt-get install -y jq curl

      - run:
          name: Build project
          command: |
            echo "Building the project..."
            # Replace with your actual build commands

      - run:
          name: Upload build to MobileBoost
          command: |
            response=$(curl -i -X POST \
                        -H "Content-Type: multipart/form-data" \
                        -F "build=@<build_file_path>" \
                        -F "organisation_key=<ORG_KEY>" \
                        -F "platform=<platform>" \
                        -F "metadata={}" \
                        https://api.mobileboost.io/uploadBuild/)

            if [ $? -ne 0 ]; then
              echo "Error: Failed to execute the curl command."
              exit 1
            fi

            status_code=$(echo "$response" | awk '/HTTP\// {print $2}' | tail -n1)
            if [ "$status_code" != "200" ]; then
              echo "Error: Upload failed with HTTP status code $status_code."
              echo "Response: $response"
              exit 1
            fi

            json_response=$(echo "$response" | awk '/^{/ {print}')
            buildId=$(echo "$json_response" | jq -r '.buildId')
            if [ -z "$buildId" ] || [ "$buildId" == "null" ]; then
              echo "Error: buildId not found in response."
              exit 1
            fi

            echo "Build ID: $buildId"
            echo "export BUILD_ID=$buildId" >> $BASH_ENV

workflows:
  build_and_upload:
    jobs:
      - upload-build

Add an Upload Build stage to your Jenkinsfile:

pipeline {
    agent any

    environment {
        ORG_KEY = '<ORG_KEY>'
        PLATFORM = '<platform>'
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }

        stage('Build') {
            steps {
                script {
                    // Replace with your actual build commands
                }
            }
        }

        stage('Upload Build to MobileBoost') {
            steps {
                script {
                    def buildFilePath = 'path/to/your/build/file'
                    def response = httpRequest httpMode: 'POST',
                                             url: 'https://api.mobileboost.io/uploadBuild/',
                                             contentType: 'MULTIPART_FORMDATA',
                                             multipartNameValue: [
                                                 [name: 'build', fileName: buildFilePath,
                                                  mimeType: 'application/octet-stream'],
                                                 [name: 'organisation_key', value: ORG_KEY],
                                                 [name: 'platform', value: PLATFORM],
                                                 [name: 'metadata', value: '{}']
                                             ]
                    def responseBody = response.content
                    def buildId = sh(
                        script: "echo '${responseBody}' | jq -r '.buildId'",
                        returnStdout: true
                    ).trim()
                    env.BUILD_ID = buildId
                    echo "Build ID: ${buildId}"
                }
            }
        }
    }

    post {
        always {
            echo "Build and upload completed"
        }
    }
}

API

For full control, use the API directly. The process has three steps: upload the build, start a run, then read its results. Send your API key as Authorization: Bearer mb_live_... on each call.

Step 1: Upload the build

Upload your build artifact using the upload endpoint. The response returns a buildId.

curl -X POST \
  -H "Content-Type: multipart/form-data" \
  -F "build=@path/to/your/build.apk" \
  -F "organisation_key=YOUR_ORG_KEY" \
  -F "platform=android" \
  -F "metadata={}" \
  https://api.mobileboost.io/uploadBuild/

Step 2: Start a run

Send the buildId from step 1 to POST /tests/execute. Run specific tests by ID:

{
  "organisationId": "org123",
  "buildId": "BUILD_ID_FROM_STEP_1",
  "testIds": ["test_abc", "test_def"]
}

Run tests by tags: tags runs every test that has at least one of the tags. Send testIds as well to add specific tests to the run.

{
  "organisationId": "org123",
  "buildId": "BUILD_ID_FROM_STEP_1",
  "tags": ["critical"]
}

Run the newest build with a name: If your pipeline doesn’t keep the buildId, name the build instead. The run uses the newest build uploaded under that name for platform:

{
  "organisationId": "org123",
  "build": { "meta": { "name": "my-app" } },
  "platform": "android",
  "tags": ["critical"],
  "trigger": "ci"
}

Field	Required	Description
`organisationId`	Yes	Your organisation ID. It must be the organisation of your API key.
`buildId`	One of `buildId`, `build` or `bundleId`	The `buildId` from step 1.
`build.meta.name`		The newest build uploaded under this name.
`bundleId`		An app already installed on your reserved device, instead of an uploaded build. Requires `platform`, and has to be enabled for your organisation.
`testIds`	`testIds`, `tags`, or both	IDs of the tests to run.
`tags`		Run every test with at least one of these tags.
`platform`	With `bundleId`	`ios` or `android`. Otherwise taken from the build.
`usePhysicalDevice`	No	`true` for a real handset, `false` for an emulator or simulator. See Device configuration.
`tunnelName`	No	A MobileBoost Local tunnel for apps that call internal hosts. See Test against internal environments.
`trigger`	No	A label stored on the run, such as `ci`. Defaults to `api`.
`variables`	No	Values for this run’s tests, as `NAME: value` pairs. See Pass values to the tests.

A test is runnable on the platform once its automation has been written and verified. GET /tests shows this as platforms.<platform>.runnable.

A test you name in testIds must be runnable, or the request is refused with 400.
A test that only matches tags and is not runnable yet is skipped, and the rest of the run starts. If none of the tests matching tags is runnable, the request is refused with 400 and the message lists them.

The response carries the ID of the run in runId, and lists any skipped test in skippedTests:

{
  "message": "Autotest run created",
  "runId": "b1111111df174791a93e560390ea7e65",
  "status": "running",
  "scheduledTests": 11,
  "skippedTests": [
    {
      "testId": "xMIt964RUpAB95b6YI2F",
      "title": "Checkout with a saved card",
      "platform": "android",
      "reason": "not_automated",
      "detail": "No automation has been generated for this test yet."
    }
  ],
  "warnings": [
    "1 of 12 tests matching tags critical was not scheduled on android: 1 not automated yet. They are listed in skippedTests."
  ]
}

reason is one of not_automated, generation_in_progress, generation_failed, generation_blocked or not_ready. Print warnings in your pipeline log so a skipped test does not go unnoticed. A skipped test never fails the run.

deviceConfigs, deviceProviderSettings, launchParams, testInputs, tagsQuery, iterations and metaData are not supported. A request that sends any of them is refused with 400, so a run never goes ahead on settings it ignored. Send the values testInputs carried as variables instead.

Pass values to the tests

Some values belong to one run rather than to your tests: the backend a canary run signs in against, or a feature flag a nightly run switches on. Send them as variables:

{
  "organisationId": "org123",
  "build": { "meta": { "name": "my-app" } },
  "platform": "android",
  "tags": ["canary"],
  "variables": { "API_BASE_URL": "https://canary-api.example.com" },
  "trigger": "ci"
}

Each test in the run reads them as environment variables. A value here wins over your organisation’s environment variable of the same name, for this run only, so a test can fall back to your organisation’s value when a run sends none. The response lists the names it received in variables.

Names use upper case letters, digits and underscores, and start with a letter. Names the test runner relies on, such as PATH or anything starting with PYTHON, are refused with 400. Values must be strings, and a run takes up to 20.
Send your API key. A request with variables needs it even if your organisation does not otherwise require one, because a variable can change which host a test signs in to.
Not for passwords or keys. The values are kept with the run for 14 days so a retry finds them. Keep credentials in your organisation’s environment variables (ENV Variables in the web app), where every read is audited.

A test only uses a variable it reads. If a test has a backend address written into it, change the test to read the variable first.

Step 3: Get the results

Pass runId to GET /runs/{runId}, and repeat the call until status is completed or cancelled:

curl https://api.mobileboost.io/runs/b1111111df174791a93e560390ea7e65 \
  -H "Authorization: Bearer mb_live_..."

{
  "runId": "b1111111df174791a93e560390ea7e65",
  "status": "completed",
  "totalTests": 2,
  "runtimeSeconds": 212,
  "succeededTests": [
    {
      "id": "e4a1c7f09b2d4e56a8f3c1d2b7e90a11",
      "title": "test_login_flow.py",
      "status": "succeeded",
      "runtimeSeconds": 94,
      "recording": "https://platform.mobileboost.io/reports/b1111111df174791a93e560390ea7e65/e4a1c7f09b2d4e56a8f3c1d2b7e90a11"
    }
  ],
  "failedTests": [
    {
      "id": "7c2e9d10a4b84f3e9e5d6a7b8c9d0e12",
      "title": "test_checkout_flow.py",
      "status": "failed",
      "runtimeSeconds": 118,
      "recording": "https://platform.mobileboost.io/reports/b1111111df174791a93e560390ea7e65/7c2e9d10a4b84f3e9e5d6a7b8c9d0e12"
    }
  ],
  "blockedTests": [],
  "runningTests": [],
  "queuedTests": [],
  "skippedTests": []
}

completed means every test has finished, not that every test passed. Fail your pipeline when failedTests is not empty.
skippedTests repeats the tests that were left out when the run started, in the same shape as in step 2. They are not counted in totalTests and never change status.
While the run is in progress, runningTests and queuedTests show which tests are on a device and which are waiting for one.
Step 2 returns just before the run is saved, which usually takes under half a second. Wait 2 seconds before your first call. If it answers 404, retry every 2 seconds. A 404 that lasts longer than 30 seconds means the run was not created: check the runId, then start the run again.
While status is initial, tests are still being added to the run, so totalTests can still grow. This takes up to about 10 seconds for a run of 100 tests.

A minimal polling loop:

RUN_ID=$(curl -s -X POST https://api.mobileboost.io/tests/execute \
  -H "Authorization: Bearer $MOBILEBOOST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"organisationId": "org123", "buildId": "'"$BUILD_ID"'", "tags": ["critical"]}' \
  | jq -r '.runId')

# The run is saved just after the call returns: allow it up to 30 seconds.
for attempt in $(seq 1 15); do
  sleep 2
  CODE=$(curl -s -o /dev/null -w '%{http_code}' https://api.mobileboost.io/runs/$RUN_ID \
    -H "Authorization: Bearer $MOBILEBOOST_API_KEY")
  [ "$CODE" != "404" ] && break
done
if [ "$CODE" = "404" ]; then echo "Run $RUN_ID was not created"; exit 1; fi

while true; do
  RESULT=$(curl -s https://api.mobileboost.io/runs/$RUN_ID \
    -H "Authorization: Bearer $MOBILEBOOST_API_KEY")
  STATUS=$(echo "$RESULT" | jq -r '.status')
  echo "Run $RUN_ID: $STATUS"
  if [ "$STATUS" = "completed" ] || [ "$STATUS" = "cancelled" ]; then break; fi
  sleep 30
done

[ "$(echo "$RESULT" | jq '.failedTests | length')" -eq 0 ]

Build requirements

Before tests can run, your application build must be uploaded to MobileBoost.

Simulators

Platform	Format	How to generate
Android	`.apk`	Build in Android Studio (Build > Build APK(s)) or via Gradle: `./gradlew assembleDebug`
iOS	`.zip` or `.tar.gz` containing the `.app` bundle	Build in Xcode targeting an iOS Simulator, then compress the `.app` folder

iOS simulator builds: Build your app in Xcode targeting an iOS Simulator. Locate the .app file at Product > Show Build Folder in Finder > Products/Debug-iphonesimulator. Then compress it:

zip -r MyApp.zip MyApp.app

Or build from the command line:

xcodebuild -workspace MyApp.xcworkspace -scheme MyApp -sdk iphonesimulator -configuration Debug

Physical devices

Platform	Format	Notes
Android	`.apk`	Same as simulator builds
iOS	`.ipa`	MobileBoost automatically re-signs the `.ipa` for the test device grid. No special certificates or provisioning profiles required on your end

Getting started

Test definitions

Test execution

Performance testing

Test reports

GitHub Actions

Add your credentials

Run your tests against a build

Run against the app installed on your device

`run-tests` inputs

Results on the pull request

Pin the version

Run the tests a pull request describes

CI provider snippets

API

Step 1: Upload the build

Step 2: Start a run

Pass values to the tests

Step 3: Get the results

Build requirements

Simulators

Physical devices

GitHub Actions

Add your credentials

Run your tests against a build

Run against the app installed on your device

run-tests inputs

Results on the pull request

Pin the version

Run the tests a pull request describes

CI provider snippets

API

Step 1: Upload the build

Step 2: Start a run

Pass values to the tests

Step 3: Get the results

Build requirements

Simulators

Physical devices

`run-tests` inputs