Troupeby AliveDocsSign in

Testing

Configuration

troupe.config.ts says which tests run, on which devices, with which app, and how the run reports. This page lists every option with its default, and explains projects, workers, the app, and the servers a run can start.

The config file

troupe test reads troupe.config.ts from the directory you run it in (troupe.config.tsx, .js and .mjs are found too), or the file --config names. The file's default export is the config:

TypeScript
import { defineConfig } from '@alivelabs/troupe'

export default defineConfig({
  testDir: 'e2e',
  retries: 1,
  reporter: [['list'], ['html']],
  use: {
    trace: 'retain-on-failure',
  },
  projects: [
    {
      name: 'ios',
      use: { platform: 'ios', deviceType: 'iPhone 18 Pro', os: '27.0', app: './build/Shop.app' },
    },
    {
      name: 'android',
      use: { platform: 'android', deviceType: 'Pixel 8', os: '36', app: './build/shop.apk' },
    },
  ],
})

The config is checked when it loads. A key Troupe does not know, at the top level or in a project, stops the run with the key's name and exit code 2, so a misspelt option is never silently ignored. Paths in the config are relative to the config file.

Projects

A project is one device setup that runs the tests. With two projects, every test runs twice: once on each. Without projects, the top-level options are the one project.

Every option in test options can be set at the top level and again in a project. The project's value wins. use is merged key by key, so a project states only what differs.

Project optionMeaningDefault
nameThe project's name, shown as [ios] in reports and used by --project. Names must differ'' for the first, project-<n> after it
dependenciesNames of projects that must pass before this one runs. When one fails, this project's tests are skippednone

bunx troupe test --project ios runs one project, and the projects it depends on.

Test options

These apply at the top level and per project.

OptionMeaningDefault
testDirThe directory that holds the teststhe config's directory
testMatchWhich files are tests: a glob relative to testDir, a regular expression tested against the absolute path, or a list**/*.e2e.{ts,tsx}
testIgnoreFiles to leave out, in the same forms**/node_modules/**
timeoutThe time one test may take, in milliseconds. 0 means no limit60,000
retriesHow many times a failed test runs again0, or 2 when CI is set
repeatEachRun every test this many times1
grepOnly tests that match this regular expression, or one of a list. It is tested against the project name, file, describe titles, test title and tagsall
grepInvertLeave out tests that matchnone
tagTags added to every test of the project, such as '@ios'none
fullyParallelLet the tests of one file run in different workersfalse
outputDirWhere traces and attachments are written, one directory per testtest-results
expectAssertion settings, below
useThe device, app and test options, plus options of your own
metadataAny JSON you want to carry. A test reads it at testInfo.project.metadata{}

expect

OptionMeaningDefault
expect.timeoutHow long a locator assertion retries, in milliseconds10,000
expect.toPass.timeoutHow long toPass retriesexpect.timeout
expect.toPass.intervalsThe pauses between toPass attempts, in milliseconds100, 250, 500, then 1,000

Run options

These exist only at the top level.

OptionMeaningDefault
projectsThe list of projectsone project from the top-level options
workersThe most workers to run at onceas many as the devices the runtime may run
shard{ current, total }: run one part of the suitenull
forbidOnlyFail the run when a test.only is presentfalse
maxFailuresStop after this many failed tests. 0 means never0
globalTimeoutThe time the whole run may take, in milliseconds. 0 means no limit0
failOnFlakyTestsCount a test that passed only on a retry as a failure of the runfalse
reporterA reporter's name, or a list of [name] and [name, options]. See reporterslist, or dot when CI is set
webServerOne server, or a list, to start before the testsnone
bundlerThe JavaScript bundler a debug build loads fromnone
quietAccepted so that a Playwright config loads. It has no effect yetfalse

Workers and devices

A worker is a process that holds one device and runs tests on it one at a time. The number of workers is the smallest of three numbers: workers, the groups of tests there are to run, and the devices the runtime may run for the platforms in the run. The runtime allows two devices per platform unless platforms.<id>.maxDevices in ~/.troupe/host.json says otherwise, and it also refuses a device when the host is short of memory. No platform gets more workers than its own limit.

Test files are spread across the workers. The tests of one file run in order in one worker, unless fullyParallel or test.describe.parallel lets them spread too. After a test fails, its worker is replaced by a fresh one, which takes over the same device when the device is healthy.

The app

use.app is the build to install. It takes three forms:

TypeScript
import { defineConfig, xcodeBuild } from '@alivelabs/troupe'

export default defineConfig({
  projects: [
    // A path, relative to the config file.
    { name: 'ios', use: { platform: 'ios', app: './build/Shop.app' } },

    // A simulator build in Xcode's DerivedData, named by its project and configuration.
    {
      name: 'ios-debug',
      use: {
        platform: 'ios',
        app: xcodeBuild({ project: 'ios/Shop.xcworkspace', configuration: 'Debug' }),
      },
    },

    // A download, checked against its SHA-256.
    {
      name: 'android',
      use: {
        platform: 'android',
        app: {
          url: 'https://example.com/builds/shop.apk',
          sha256: '9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08',
        },
      },
    },
  ],
})

An iOS project takes a simulator .app, or a .tar.gz, .tgz, .zip or .ipa that holds one. An Android project takes an .apk. A Mac Catalyst project takes a Catalyst .app that links Troupe's host.

xcodeBuild({ project, configuration, app? }) finds the newest build that xcodebuild, expo run:ios or react-native run-ios left in ~/Library/Developer/Xcode/DerivedData, so the config holds no path of one machine. project is the .xcworkspace or .xcodeproj, relative to the config. app names the .app when the build holds several. When there is no such build yet, the config still loads, and troupe test stops before it starts anything and prints the xcodebuild command that makes one.

Without app, set bundleId to test an app that is already on the device.

Servers

webServer starts a command before the tests and stops it after them, for a backend or mock your app talks to. bundler is the same for the JavaScript bundler of a React Native or Expo debug build: the run passes its address to the app at launch, and on Android rewrites the host's loopback so the emulator can reach it.

TypeScript
export default defineConfig({
  webServer: { command: 'bun run mock-api --port {port}', url: 'http://127.0.0.1:{port}/health' },
  // What `troupe init` writes for an Expo project: Metro on 127.0.0.1 alone.
  bundler: {
    command:
      'NODE_OPTIONS="$NODE_OPTIONS --dns-result-order=ipv4first" ' +
      'bunx expo start --dev-client --localhost --port {port}',
    url: 'http://127.0.0.1:{port}/status',
    timeoutMs: 120_000,
  },
})
OptionMeaningDefault
commandThe shell command to run, in the config's directoryrequired
urlThe address to poll. The server is up when it answers with a 2xx or 3xx statusrequired
timeoutMsHow long to wait for that answer60,000
reuseExistingUse a server that already answers at url instead of refusing to starttrue, or false when CI is set
envExtra environment variables for the commandnone

{port} in command, url or env is replaced by a free port that Troupe picks. Servers start once per run, outside every test timeout. A server that does not come up ends the run with exit code 3.

A release build embeds its JavaScript and needs no bundler. troupe init writes the bundler entry for an Expo project that uses expo-dev-client.

Command-line overrides

Options on the command line win over the config for one run:

FlagOverrides
--project <name>Which projects run
-g, --grep, --grep-invertgrep, grepInvert
-j, --workersworkers
--shard i/nshard
--retries, --timeout, --repeat-eachretries, timeout, repeatEach
-x, --max-failuresmaxFailures
--forbid-onlyforbidOnly
--reporter a,breporter, with no options
--trace <mode>use.trace
--output <dir>outputDir

The CLI describes each flag.

Next

esc

    Type to search every page, the protocol reference and the cloud API.