Testing
Configuration
troupe.config.ts says which tests run, on which devices, with which app, and how the run reports. This page
lists every option with its default, and explains projects, workers, the app, and the servers a run can start.
The config file
troupe test reads troupe.config.ts from the directory you run it in (troupe.config.tsx, .js and .mjs
are found too), or the file --config names. The file's default export is the config:
import { defineConfig } from '@alivelabs/troupe'
export default defineConfig({
testDir: 'e2e',
retries: 1,
reporter: [['list'], ['html']],
use: {
trace: 'retain-on-failure',
},
projects: [
{
name: 'ios',
use: { platform: 'ios', deviceType: 'iPhone 18 Pro', os: '27.0', app: './build/Shop.app' },
},
{
name: 'android',
use: { platform: 'android', deviceType: 'Pixel 8', os: '36', app: './build/shop.apk' },
},
],
})The config is checked when it loads. A key Troupe does not know, at the top level or in a project, stops the run with the key's name and exit code 2, so a misspelt option is never silently ignored. Paths in the config are relative to the config file.
Projects
A project is one device setup that runs the tests. With two projects, every test runs twice: once on each.
Without projects, the top-level options are the one project.
Every option in test options can be set at the top level and again in a project. The
project's value wins. use is merged key by key, so a project states only what differs.
| Project option | Meaning | Default |
|---|---|---|
name | The project's name, shown as [ios] in reports and used by --project. Names must differ | '' for the first, project-<n> after it |
dependencies | Names of projects that must pass before this one runs. When one fails, this project's tests are skipped | none |
bunx troupe test --project ios runs one project, and the projects it depends on.
Test options
These apply at the top level and per project.
| Option | Meaning | Default |
|---|---|---|
testDir | The directory that holds the tests | the config's directory |
testMatch | Which files are tests: a glob relative to testDir, a regular expression tested against the absolute path, or a list | **/*.e2e.{ts,tsx} |
testIgnore | Files to leave out, in the same forms | **/node_modules/** |
timeout | The time one test may take, in milliseconds. 0 means no limit | 60,000 |
retries | How many times a failed test runs again | 0, or 2 when CI is set |
repeatEach | Run every test this many times | 1 |
grep | Only tests that match this regular expression, or one of a list. It is tested against the project name, file, describe titles, test title and tags | all |
grepInvert | Leave out tests that match | none |
tag | Tags added to every test of the project, such as '@ios' | none |
fullyParallel | Let the tests of one file run in different workers | false |
outputDir | Where traces and attachments are written, one directory per test | test-results |
expect | Assertion settings, below | |
use | The device, app and test options, plus options of your own | |
metadata | Any JSON you want to carry. A test reads it at testInfo.project.metadata | {} |
expect
| Option | Meaning | Default |
|---|---|---|
expect.timeout | How long a locator assertion retries, in milliseconds | 10,000 |
expect.toPass.timeout | How long toPass retries | expect.timeout |
expect.toPass.intervals | The pauses between toPass attempts, in milliseconds | 100, 250, 500, then 1,000 |
Run options
These exist only at the top level.
| Option | Meaning | Default |
|---|---|---|
projects | The list of projects | one project from the top-level options |
workers | The most workers to run at once | as many as the devices the runtime may run |
shard | { current, total }: run one part of the suite | null |
forbidOnly | Fail the run when a test.only is present | false |
maxFailures | Stop after this many failed tests. 0 means never | 0 |
globalTimeout | The time the whole run may take, in milliseconds. 0 means no limit | 0 |
failOnFlakyTests | Count a test that passed only on a retry as a failure of the run | false |
reporter | A reporter's name, or a list of [name] and [name, options]. See reporters | list, or dot when CI is set |
webServer | One server, or a list, to start before the tests | none |
bundler | The JavaScript bundler a debug build loads from | none |
quiet | Accepted so that a Playwright config loads. It has no effect yet | false |
Workers and devices
A worker is a process that holds one device and runs tests on it one at a time. The number of workers is the
smallest of three numbers: workers, the groups of tests there are to run, and the devices the runtime may
run for the platforms in the run. The runtime allows two devices per platform unless
platforms.<id>.maxDevices in ~/.troupe/host.json says otherwise, and it also refuses a device when the
host is short of memory. No platform gets more workers than its own limit.
Test files are spread across the workers. The tests of one file run in order in one worker, unless
fullyParallel or test.describe.parallel lets them spread too. After a test fails, its worker is replaced
by a fresh one, which takes over the same device when the device is healthy.
The app
use.app is the build to install. It takes three forms:
import { defineConfig, xcodeBuild } from '@alivelabs/troupe'
export default defineConfig({
projects: [
// A path, relative to the config file.
{ name: 'ios', use: { platform: 'ios', app: './build/Shop.app' } },
// A simulator build in Xcode's DerivedData, named by its project and configuration.
{
name: 'ios-debug',
use: {
platform: 'ios',
app: xcodeBuild({ project: 'ios/Shop.xcworkspace', configuration: 'Debug' }),
},
},
// A download, checked against its SHA-256.
{
name: 'android',
use: {
platform: 'android',
app: {
url: 'https://example.com/builds/shop.apk',
sha256: '9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08',
},
},
},
],
})An iOS project takes a simulator .app, or a .tar.gz, .tgz, .zip or .ipa that holds one. An Android
project takes an .apk. A Mac Catalyst project takes a Catalyst .app that links Troupe's host.
xcodeBuild({ project, configuration, app? }) finds the newest build that xcodebuild, expo run:ios or
react-native run-ios left in ~/Library/Developer/Xcode/DerivedData, so the config holds no path of one
machine. project is the .xcworkspace or .xcodeproj, relative to the config. app names the .app when
the build holds several. When there is no such build yet, the config still loads, and troupe test stops
before it starts anything and prints the xcodebuild command that makes one.
Without app, set bundleId to test an app that is already on the device.
Servers
webServer starts a command before the tests and stops it after them, for a backend or mock your app talks
to. bundler is the same for the JavaScript bundler of a React Native or Expo debug build: the run passes its
address to the app at launch, and on Android rewrites the host's loopback so the emulator can reach it.
export default defineConfig({
webServer: { command: 'bun run mock-api --port {port}', url: 'http://127.0.0.1:{port}/health' },
// What `troupe init` writes for an Expo project: Metro on 127.0.0.1 alone.
bundler: {
command:
'NODE_OPTIONS="$NODE_OPTIONS --dns-result-order=ipv4first" ' +
'bunx expo start --dev-client --localhost --port {port}',
url: 'http://127.0.0.1:{port}/status',
timeoutMs: 120_000,
},
})| Option | Meaning | Default |
|---|---|---|
command | The shell command to run, in the config's directory | required |
url | The address to poll. The server is up when it answers with a 2xx or 3xx status | required |
timeoutMs | How long to wait for that answer | 60,000 |
reuseExisting | Use a server that already answers at url instead of refusing to start | true, or false when CI is set |
env | Extra environment variables for the command | none |
{port} in command, url or env is replaced by a free port that Troupe picks. Servers start once per run,
outside every test timeout. A server that does not come up ends the run with exit code 3.
A release build embeds its JavaScript and needs no bundler. troupe init writes the bundler entry for an
Expo project that uses expo-dev-client.
Command-line overrides
Options on the command line win over the config for one run:
| Flag | Overrides |
|---|---|
--project <name> | Which projects run |
-g, --grep, --grep-invert | grep, grepInvert |
-j, --workers | workers |
--shard i/n | shard |
--retries, --timeout, --repeat-each | retries, timeout, repeatEach |
-x, --max-failures | maxFailures |
--forbid-only | forbidOnly |
--reporter a,b | reporter, with no options |
--trace <mode> | use.trace |
--output <dir> | outputDir |
The CLI describes each flag.
Next
- Fixtures: the options that go in
use - Reporters
- Running in CI
- The CLI