Troupe Cloud
Test runs
A cloud test run splits one troupe test suite into shards, runs each shard beside its own hosted devices,
and reports them as one run. This page describes that model and what to use until it is offered.
What to use today
- Run the suite yourself.
troupe testruns on your Mac and on a macOS CI runner, where the runtime runs inside the test process.--shardsplits a suite over several runners andtroupe merge-reportsjoins their reports. See running in CI. - Build in the cloud, test where you like. A cloud build gives you an artifact to download and hand to your own run.
- Drive a hosted device from a script. Device sessions take the same procedures a test uses, through the client.
The model
A run belongs to a project and is described by one request:
| Field | Meaning | Bounds |
|---|---|---|
suite | The test suite, uploaded as one gzipped tar. The request states its size and SHA-256 and the answer carries a signed upload request | 64 MiB |
device | The device spec every device of the run gets | A device spec |
shards | How many parts the suite is split into | 1 to 16; 1 by default |
workers | How many devices each shard drives | 1 to 4; 1 by default |
install | Build artifacts to install on every device before its shard starts | Up to 5 |
projects, grep | Which of the suite's projects and tests to run | Up to 16 projects |
timeoutMs | How long the run may take | 10 seconds to 3 hours; 30 minutes by default |
What happens then:
- The run waits for the suite upload, which must arrive within an hour. Troupe computes the upload's SHA-256 itself and starts nothing if it differs from the one you stated.
- Each shard runs
troupe test --shard=<index>/<total>in a sandbox, apart from the devices. - A shard's workers attach to devices started for that shard, with the spec and the installs of the run. A shard cannot start other devices.
- Shards publish their results as they go. The run's status and counts are the sum of its shards'.
Runs and shards share one set of statuses: queued, running, then passed, failed, errored, timedOut
or canceled. failed is your tests or your app; errored is infrastructure.
Procedures
| Procedure | What it does |
|---|---|
runs.create | Starts a run; needs devices:acquire and runs:report |
runs.wait | Waits until the run ends or a timeout passes |
runs.shards | Lists the shards with their status, devices, exit code and counts |
runs.cancel | Ends the unfinished shards and releases their devices |
runs.get, runs.list, runs.events | Read a run, its tests and its reporter events, as for a local run |
The troupe CLI has no command that starts a cloud run.
Limits
| Limit | Value |
|---|---|
| Shards running at once | The test_shards limit: see limits |
| Runs waiting for their suite upload | 8 in a project, 32 in an organization |
| Metered | Each shard's running time, as test_run_seconds, and its devices' time, as device_seconds |
In the web app
A project's page has a Test runs list, and a run's page shows its summary, its shards and each test's outcome, with Cancel while it runs. The web app does not start runs.