Testing

Writing a test

A Troupe test is a TypeScript file that finds elements on a device, acts on them and asserts what the screen shows. This page covers the file, test, the screen and app fixtures, a first assertion, and running it.

The test file

Tests live in files whose names end in .e2e.ts or .e2e.tsx, under the config's testDir. troupe init sets testDir to e2e. Import test and expect from @alivelabs/troupe:

TypeScript

// e2e/sign-in.e2e.ts
import { expect, test } from '@alivelabs/troupe'

test('signs in', async ({ screen }) => {
  await screen.getByLabel('Email').fill('ada@ex.co')
  await screen.getByLabel('Password').fill('pw')
  await screen.getByRole('button', { name: 'Sign in' }).tap()
  await expect(screen.getByText('Signed in as ada@ex.co')).toBeVisible()
})

Run these files with troupe test. bun test does not collect *.e2e.ts files, and a file run outside the Troupe runner fails with a message that says so.

Fixtures

A test receives what it needs through its first parameter. Each name is a fixture: something the runner sets up before the test and tears down after it.

Fixture	What it is
`screen`	The foreground app's screen: where you build locators and send screen-wide input
`app`	The app under test: launch, terminate, open a deep link, clear its data, set permissions
`device`	The device session this worker holds: install apps, rotate, take a screenshot
`runtime`	Typed calls to the runtime, for anything the fixtures above do not wrap
`testInfo`	The running test: its title, retry number, output directory and attachments

You must destructure the fixtures: write async ({ screen, app }) => …, not async (fixtures) => …. Troupe reads the names to know what to set up.

Before each test the app fixture terminates your app and launches it again, so every test starts on the app's first screen. screen depends on app, so asking for screen is enough. Fixtures covers the options and other isolation levels.

TypeScript

test('open a deep link', async ({ app, screen }) => {
  await app.openUrl('troupefixture://hello/world')
  await expect(screen.getByTestId('opened-url')).toHaveText('Opened: troupefixture://hello/world')
})

Find, act, assert

A test is made of three kinds of line:

A locator names an element: screen.getByRole('button', { name: 'Sign in' }). It is lazy: nothing is looked up until you use it. See locators.
An action does something to it: .tap(), .fill('…'), .swipe('up'). The action waits until the element is there, visible, enabled and not moving, then acts. See actions.
An assertion checks the screen: await expect(locator).toHaveText('Count: 1'). It retries until it holds or 10 seconds pass. See assertions.

You do not write sleeps or waits between them:

TypeScript

test('Increment counts', async ({ screen }) => {
  await screen.getByTestId('nav-controls').tap()
  await expect(screen.getByTestId('count')).toHaveText('Count: 0')
  await screen.getByRole('button', { name: 'Increment', exact: true }).tap()
  await expect(screen.getByTestId('count')).toHaveText('Count: 1')
})

Groups, hooks and steps

test.describe groups tests. Hooks run around them, and take fixtures the same way a test does. test.step names a part of a test in the report and the trace.

TypeScript

test.describe('cart', () => {
  test.beforeEach(async ({ screen }) => {
    await screen.getByRole('tab', { name: 'Cart' }).tap()
  })

  test('removes an item', async ({ screen }) => {
    await test.step('remove the first item', async () => {
      await screen.getByRole('listitem').first().getByRole('button', { name: 'Remove' }).tap()
    })
    await expect(screen.getByRole('listitem')).toHaveCount(1)
  })
})

API	What it does
`test.beforeEach`, `test.afterEach`	Run before and after every test of the group
`test.beforeAll`, `test.afterAll`	Run once per worker for the group, with worker fixtures such as `device`
`test.describe.serial`	Tests that depend on each other: after a failure the rest are skipped, and a retry runs the group again
`test.describe.parallel`	Each test of the group may run in its own worker
`test.describe.configure({ mode, retries, timeout })`	Sets the mode, retries or timeout for the enclosing group or file

Skip, focus and tag

TypeScript

test.skip('exports a receipt', async ({ screen }) => {
  // skipped until the screen exists
})

test('pays with a saved card @smoke', async ({ screen }) => {
  test.slow()
  await screen.getByRole('button', { name: 'Pay' }).tap()
  await expect(screen.getByText('Paid')).toBeVisible()
})

test('a crash of the app under test is APP_CRASHED', async ({ screen }) => {
  test.fail(true, 'the fixture crashes on purpose: the tap that crashes it fails with APP_CRASHED')
  await screen.getByTestId('nav-diagnostics').tap()
  await screen.getByTestId('crash').tap()
  await screen.getByTestId('store-value').tap({ timeout: 10_000 })
})

test.skip and test.fixme skip a test. test.fail expects it to fail. test.slow triples its timeout. Each also works inside a test, with an optional condition and reason, or at the top of a file or describe.
test.only runs only the focused tests. --forbid-only makes a run fail when one is left in.
A word that starts with @ in a title is a tag. You can also pass { tag: '@smoke' } as the second argument. Run tagged tests with bunx troupe test --grep @smoke.

To skip by platform, pass a function. It receives the project's option values, with platform as the full id, not the live fixtures:

TypeScript

test.skip(({ platform }) => platform === 'android-emulator', 'iOS only for now')

Run it

Shell

bunx troupe test                            # every test, every project
bunx troupe test e2e/sign-in.e2e.ts         # files whose path contains this
bunx troupe test e2e/sign-in.e2e.ts:6       # the test or describe on that line
bunx troupe test --grep "signs in"          # tests whose title matches
bunx troupe test --project ios              # one project
bunx troupe test --list                     # print the tests, start nothing

--list needs no runtime and no device:

Output

Listing tests:
  [ios] › e2e/example.e2e.ts:11:1 › the app launches and shows its first screen
  [android] › e2e/example.e2e.ts:11:1 › the app launches and shows its first screen
Total: 2 tests in 1 file

The CLI lists every option of troupe test.

When a test fails

The reporter prints the failed expectation, the source line, an excerpt of the accessibility tree at that moment, and where the trace is:

Output

  1) [ios] › e2e/cart.e2e.ts:20:3 › cart › shows the total ─────────────────────────────────────

    Error: expect(locator).toHaveText(expected) failed

    Locator: getByText('Total')
    Expected: "12.00"
    Received: "Total: 11.00"
    Timeout: 10000ms

      18 |   test('shows the total', async ({ screen }) => {
      19 |     await screen.getByRole('button', { name: 'Checkout' }).tap()
    > 20 |     await expect(screen.getByText('Total')).toHaveText('12.00')
         |                                             ^
      21 |   })

    at e2e/cart.e2e.ts:20:5

    Accessibility snapshot:
      - text: "Total: 11.00"

    attachment #1: trace (application/zip)
    test-results/<test>/trace.zip
    Usage:

        troupe show-trace test-results/<test>/trace.zip

<test> stands for the test's own directory, named after its file, its titles, its project and a short hash.

A test ends with one of these statuses:

Status	Meaning
`passed`	The test did what it expected
`failed`	Your test or your app: an assertion, a locator that found nothing, a crash of the app
`timedOut`	The test ran longer than its timeout (60 seconds by default)
`errored`	Infrastructure: the device, the driver or the host failed. It is not counted as your failure
`skipped`	It did not run
`canceled`	The run was stopped while the test was running

Starting the device and installing the app happen outside the test's timeout. Traces shows how to open the trace.

Writing a test

The test file

Fixtures

Find, act, assert

Groups, hooks and steps

Skip, focus and tag

Run it

When a test fails

Next