# Run tests from GitHub Actions and other CI

URL: https://testmode.ai/docs/github-action/

> Run Testmode tests from a pipeline: the Testmode AI GitHub Action with its inputs and outputs, and the same script for GitLab, CircleCI, Jenkins or any CI with bash, curl and jq.

Run your Testmode tests from CI, for example after every deploy to staging, and fail the build when a test fails. You need a [project API key](https://testmode.ai/docs/api-keys/).

## GitHub Actions

1. Create an API key and add it to your repository as the secret `TESTMODE_API_KEY` (**Settings** > **Secrets and variables** > **Actions** in GitHub).

2. Save a workflow such as `.github/workflows/testmode.yml`:

```yaml
name: Testmode
on:
  push:
    branches: [main]
  workflow_dispatch:
jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: amber-digital-bv/testmode-ai-action@v1
        with:
          api-key: ${{ secrets.TESTMODE_API_KEY }}
          environment: Staging
          test-plan: Smoke
```

3. Push. The job starts the run, waits for it, prints each test's result, and fails when a test fails.

**Settings** > **API & CI** shows this snippet filled in with the environment and plan you pick. The action is [Testmode AI on the GitHub Marketplace](https://github.com/marketplace/actions/testmode-ai).

### Inputs

| Input | Default | Meaning |
| --- | --- | --- |
| `api-key` | (required) | The project API key, from a secret |
| `environment` | The project's default environment | Environment name or id |
| `test-plan` | | Test plan name or id |
| `test-case-ids` | | Test case ids, separated by commas or spaces |
| `tags` | | Run every enabled test case with any of these tags |
| `name` | `<workflow> #<run> · <branch> · <sha>` | The run's name in Testmode |
| `wait` | `true` | Wait for the run and fail the job unless it passed. `false` returns once it is queued. |
| `timeout-minutes` | `30` | Stop waiting after this long. The run itself goes on. |
| `api-url` | `https://api.testmode.ai` | Only for another deployment |

Give exactly one of `test-plan`, `test-case-ids` or `tags`. Names match exactly, ignoring case; pass the id when two share a name.

### Outputs and exit

The action sets `run-id`, `run-url` and `status` (`PASSED`, `FAILED`, `SKIPPED`, or `QUEUED` when not waiting), and writes a job summary with each test. The step succeeds when the run passed, or once it is queued with `wait: false`; it fails when the run didn't pass or waiting timed out, and when the run couldn't be started or read.

### Run after a deploy

```yaml
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - run: ./deploy-staging.sh
  e2e:
    needs: deploy
    runs-on: ubuntu-latest
    steps:
      - uses: amber-digital-bv/testmode-ai-action@v1
        with:
          api-key: ${{ secrets.TESTMODE_API_KEY }}
          environment: Staging
          tags: smoke
```

## Other CI

The action's script runs unchanged in any CI with bash, curl and jq. Set the key and what to run as environment variables:

```bash
export TESTMODE_API_KEY=tmk_...        # from your CI's secret store
export TESTMODE_TEST_PLAN="Smoke"      # or TESTMODE_TAGS / TESTMODE_TEST_CASE_IDS
export TESTMODE_ENVIRONMENT="Staging"  # optional
curl -fsSL https://raw.githubusercontent.com/amber-digital-bv/testmode-ai-action/v1/run.sh | bash
```

It exits 0 when the run passed, 1 when it didn't pass or waiting timed out, and 2 when the run couldn't be started or read. The script's header lists every setting.

Prefer your own calls? Use the [REST API](https://testmode.ai/docs/rest-api/) directly.

## Good practice

- **Run against a test environment.** Tests do what their steps say, including orders and messages.
- **Give busy pipelines their own keys.** One key allows 60 requests a minute, and a waiting job polls every 15 seconds.
- **Name runs after the change**, such as the pull request, so **Runs** shows which change a result belongs to. The action does this by default.
- **Every test counts** one of your plan's test runs, like a run from the app.

## Related

- [API keys](https://testmode.ai/docs/api-keys/): Create and revoke keys.
- [REST API reference](https://testmode.ai/docs/rest-api/): The endpoints the action uses.
- [Test plans](https://testmode.ai/docs/test-plans/): Group what CI should run.