# How Self-driving investigates failed runs

URL: https://testmode.ai/docs/investigate-failed-runs/

> When a scheduled, CI or plan run fails, Self-driving re-runs each failed test, reads its evidence and files a verdict: bug, intended change, test problem, environment or flaky. For an outdated test it writes new steps and tries them first.

When a run fails, Self-driving works out why, so nobody has to open a red run and dig through it. It looks at every failed test, re-runs it in a fresh browser, and files a verdict in the [Inbox](https://testmode.ai/docs/inbox/). For a test that is out of date, it also writes new steps, tries them, and offers them for you to apply in one click.

## When it investigates

Self-driving investigates a run that ends **Failed** when the run was started:

- by a schedule;
- by CI or the API;
- for a test plan.

A single test started by hand is left alone: that is usually a test still being written. Each run is investigated once.

Tests that failed on Testmode's side, such as an AI model or browser outage, are left out: they are not your app's problem, and they are not charged. A login that failed or a start page that didn't load is your environment's problem, and is investigated.

It needs Self-driving to be on, with **Investigate failed runs** switched on in its settings (the default). See [Turn on Self-driving](https://testmode.ai/docs/self-driving/).

## What it does

For each failed test, Self-driving:

1. reads the test as written and every operation it ran, with the browser's console errors and failed network requests;
2. looks at the screenshot taken the moment the test failed;
3. checks the test's history: when it last passed, how often its result changes, and whether it was edited since;
4. re-runs the test in a fresh browser, on the same environment and login. A re-run is a run of its own, named **Re-run: …**. An investigation re-runs at most 5 tests.

Then it files one verdict per test:

| Verdict | Meaning | In the Inbox |
| --- | --- | --- |
| **Bug** | The app does something wrong. | Needs you |
| **Intended change** | The app changed on purpose and the test still expects the old behaviour. | Ready to review when a fix passed, otherwise needs you |
| **Test problem** | The test is wrong or too vague to pass reliably. | Ready to review when a fix passed, otherwise needs you |
| **Environment** | The login, the site or the test data is not usable. One item covers every test that failed for that reason. | Needs you |
| **Flaky** | It passed when re-run. | Kept under **Done**; it needs you from its third time |

[Screenshot: An environment verdict in the Inbox: the Long Nose Pliers test failed because the product is out of stock on Production, with the evidence from the failed run and its re-run, the screenshot of the product page, and Fixed and Dismiss. Caption: A real verdict: the re-run failed the same way, and the cause is the shop's stock, not the app or the test.]

It is built to be precise rather than to report everything. A verdict needs a re-run that agrees with it: a flaky verdict needs a re-run that passed, any other verdict one that failed. After a re-run that passed it may re-run once more, since a check can misread a page. A failure it can't explain from evidence is left unreported.

## Fixes it has already tried

For an intended change or a test problem, Self-driving writes the test's steps as they should read and runs them as a **draft**: a run named **Draft: …** that uses the new steps without changing your test. It keeps what the test checks and changes how it gets there, unless the check itself is wrong. An investigation tries at most 3 drafts.

When a draft passes, the Inbox item is ready to review. It shows the new steps under **New steps, tried and passing**, links to the draft run, and offers **Apply to test**. When no draft passed, the item needs you and may show **Suggested steps** that were not run.

## Where you see it

- **The Inbox** lists each verdict that needs someone. See [Work through the Inbox](https://testmode.ai/docs/inbox/).
- **Runs** says under a failed run whether Self-driving is still investigating it, and what its failures turned out to be.
- **Activity** lists every investigation, with **Open run** and, while one is working, **Stop**.
- **Findings** records each failure with the verdict as its triage. See [Findings](https://testmode.ai/docs/findings/).

## Teach it about your app

Under any Inbox item, **Tell Self-driving something about your app** saves a note that every later investigation reads. Say once what it keeps getting wrong: "the cookie banner only shows in the EU", "the demo starts empty on every load".

## Turn it off

In **Self-driving settings**, switch off **Investigate failed runs**. Failed runs are then left for you; their failures still file [findings](https://testmode.ai/docs/findings/) for you to triage.

## Common questions

### Does Self-driving investigate a test I started by hand?

No. A single test started by hand is usually a test still being written, so it is left alone. Runs started by a schedule, by CI or the API, or for a test plan are investigated.

### Do re-runs and drafts cost credits?

Yes. An investigation costs 50 credits when it completes, and its re-runs and draft runs pay for the actions they run, like any test.

### Can it change my test without asking?

Not from an investigation. It runs new steps as a draft that leaves your test as it is, and offers them in the Inbox. They replace your steps only when someone clicks Apply to test.

## Related

- [Work through the Inbox](https://testmode.ai/docs/inbox/): Apply a fix, confirm a bug, dismiss.
- [Troubleshoot failed tests](https://testmode.ai/docs/troubleshoot-failed-tests/): Work out a failure yourself.
- [How credits are counted](https://testmode.ai/docs/how-credits-are-counted/): What an investigation costs.