Read run results and evidence

Every Testmode verdict comes with its evidence. While a test runs, you can watch the browser live. When it finishes, you get the agent’s summary, a screenshot after every action it took, and its reasoning for each one.

So a PASSED or FAILED is something you can check, not something you take on trust.

Click Runs in the sidebar. The list shows the project’s runs, newest first, with Name, Triggered By, Status, Time and Environment, 15 to a page. It updates by itself as runs progress.

• Search by run name or by who started it.
• Filter by status or environment.
• Triggered By shows the person who started the run, or Scheduled.
• A failed run shows its first failed test’s summary in red under its name, so you can often see what broke without opening it.

Click a row to open the run.

While a run is in progress, its page shows the Live Browser View: the cloud browser, streamed as the agent works. Results appear as each test finishes, without reloading.

• The view is view-only. Your clicks and typing do not reach the browser.
• On a run with several tests, the run page shows one running test at a time. Open a test’s own page to watch that test.
• Click the Live Browser View header to collapse or expand it.

Until a test has a browser, a blue line takes the place of the live view and says what the test is waiting for:

For example, “Waiting for a browser (N ahead)” means N tests are ahead of this one, and the number updates while you watch. See Why a test is waiting for every message and what it means.

Runs from different organizations, and different runs within your organization, take turns for browsers. A large run does not hold up a small one until it finishes, so the tests of your run may start in between tests of other runs.

A test can also go back to Queued after it started, for example when Testmode restarts. It then starts again from the beginning, and Test Steps shows only the new attempt.

A plan run, or an All Test Cases run, opens with a summary:

• a progress bar with one segment per test
• counts of passed and failed tests, plus pending ones while the run is active
• Triggered By, Environment, Duration and Started
• Test executions: one card per test, with failed tests first, then running, queued and passed

A queued test’s card shows what it is waiting for under its name, such as “Waiting for a browser”.

When at least one test failed, click Failures only to hide the rest. Click Show all to bring them back.

Click a card to open that test’s page.

A test’s own page, and the page of a run with only one test, shows:

• the verdict, Passed, Failed or In progress, with the test’s duration
• the agent’s summary of what it found, when the test did not fail
• the Failure reason, when it did
• who started the run, the environment and the start time
• Test Steps, and below them the console and network logs

When a test fails, a red Failure reason box explains why. It is one of two things:

• The agent’s explanation. The test ran, and what the agent saw did not match your Expected Outcome. The agent says what it found instead.
• A system reason. The test could not be carried out or finished. For example, “Login failed: …” means automatic login did not succeed. “Timed out: the test did not finish within N minutes …” means the test used up its maximum duration. “Stopped early: …” means the agent stopped making progress, so Testmode ended the test before its time limit.

Run and test statuses lists the reasons you may see and what each means for your credits.

In Pellbrook Digital’s Checkout regression run, the test “Long Nose Pliers can be added to the cart” fails. The product is out of stock on the Toolshop, so the confirmation in the Expected Outcome never appears. The agent’s attempts leave the page unchanged, so its Failure reason starts with “Stopped early:” and names the product page. Open the step whose screenshot shows “Out of stock” to see why.

A test that failed before the agent took any action, such as after a failed login, shows “No Steps Recorded” instead of a timeline. Its credits are returned; see How credits work. To find the cause, see Troubleshoot failed tests and Troubleshoot login.

Test Steps lists every browser action the agent took, in order. The number in the heading counts those actions.

Each row shows:

• what the agent did, such as Clicked, Typed 'Long Nose Pliers' or Navigated to practicesoftwaretesting.com
• the host of the page it was on
• an image icon when a screenshot was taken after the action

Click a row to expand or collapse it; the first one starts expanded. You see the screenshot taken right after the action, and the agent’s reasoning. Long reasoning is shortened; click Show full context to read all of it. When the agent took several actions in one go, they share the same reasoning.

If the agent reached its verdict just by looking at the page, the timeline has a single “Observed the page” row with a screenshot and the full reasoning.

While a test runs, new rows appear as the agent acts, ending with “executing next step…”.

Click any screenshot thumbnail in Test Steps to open the viewer.

• Press the Left and Right arrow keys to move between steps, and Esc to close.
• On a test with several screenshots, the list on the left jumps to any step, and the ACTION and REASONING panel shows what the agent did and why.
• Zoom from 50% to 300%. Moving to another step resets the zoom to 100%.
• Download the image you are looking at.

Screenshots are private to your organization. If a page has been open for a long time and images stop loading, reload the page.

In the row menu on the Runs page, choose Copy link to copy the run’s address. Only members of your organization can open it.

If the run belongs to another project or organization you are a member of, opening the link switches Testmode to that project for you.