Email run reports

Testmode can email a report when a run finishes, with the verdict, every test’s result and a link to the run. You set reports in two places: on each schedule, for scheduled runs, and on your profile, for runs you start yourself.

Each schedule has its own report settings, under Email reports in the schedule dialog.

1. Open the plan from Test Plans, select Schedules, and click Add Schedule, or the pencil icon on an existing schedule.

2. Under Email reports, choose when to send. The options are described in the table below.

3. Add at least one address under Recipients, unless you chose Never.

4. Click Save.

On the Schedules tab, the schedule then shows its recipients followed by “on failure” or “every run”.

For example, Pellbrook Digital’s Storefront smoke schedule sends a report only when a test fails, to qa@example.com and maren.veld@example.com.

Type an address and press Enter, a comma or a space, or click away, to turn it into a chip. You can also paste several addresses at once, separated by commas, semicolons or spaces.

• To remove an address, click the x on its chip, or press Backspace in the empty field to remove the last one.
• You can add up to 20 addresses per schedule.
• As the dialog says: “Any address works, not only members of this organisation.” A client or a shared team inbox can receive reports without a Testmode account.

Testmode reads the recipient list when a run finishes. An address you add later receives reports for later runs, not for runs that already finished.

Runs you start yourself, from the Run tests dialog or with Re-run, do not use any schedule’s list. Instead, each person chooses on their own profile.

1. Open the organization menu at the bottom of the sidebar and choose Profile.

2. In Email Notifications, open Runs I start and choose Don’t email me (the default), Email me when a test fails or Email me after every run.

3. Click Save.

Reports go to your profile’s email address, which the page shows under the setting. The setting covers every run you start, including plan runs, in every organization you belong to. It does not cover scheduled runs.

If a run qualifies under a schedule’s list and your own setting, you receive one email, not two.

• Testmode sends one report per run, when the run ends Passed or Failed.
• Only when a test fails means the run ended Failed, which happens when any test in it failed.
• Runs recorded as Skipped send no report. A scheduled run is skipped when the organization does not have enough credits, or when none of the plan’s enabled test cases has steps or an Expected Outcome. See When a schedule does not run.
• The email can arrive a little after the run ends.

The subject line tells you the verdict before you open it. After [Testmode] it has a check mark emoji for a passed run or a cross mark emoji for a failed one, then the result:

[Testmode] (check mark) Storefront smoke passed on Production
[Testmode] (cross mark) Checkout regression: 1 of 4 tests failed on Production

For a plan run, the run name is the plan name.

The email body has:

Under the button, the email repeats the run’s address as a plain link, for mail clients that block buttons. The footer says why you received the report and where to change that.

A plain-text version is included for mail clients that do not show HTML.