Write effective steps and Expected Outcomes

Reliable Testmode tests start on the right page, let a credential handle login, quote the site’s exact wording and describe an outcome the agent can see. This page covers the rules that follow from how Testmode runs a test; for general principles of writing tests in plain English, read natural language test automation.

When a test starts, Testmode opens a fresh cloud browser and, if a Login Credential is set, signs in first or loads a login saved earlier for that credential. It then opens your environment’s Base URL plus the Start Path. The agent receives the test name, every step and the Expected Outcome together, as one task. It works out how to carry out the steps, then reports Passed or Failed with a summary.

Two things follow from that:

• The name matters. The agent reads it, so a name that no longer matches the steps can mislead it. Rename a test when you change what it checks.
• There is no result per step. The agent gives one verdict for the whole test, judged against your steps and the Expected Outcome.

Each test also has limits:

• Time. A test may run for its Maximum duration (minutes), which you can set per test case from 1 to 60 minutes. Left empty, it uses the project’s default, which is 15 minutes unless changed. A test that runs out of time fails with “Timed out: the test did not finish within …”. See Maximum duration.
• Turns. The agent can take only a limited number of turns, and a longer time limit allows more.
• Progress. If the agent keeps repeating an action, or the page stops changing, Testmode ends the test early. Its failure reason starts with “Stopped early:” and ends with “Check that the steps match what this page offers.”

Short, focused tests that match the page leave plenty of room.

Use Start Path instead of a “go to” step. The browser opens that page before the agent starts, so it spends no turns getting there.

Write a path, never a full URL. The path is added to the environment’s Base URL, so a full URL produces a broken address. Leave Start Path empty to start on the Base URL.

For pages whose address can change, such as a product page, start on a stable page and search or click your way there. The Toolshop examples search for a product by name rather than putting a product id in the path.

Attach a Login Credential instead of writing login steps. Testmode signs in before the test starts, and the agent is told it is already signed in.

• Don’t write “Log in as Jane” or type a password in a step. With a credential attached, the agent does not log in itself. If it meets a login page, the test fails with “Session expired or login required”. If the test used a saved login, Testmode also discards that login, so the next test with the credential signs in again.
• Keep passwords in credentials, not in steps or variables. What the agent types is shown on the run’s timeline.
• Start on a page that clearly shows a signed-in state, such as an account page. Testmode looks at the start page to decide whether a saved login is still valid, and a clear signal makes that check reliable.

1. Look at the page heading and the navigation menu. Do not click "Sign out".

Expected Outcome: The page heading reads "My account", and the navigation menu shows the customer's name instead of "Sign in".

See Credentials and automatic login for setup.

Put buttons, fields, options and messages in quotation marks, exactly as the site shows them, including capitals and punctuation. The agent finds buttons, fields and text by their labels, so exact wording removes guesswork.

If a label changes on the site, update the step. A test that quotes old wording may still find the element, but it can also fail for the wrong reason, or be stopped early because the agent keeps looking for something that is no longer there.

Describe what is on the screen when the test has passed: a heading, a message, a list of items, a value in a field. Avoid outcomes the agent cannot see, such as “the email is sent” or “the database is updated”.

Say what must not happen too, when it matters. A validation test is stronger when it also says the success message does not appear.

The agent reads the page’s text and structure first. It takes a screenshot when a check is visual, such as layout, images or color. If your outcome depends on how something looks, say so in words:

The product page for "Combination Pliers" shows a product photo, not a broken image.
The "Sort" dropdown is shown above the product list.

Without that cue, the agent may confirm the text is present and never check how it looks.

Tests in a run execute independently, each in its own fresh browser, in no guaranteed order.

• Don’t rely on another test. If one test adds a product to the cart, another test in the same run starts in a different browser and should not expect to see it. Each test sets up what it needs in its own steps.
• Split long journeys. A test that searches, filters, adds to cart, checks out and reviews an order has many chances to run out of time. You can raise its Maximum duration (minutes), but several shorter test cases, each with one clear outcome, point to the one thing that broke.
• One outcome per test. When a test fails, its name should already tell you what broke.

The agent may only navigate directly within the start page’s host and its subdomains, and it is told to fail rather than leave your site. Write tests that begin and end on one site.

• Flows that move to another domain partway through, such as an external payment page or “Sign in with Google”, are not guaranteed to work.
• Tests cannot read email or SMS, so they cannot follow a confirmation link or enter a one-time code.
• This restriction guides the agent. It is not a security boundary.

When a page loads content after an action, tell the agent what to wait for. Name the element or message, not a length of time:

Wait until the heading "Searched for:" appears above the products.
Wait until the product list updates.

Each wait the agent makes is short, so naming what should appear tells it what to check for once the page has settled. If the agent waits many times in a row and the page never changes, Testmode stops the test early rather than letting it wait out its time limit.

Some sites need a second click to finish an action. On the Toolshop’s payment step, the first “Confirm” only validates the payment and shows a message; the second “Confirm” places the order. Write the second click as its own step, with the condition that triggers it:

6. On the "Payment" step, choose "Cash on Delivery" under "Payment Method" and click "Confirm".
7. When a message about the payment appears under the form, click "Confirm" again.

Expected Outcome: The page shows "Thanks for your order! Your invoice number is" followed by an invoice number.

Name: Contact form test
1. Go to the contact page.
2. Fill in the form with a short message.
3. Submit it.

Expected Outcome: Validation works.

Name: Contact form requires a 50-character message
1. Fill "First name" with "Maren", "Last name" with "Veld" and "Email address" with "maren.veld@example.com".
2. In "Subject", choose {{Contact Subject}}.
3. Type {{Short Message}} into "Message" and click "Send".

Expected Outcome: The form shows "Message must be minimal 50 characters" and the text "Thanks for your message! We will contact you shortly." does not appear.

What changed: the name says what is checked, Start Path replaces the navigation step, every field and button is quoted, the values come from variables, and the outcome names the exact message and what must not appear.

Name: Send contact message
1. Log in, open Contact and send a support request about a broken drill.

Expected Outcome: The message gets sent.

Name: Contact form sends a support message
1. Fill "First name" with "Maren", "Last name" with "Veld" and "Email address" with "maren.veld@example.com".
2. In "Subject", choose {{Contact Subject}}.
3. Type {{Support Message}} into "Message" and click "Send".

Expected Outcome: The page shows "Thanks for your message! We will contact you shortly."

What changed: the login step is gone (the contact form needs no login; a test that does would use a credential), one crowded step became three, the message text lives in {{Support Message}}, and the outcome is a message the agent can read on the page.

A variable name must match exactly, including capitals and spaces. {{Support Message}} is replaced with its value, but a misspelling such as {{Suport Message}} reaches the agent as literal text, braces included. Add variables with Insert Variable rather than typing them to avoid this. See Variables.