Create tests with the Assistant

The Assistant is a chat for the current project. You describe a test in plain English; the Assistant proposes the name, steps and Expected Outcome, and saves the test case once you agree. It can also create plans and variables, change existing test cases and start an exploration of your site.

Everything happens in the current project. Anyone in the organization can chat and look things up. Creating or changing test cases, plans and variables, starting explorations and accepting proposals need the Editor role or above. If a Viewer asks for one of these, the action fails with “You need the Editor role or above to change this project.”

1. Open Assistant in the sidebar, or press G then A.

   With no messages yet, the page shows “What do you want to test?” and suggestions such as “What tests do we have so far?”.

2. Click a suggestion, or type your request in the message box.

3. Press Enter to send. Use Shift+Enter for a new line.

The reply streams in under Assistant. While it is working, click Stop to end the reply. Anything it already saved stays saved.

The more specific you are, the better the draft. Name the page, quote the labels you see, and say what success looks like:

Don't look at the website, I'll describe it. I need a test case for sorting the Toolshop
catalogue by price, highest first. On the home page there is a Sort dropdown above the
product list with the option "Price (High - Low)".

The suggestion Explore my site and suggest test cases does not send a message. It opens the Explore page and, if you have the Editor role or above, the New exploration dialog.

The Assistant is instructed to show you the name, steps and Expected Outcome before it creates a test case, and to show the new version before it changes one. Read the proposal, then reply:

• To save it, say so: “Looks good. Create it and add it to the Storefront smoke plan.”
• To change it, say what to change: “Quote the button label exactly” or “Use Start Path /contact instead of the first step.”

The Assistant is also told to keep every test case in a plan, so expect it to ask which plan to use or offer to create one.

Before anything is saved, Testmode checks the draft:

• Steps and an Expected Outcome are required. Each line of the steps becomes one step, and blank lines are dropped. The Assistant cannot create a test case without steps or without an Expected Outcome, and it cannot replace a test case’s steps with none. The action fails and its card shows the error.
• At most 60 steps. A longer test case is refused with “A test case can have at most 60 steps (got N). Split it into several test cases.” The same limit applies when the Assistant replaces the steps of an existing test case. Ask it to split the journey into shorter tests, each with its own outcome.

Every action the Assistant takes appears as a card in its reply, such as looking up plans or creating a test case. The card shows a spinner while the action runs, then a check mark or an error, with a one-line result. An action card confirms the change, for example that a test case was created and added to a plan.

When an action succeeds, some cards link to the result. A card for a test case opens the Test Cases page, a card for a plan opens that plan, a card for a variable opens Settings > Variables, and a card for an exploration opens the exploration.

When the Assistant thinks through a request, a collapsible row appears: Thinking while it works, then Reasoning. Click it to read a summary of its thinking.

Give the Assistant a table of test data, such as a list of products or search terms, to base tests on.

1. Click the paperclip at the bottom of the message box.

   The Upload File panel opens.

2. Drop a CSV or Excel file (.csv, .xlsx or .xls) on the panel, or click it to choose one.

   A preview shows the file name, its rows and columns, and the first few rows.

3. Click Use This File, or Choose Different File to pick again.

4. Type a message about the file, then send it.

   The file goes with that one message.

Good to know:

• Always type a message with the file. Sending a file with an empty message fails.
• A file can have up to 500 rows and 100 columns. For Excel files, only the first sheet is read.
• The Assistant does not read every row. It sees the column headers, the first 5 rows and the total row count. Use a file to show the shape of your data, not to generate one test per row.
• Later messages do not resend the file. The Assistant only remembers what it wrote about it.

If you give the Assistant a URL, or ask about “the site”, it can fetch the page’s text to suggest steps that match the real labels.

It reads public pages only, and it does not run JavaScript. Pages behind a login, apps that build the page in the browser, and sites that block automated requests come back nearly empty or with an error. In that case, describe the page yourself, as in the example above, or start an exploration, which uses a real browser.

Ask, for example: “Explore the Production environment and tell me which tests we’re missing.”

• The Assistant is told to confirm the environment, and a credential if the site needs a login, before it starts.
• You can give a focus, such as “the checkout flow”, starting pages, a page budget, and whether it may propose updates to existing tests.
• Explorations started from chat always use Observe mode.
• Only one exploration can run per project at a time.
• Your organization needs a credit balance above zero. Otherwise the action fails with “Your organization has no credits left. Buy credits to start an exploration.”

While it runs or after it finishes, ask “How is the exploration going?” or “What did it propose?” The Assistant can accept proposals you choose, and accepting them from chat applies them to the project straight away. To reject or edit a proposal, use the exploration page. See Review and apply proposals.

In browsers that support speech recognition, a microphone button sits next to the paperclip.

1. Click the microphone and speak.

   The message box shows “Listening…” and then your words.

2. Click Use transcript to put the text in the message box, or Cancel to discard it.

3. Review the text and send it. Dictation never sends by itself.

Dictation uses US English. In browsers without speech recognition, the microphone does not appear. If the microphone is blocked, recording stops without an error.

Chats are kept per project, in the current browser tab only.

• Moving to another page and back keeps the conversation. Switching project shows that project’s chat and stops a reply that was still coming in.
• Closing the tab loses the conversation. Another tab or device does not see it.
• New chat, at the top right once a conversation has started, clears the conversation immediately, with no confirmation. It also stops a reply that is still coming in.

What the Assistant created stays in your project either way.

What the Assistant cannot do:

• Delete test cases, plans or variables. Delete test cases on the Test Cases page.
• Edit or delete variables. Use Settings > Variables.
• Reject or edit exploration proposals. Use the exploration page.
• Read every row of a file, or read pages that need a login or JavaScript.
• Set a test’s time limit. Set Maximum duration (minutes) in the test case dialog. See Write a test case.
• Create a test case with more than 60 steps, or without steps or an Expected Outcome.

A conversation holds up to 60 messages. When a conversation gets too long, an error asks you to start a new chat: click New chat and summarize where you left off.