Testmode: a quality gate for your software factory

A software factory can turn a requirement into code, open a pull request and build a preview of the application. One question still needs an answer: can someone actually use what it produced?

That is where Testmode fits. It runs tests described in plain language against a web application in a real browser. Inside a software factory, that makes it a component of a quality gate: a way to check the PR’s running application before the change moves forward, and a source of evidence when something fails.

The factory produces the change. Testmode checks the resulting experience. Your release process decides whether the evidence is sufficient to ship.

The connection to Uber’s software factory

In Running a Software Factory Efficiently at Uber Scale, published on August 27, 2026, Uber describes a shift from interactive coding sessions toward managed agents working across the development lifecycle. One example is especially relevant: “completing E2E PRs with visual validation.” The article names that workflow without specifying exactly how its visual checks work. We read E2E here as completing the PR workflow end to end; the phrase alone does not establish a particular end-to-end testing technique.

That suggests a useful design principle for teams building their own factory: give each component a defined job, then evaluate whether it does that job well. For Testmode, the fit is the browser-validation step within that PR workflow: open the application produced by the change, exercise the relevant journeys and check the visible result against the requirement. The architecture below is our proposed application of that principle; it does not describe an Uber deployment of Testmode.

Where Testmode sits

The clearest place is between deploying a PR preview and approving the PR for merge. The coding agent produces a change; the build system gives it a reachable preview URL; Testmode checks that running version of the application. The PR reviewer can then inspect the code and the browser-test evidence together.

The same component can support a later gate between staging and production. A passing preview run does not automatically cover changes introduced when the PR is merged with other work: validate the resulting release candidate too.

The important boundary is the same: there is a working application to test, and a decision that the result can still influence.

Proposed factory architecture. Testmode supplies browser-test evidence; the team's PR policy enforces the gate. The merged release candidate still needs validation.

Each line of defence asks a different question. Code review examines the implementation. Unit tests check individual rules. Integration tests check components working together. Testmode adds a check of the journey through the assembled application: can a customer sign in, place an order or update a record?

Consider a checkout change. A unit test can establish that a discount function returns the right number. An integration test can check that the order service accepts a valid request. A browser journey can expose a different failure: the checkout page never sends the voucher to that service.

Keep the checks that answer each question well. Thousands of calculation cases belong in fast, precise tests close to the code. A smaller set of critical journeys checks whether the pieces deliver the intended experience together. Security, accessibility and performance also need their own appropriate checks.

What visual validation means here

For a PR that changes checkout, a useful check goes beyond reaching the page. Does the voucher discount appear? Is the order total correct? Is the confirmation message visible after payment? A layout change can also require a visual check of whether important content is obscured or an image is missing.

Testmode’s browser runner can inspect the page structure, interact with its elements and request screenshots for checks involving layout, images or colours. It can therefore support both functional journeys and specified visual checks of the resulting interface. Recorded screenshots also give a reviewer evidence to inspect alongside the result.

That is a concrete fit for the validation step Uber names. It does not mean every visual defect will be caught. State the expected outcome and the relevant viewport, and use dedicated visual regression tooling when the requirement is an exact comparison against approved screenshot baselines. Testmode’s screenshot-based inspection should not be presented as pixel-perfect comparison.

Give the component a clear contract

A factory component needs clear inputs and useful outputs. For PR validation, those inputs are the preview URL, a known starting state and an agreed description of success. Select checks from the change’s acceptance criteria and affected journeys, alongside a small set of critical regression checks. That selection is part of the factory’s workflow; it does not require Testmode to understand the code diff.

With Testmode, the team describes the journey in plain language and runs it against the selected environment. The run records browser actions and screenshots, with console and network information available for investigation. That lets a tester, developer or product owner inspect what happened.

The release process must associate the run with the exact candidate build and required tests. A result from an older build cannot approve a newer one.

For a dependable gate, the surrounding factory must preserve that association: which commit and build ran, in which environment, against which version of the test plan. A shared staging URL that changes halfway through a run weakens the evidence. Prefer a stable candidate environment and predictable test accounts and data.

There are two practical ways to use this architecture. A team can operate the gate today by running a Testmode plan against a reachable preview, reviewing its results and holding the PR when a required journey fails. Automating that handoff requires the pipeline to start the right run, wait for completion and attach the outcome to the correct PR commit. It must then enforce that result as a required check. CI/CD triggering and an MCP interface for coding agents are planned integrations; the automated connections in this article describe the intended architecture.

This distinction matters. A testing tool becomes part of a quality gate when its results change what the factory is allowed to release.

Make the gate check the requirement

Suppose the requirement is that a valid €10 voucher reduces a €100 basket to €90. A useful browser test might read:

Using the test customer with an empty basket, add the €100 item. Apply the valid €10 voucher. Verify that checkout shows €90 with no shipping charge. Complete payment using the sandbox payment method, then open order history and verify that the new order also shows €90.

The last step matters. A page could display the discount correctly while the saved order still contains the old total. The check follows the outcome far enough to test the customer’s actual requirement.

The expected €90 should come from an agreed business rule. If the coding agent invents both the implementation and its own definition of success, an incorrect assumption can survive both. Another AI checking the browser does not, by itself, make the expectation independent. A product owner or tester can supply that independent basis through the acceptance criteria.

Testmode’s exploration feature can help grow coverage: it examines application pages, compares them with the existing suite and proposes changes for review. Those proposals are useful starting points. Review them against the intended behaviour before adopting them; the current application may already be wrong.

Close the loop when a journey fails

Imagine that the checkout page shows €90 but the order history shows €100. The gate should hold that candidate and return evidence that someone can act on: the agreed expectation, the observed result, the failing step and the relevant screenshots and logs.

Illustrative checkout example. A retry that eventually passes is not an explanation of the original failure.

A developer or repair agent can investigate that evidence. The cause might be a product bug, an incorrect expectation or a problem with the environment. Correct the cause, then run the checks again against the resulting candidate, including related journeys such as checkout without a voucher.

The release policy should also say what happens when a run cannot complete. An unavailable environment, missing test data or a timeout provides no basis for approval. Keep the candidate on hold until the result is resolved or an authorised exception is recorded. Bound repair attempts and escalate repeated failures so an automated loop cannot keep retrying indefinitely.

Changing an expected result deserves separate review. If an agent changes the assertion from €90 to €100 simply to make the test pass, the factory has lost the requirement it was supposed to protect.

Measure the cost of confidence

Uber’s article connects efficiency to real workloads, model evaluation and cost per completed task. It also shows why unnecessary tool interactions and large context payloads can make an agent session more expensive. That perspective is useful when designing the testing step as well.

Our proposal is to measure the cost of reaching a trustworthy release decision. Include browser execution, investigation, reruns and human review. A cheap run that repeatedly blocks healthy builds may be an expensive gate. A suite that always passes but misses critical defects provides little confidence.

Track four things together:

• Feedback time: how long a candidate waits for the required evidence.
• Signal quality: how often failures reveal real defects versus problems with the test or its environment.
• Escaped defects: which important failures reach production, including whether the suite was meant to cover them.
• Total effort per release decision: execution cost plus investigation, maintenance and review time.

For a future automated handoff, return a concise result with links to the evidence. Let the repair agent retrieve detailed logs when needed. Repeatedly passing an entire browser transcript around the factory adds overhead without necessarily improving the diagnosis. This is a design recommendation, not a measured Testmode cost-saving claim.

Start with the journeys that justify a stop

A useful first gate can be small. Choose a handful of journeys whose failure would make you hold a release: creating an account, signing in, completing a purchase or saving a critical record. Define their starting data and expected outcomes, assign an owner and run them against release candidates.

Initially, review failures together. Learn which ones expose product defects and which ones reflect unstable setup or unclear instructions. Use that evidence to decide which checks are ready to block a release automatically as integration becomes available.

After deployment, a small set of controlled checks can also catch differences between test and production environments. Use dedicated accounts and safe transactions, with a clear response when something fails. Those checks feed the next investigation; they do not retroactively validate the earlier build.

The place for Testmode in your software factory is specific: a quality gate for the user journeys your software must deliver. Your factory can keep producing code, while testers, developers and business teams share a concrete way to judge its output.

Bring one journey you would stop a release for to a Testmode demo. That is a practical place to start building the gate.