Contact salesStart now

Research & technology

Show, don't prompt: how QA Wolf turns a recording into a real test

Goran Gajic

Point QA Wolf's agent at an app and it'll explore, map the pages, propose coverage, and write the tests on its own. That's what you want when you're after breadth. But a lot of testing knowledge lives in someone's head, like the promo code that only works on Tuesdays, the approval step that only appears for expenses over $500, or the export button only admins can see. Describing that in a prompt is slow and loses detail, while showing it takes about 30 seconds.

So we built a recorder into the agent's chat. You open a live browser next to the conversation, press record, and click through the flow. The agent watches each action as it happens, keeps a clean list of steps, and turns that list into a test when you're done.

A test built from every click and keystroke you make while recording would be a mess, though. So the agent cleans up. It tidies the steps, fills gaps, and adds the checks you implied before writing the code. What it can't do is change what you meant to test. Every edit lands in the step list you're watching, and every step it adds comes from something that really happened in the browser.

Recording Starts Where Your Tests Start

Most flows begin the same way: sign in as someone, maybe create some data, then get to the part that matters. Recording that every time is slow, and it gives you a second copy of a login your test suite already has.

So before recording starts, the agent looks for the setup your flows already use. Press record and the panel says "Looking for setup steps…" while the agent searches your workspace for helpers that sign in, create test data, or switch accounts. It reads each helper to learn what it takes, then offers what it found as a short list in the chat, like "Sign in as an admin" for a helper such as loginAs('admin'). Login helpers carry a note: "Uses your workspace's login, so you don't have to record it."

Keep the ones you want and press Start recording. The setup runs in the live browser, and recording starts on whatever page it leaves you on. To record the same flow as a regular user instead of an admin, ask, and the agent offers the same helper with a different persona. If your workspace has no helpers yet, the agent offers to open the app's start page, and Skip setup is there if you'd rather do it all by hand.

A person asks the agent to record the checkout as an admin. The recording panel says it is looking for setup steps, while the agent searches the workspace. The agent then offers two helpers the workspace already has, in a card in the chat: Sign in as an admin, marked as using the workspace's login so it doesn't have to be recorded, and Add a tent to the cart. The person unticks the cart helper and presses Start recording. The sign-in helper runs, the shop in the live browser now shows Ada signed in as an admin, and the panel starts recording. Their first clicks, Tents and Add to cart, become the first steps.
Setup runs your workspace's own helpers, so the recording starts signed in. Sped up; the store is a stand-in.

The setup doesn't end when recording starts. The replay runs it again in a clean browser, and the finished flow calls the same helpers on its first lines. Nothing about signing in is recorded twice, and when your login helper changes, the recorded test changes with it.

You and the Agent Share One Step List

The core problem is keeping you and the agent looking at the same list of steps when three machines are involved. Your clicks land in a browser that runs in the cloud. You watch it, and the step list, from the QA Wolf app, while the agent works on a machine of its own. Every click shows up in the step list for both of you as it happens, and any change either of you makes shows up for the other right away. If a connection drops, the list picks up where it left off.

For each click, the cloud browser also works out a locator, the Playwright code that finds the same element again later, like getByRole('button', { name: 'Add to cart' }). The locator travels with the step.

QA Wolf during a recording. In the middle, the Live browser pane shows the cloud browser, Chrome, open on the store being tested. On the right, the chat holds the recording panel, showing the newest step, Click Tents. Below the app, the agent's machine holds its own copy of the step list. Your pointer clicks Add to cart, and Click Add to cart appears in both lists at once. Then it clicks Cart, and Click Cart (1) appears in both. The two lists stay the same.
Each click adds the same step to your list in the chat and to the agent's copy on its own machine. The store is a stand-in.

The Recorder Writes Down What a Person Would

Of course, browsers are noisy. Typing an email address fires a key press for every letter, and filling in one form can add a few corrections, a scroll, and a click that gets reported twice. A step list showing all of that is useless, and the agent would spend its first turn cleaning it up. The list should read the way a person would write the flow down, and it takes two passes to get there: one in the page, one on the list.

The first pass happens in the recording script. Instead of sending each key press, it waits until you stop typing and sends one fill with what the field now says. If you leave the field or the page goes away before then, it sends the fill right away. Otherwise a one-time-code field that submits on the last digit would lose its value.

But that still leaves plenty of noise. Let's look at someone signing in. The script sends 11 events. There's a click into the Email field, four fills, a Backspace, a scroll, the Sign in click reported twice, and two page changes as the app redirects. The second pass, a set of capture rules, turns those into three steps: type the email, type the password, and click Sign in.

A person signs in while recording, and the page sends eleven events. Each passes through the recorder's capture rules on its way to the recording panel, which keeps three steps: type shopper@example.com into Email, type the password into Password, and click Sign in. The click into the Email field folds into the typing, and each later fill on that field updates the same step. A Backspace shows as a step until the next fill replaces it. A scroll is noise. The second report of the same click, 90 ms later, is dropped. The two page changes the app makes on its own after signing in aren't steps.
Eleven events in, three steps out. The panel is the app's own; the rules are the recorder's. The timestamps are illustrative.

Each rule is small and has its own tests:

  • Typing is one step: Repeated fills on the same field merge into one, and clicking into a field before typing isn't a step of its own.
  • Corrections disappear: Backspace, Delete, the arrow keys, etc. drop out when a later fill on the same field replaces them.
  • Noise is dropped: Hovers and scrolls never become steps, though the agent still sees them.
  • Double fires are ignored: When the recorder reports one click twice within 250 ms, the step list keeps one.
  • Redirects collapse: When the app changes the address on its own within three seconds of your last step, the step list keeps the address it settled on, not every hop.

The Agent Improves Your Steps but Never Invents One

What you record is rarely a finished test. You know the cart should show one item, but you never added that check. The recorder might miss an action, you might click something you didn't need to, and the order you clicked in isn't always the order that matters. So the agent closes the gap. It can add a check you implied, put back an action the recorder missed, reword a step so the next person can read it, drop a redundant click, or move steps into the order that matters. Every edit shows up in your step list as it's made.

What it can't do is add a step for something it never saw. The agent sees every browser event, not just the ones that become steps, so it knows the locator of anything you hovered over. To add a step you didn't record, it hovers over the element in the live browser and copies the locator from that event. It never makes one up.

You and the agent also share the same controls: record, pause, replay, start over, and discard.

The Steps Aren't a Test Until They Replay

Steps that worked once, with your cookies, aren't a test yet. So when you press Create flow, the agent reviews the step list and then replays it from a clean start before writing any code. The hard part is that the step list records what you did, not how long the app took to respond. Replay the steps too quickly and a good step fails on a page that's still loading. Retry blindly and the replay can click the same button twice. The replay has to tell a step that's actually broken from one that only needed another moment.

The clean start comes first. The replay closes any other tabs, clears the browser's cookies, and clears the data the app's site has saved, which signs you out. Then it runs the setup you chose and opens the start page.

Then the replay runs the steps one at a time. It skips checks like “The cart shows 1 item,” because a check stays in words until the agent turns it into code for the finished test. Each other step gets up to three tries, a second apart, since a page that's still loading can make a good step fail once. A click that opens a slow page can look like a failure even though it worked, so a step whose page has already moved on counts as passed instead of being clicked again.

When a step still fails after all that, the problem is real, and the replay stops and says why in plain words. For example, if the step list is missing the click on Checkout, the replay stops before typing the name and points out that no step before it goes to /checkout.

After Create flow, the agent replays a recorded checkout in a clean browser, driving it itself. It clicks Tents, Add to cart and Cart, then stops at the step that types a name on the checkout page: the panel turns that step yellow and says no step before it goes to /checkout, so one may be missing. The agent says the same in the chat. The person presses Resume, which records again from where the replay stopped, and clicks Checkout; the new step goes in before the one that stopped. They press Create flow again, the replay runs every step through to the confirmed order, and the flow is created.
A stopped replay, in the agent's own words. New steps go in before the step that stopped, not at the end. The store is a stand-in.

A stopped replay has three ways out, and either you or the agent can take them:

  • Record from here: Press Resume and do the missing part by hand. New steps go in before the step that stopped, not at the end.
  • Let the agent fix it: If you started the replay, the agent learns where the replay stopped. When the agent can't find an element, the agent hovers over that element in the live browser, removes the broken step, and adds the step back with the real locator.
  • Drop the step: Sometimes the step was never needed. Removing the step starts the replay again from the top.

Recorded Tests Get the Same Code as Any Other

Even after the replay passes, the step list is just raw Playwright code, one line per step. That's a fine record of what happened, but the locators repeat, nothing is reused, and the next person can't tell what each block is for. We didn't want recorded tests to be a second-class kind of test, and the step list is never saved as is. It becomes a brief, and the agent writes the flow from it the same way it writes every other flow, in our page-object style (@qawolf/pom).

The brief is a flow file that opens with your setup calls, followed by a numbered list of to-dos, one per step. Each one carries the locator the recorder chose, the alternates it found, the value typed, and your checks in your own words. A subagent picks it up with the page-object skill loaded and rebuilds it: Each locator lives once, in a page object, and each action is a named method that returns the page it leads to.

flows/cart-shows-the-added-tent.flow.ts
import { flow } from "@qawolf/flows/web";

export default flow("Cart shows the added tent", {
  target: "Web - Chrome",
}, async ({ test }) => {
  await test("Cart shows the added tent", async () => {
    // TODO 1: Click Tents
    //   locator:    getByRole("link", { name: "Tents" })
    //   on:         howl-outfitters.example/
    // TODO 2: Click Add to cart
    //   locator:    getByRole("button", { name: "Add to cart" })
    //                 .first()
    //   alternate:  getByText("Add to cart").first()
    // TODO 3: Click Cart (1)
    //   locator:    getByRole("link", { name: /^Cart/ })
    // TODO 4: Check, in your words
    //   "the cart shows 1 item"
    //   on:         howl-outfitters.example/cart
  });
});

What the recorder hands over: each step with the locator it used, the alternates it found, and your check in your words.

The same recording at three stages. The marked lines carry what you recorded: the Tents link and the cart.

Because the last step is ordinary flow writing, the result gets everything a flow the agent wrote on its own gets, including the page objects your suite already has, which it reuses instead of making new ones. There's no separate “recorded test” style to maintain, and no drop in quality for having recorded it.

Recording and Exploring Work Together

Recording doesn't replace the agent exploring on its own, and most sessions won't be all one or the other. Sometimes the agent leads and you review, sometimes you lead and the agent tidies up, and often it's both. The agent gets stuck on a login that needs a hardware key, hands you the browser, and takes it back once you're through.

What makes that work is boring on purpose. There's one step list that both sides edit, setup that reuses what your suite already has, tools that let the agent improve your steps but never invent what you didn't show it, and a last step that writes the same quality of code no matter who did the clicking.

More from the blog

All posts