Visual Regression Testing: How It Works

Visual regression testing compares screenshots of your app against approved baseline images to catch changes in how it looks. When a new screenshot differs from its baseline by more than the tolerance you set, the test fails and shows you which pixels changed.

It catches the bugs functional tests walk past. In a test run for this post, a CSS refactor renamed one variable and turned a pricing card's "Start free trial" button white on white. The Playwright test that checks the button is visible still passed, but the screenshot test failed on 11,141 changed pixels.

What is visual regression testing?

Visual regression testing is automated testing that checks a user interface still looks the way it did, by comparing screenshots against approved baselines. It's a kind of regression testing, where the goal is to catch a change that broke something that used to work and "work" means "look right."

Visual tests catch the bugs that sit between the code and the eye:

• Layout shifts: a padding change pushes a card grid onto two rows, or a sidebar overlaps the content.
• Hidden or missing elements: a button that renders but can't be seen, or an icon that fails to load.
• Wrong styles: a color, font weight or border radius that changed in a shared stylesheet.
• Responsive breakage: a layout that holds on desktop and breaks at a phone's width.
• Graphics a locator can't read: charts, maps and anything drawn on a <canvas>, which isn't part of the DOM.

Visual testing is the broader term, covering any check of how a UI looks, including a person clicking through a staging build. Visual regression testing is the automated, repeatable version: the same screens, in the same conditions, compared on every run.

How does visual regression testing work?

Every visual regression test runs the same loop: capture a baseline, capture a new screenshot on each run, compare the two, and fail if the difference passes a tolerance.

1. Capture a baseline. The first run has nothing to compare against, so the tool saves the screenshot as the expected image. Review it before you trust it, since a baseline of a broken screen passes forever.
2. Capture a new screenshot on every run, in the same browser, viewport and operating system as the baseline.
3. Compare the two images with a diffing method: pixel, perceptual or DOM.
4. Pass or fail against a tolerance. On a failure, most tools save three images: expected, actual and a diff that highlights the changed pixels.
5. Fix or accept. If the change is a bug, fix the code. If it's intended, accept the new screenshot as the baseline.

In Playwright visual regression testing, that whole loop is one assertion, toHaveScreenshot(). The first run fails on purpose, with "A snapshot doesn't exist… writing actual," and saves the baseline in a -snapshots folder next to the test file, and every later run compares against it.

Pixel, perceptual and DOM diffing: what's the difference?

The comparison step is where tools differ most, and there are three main approaches:

• Pixel diffing compares the two images pixel by pixel and counts the pixels that differ. It's exact and simple, and it's what Playwright and Vitest do by default, with the pixelmatch library. It also flags differences no person would notice, like anti-aliasing that renders a hair differently on another machine.
• Perceptual diffing compares images the way a person sees them. A per-pixel color tolerance is the simplest form: Playwright's threshold option sets how different two pixels can look before they count as changed, from 0 (strict) to 1 (lax), with a default of 0.2. Commercial tools go further with AI models; Applitools calls its approach Visual AI.
• DOM diffing compares the page's structure, positions and computed styles instead of its pixels. Rendering noise can't touch it, but it can't see what only exists as pixels: a canvas chart, an image that loaded blank, a web font that fell back to a system font.

Playwright's built-in check is pixel diffing with that per-pixel color threshold.

How do thresholds and masking keep visual tests stable?

Thresholds decide how much difference is allowed, and masks take changing content out of the comparison.

Playwright gives you three tolerance settings:

• threshold: how different one pixel's color can be before it counts as changed (default 0.2).
• maxDiffPixels: how many changed pixels the whole screenshot can have.
• maxDiffPixelRatio: the same limit as a share of all pixels, so 0.01 allows 1%.

Masks cover elements you expect to change, such as timestamps, ads, avatars and live counts. Playwright paints a solid box over each masked element (pink, #FF00FF, by default) before it compares.

In the run for this post, an unmasked timestamp alone changed 28 pixels between two loads of the same page, enough to fail a comparison with no tolerance. With the timestamp masked, the same test passed on every run.

A tolerance can hide a real change, too. On the 328 × 301 pixel pricing card in that run, a 1% tolerance allows about 990 changed pixels, and changing the price from $49 to $59 changed 113, so the visual test passed. Keep tolerances as low as your rendering allows, screenshot the smallest element that matters, and check text and numbers with a functional assertion.

Drag the slider to see how a 1% tolerance treats a screenshot as more of it changes:

What does a visual regression test look like?

Here's the test file from the run above, on Playwright 1.63 in Chromium. It assumes baseURL in playwright.config.ts points at the app. The first test is a functional check, and the second is the visual one.

import { test, expect } from '@playwright/test';

test('pricing card has a working call to action', async ({ page }) => {
  await page.goto('/');
  await expect(page.getByRole('button', { name: 'Start free trial' })).toBeVisible();
});

test('pricing card matches its baseline', async ({ page }) => {
  await page.goto('/');
  const card = page.getByTestId('pricing-card');

  await expect(card).toHaveScreenshot('pricing-card.png', {
    mask: [card.getByTestId('updated-at')], // a timestamp that changes on every load
    maxDiffPixelRatio: 0.01,                 // allow up to 1% of pixels to differ
  });
});

What happened across four runs:

1. No baseline yet: the visual test failed with "A snapshot doesn't exist," and Playwright saved pricing-card-linux.png as the baseline. The functional test passed.
2. Same page, new timestamp: both tests passed. The mask hid the timestamp.
3. A CSS refactor renamed --brand to --brand-primary: the button lost its background, and its white label vanished against the white card. The functional test still passed. The visual test failed with "11141 pixels (ratio 0.12 of all image pixels) are different," and saved expected, actual and diff images.
4. Stylesheet fixed: both tests passed.

To accept an intended change, run npx playwright test --update-snapshots and commit the new image with the code that changed it.

When should you use visual tests instead of functional assertions?

Use a visual check when the question is how something looks, and a functional assertion when it's what something says or does. Most screens need both: a functional check first, so the page is in the right state, then a screenshot.

Visual checks earn their place for:

• Layout and styling, especially after changes to shared CSS or a design system.
• Charts, maps and canvas elements that locators can't read.
• Screens with strict design requirements, such as checkout or pricing.
• Responsive layouts at each breakpoint, and the same screen in different browsers, where cross-browser testing catches engine-specific rendering bugs.

Assertions from functional testing are the better tool for:

• Text, prices, counts and anything else you can read from the DOM.
• State and behavior: a form submits, a toggle switches, an error appears.
• Content that changes on every load, like timestamps, feeds and live data.

A full-page screenshot fails for dozens of reasons, and the diff won't tell you which one matters. Keep visual checks on the few screens and components where looks are the requirement.

Why are visual regression tests flaky?

Visual tests flake when something changes the pixels between runs without anyone changing the code. The usual causes:

• Fonts. A web font that hasn't loaded yet renders in a fallback font, and operating systems smooth text differently. Wait for fonts before capturing (Playwright's screenshot assertion does), and render in one environment.
• Animations and transitions. A screenshot taken mid-animation never matches. Playwright's toHaveScreenshot() disables CSS animations and transitions and hides the text caret by default, then waits until two screenshots in a row match.
• Rendering differences. The same page renders differently across operating systems, browser versions, hardware and headless modes. Playwright's docs warn about this, and Playwright puts the platform in each baseline's file name (pricing-card-linux.png) for the same reason.
• Dynamic content. Timestamps, ads, avatars, carousels and random test data. Mask them, freeze them or seed the data.
• Timing and state. Lazy-loaded images, a scroll position, a hover effect left on a button. Wait for the content, scroll to the element, and move the mouse away before capturing.
• Viewport size. A different window size is a different image. Pin the viewport and device scale factor in your config.

Visual tests that fail for these reasons teach a team to accept diffs without looking, and then the suite catches nothing. Treat them like any other flaky tests: find the cause and fix it instead of retrying until the build is green.

What are the best visual regression testing tools?

The best tool is usually the one that runs where your tests already run. The main options:

• Playwright: toHaveScreenshot() is built into the test runner. Pixel diffing with pixelmatch, baselines stored next to your tests, and no extra service.
• Vitest (browser mode): toMatchScreenshot() for component tests, with pixelmatch as its built-in comparator.
• Cypress: cy.screenshot() captures images but doesn't compare them. Cypress's docs point to plugins and integrations for the comparison.
• BackstopJS: an open-source (MIT) tool that compares screenshots of a web app over time, with Playwright or Puppeteer scripts for interactions and Docker rendering for consistent results.
• Percy (BrowserStack): a hosted visual testing and review platform, with DOM snapshotting and a build review and approval step.
• Applitools Eyes: a hosted platform that compares screenshots with what it calls Visual AI, and updates baselines in one click.
• Chromatic: from the team behind Storybook. It runs visual tests on every push and integrates with Storybook, Playwright, Cypress and Vitest.

Open-source tools cost nothing to run, and your team owns baseline storage, review and upkeep. Hosted platforms store baselines for you and add a place to review and approve changes.

How do you add visual regression tests to CI?

Run them on every pull request, in the same environment that made the baselines, and send every baseline change through review.

1. Make baselines where CI runs. A baseline made on a Mac won't be used on a Linux runner, because the platform is part of the file name. Generate and update baselines in the same container image your pipeline uses.
2. Commit baselines with the code. Playwright keeps them in a -snapshots folder next to each test file, so a UI change and its new baseline land in the same pull request.
3. Run on every pull request, and fail the check on any diff past tolerance.
4. Save the diff images as build artifacts. Playwright writes expected, actual and diff images to test-results/, so a reviewer can see the change without rerunning anything.
5. Accept changes on purpose. Run npx playwright test --update-snapshots for an intended change, and review the new image in the pull request.
6. Upgrade browsers on purpose. Each Playwright version needs specific browser builds, and a new build can shift rendering. Expect some baselines to change in the upgrade's pull request, and review them there.

Keep the visual suite small enough to run on every pull request, with screenshots of the screens that matter rather than full-page captures nobody reviews.

How does QA Wolf handle visual checks and screenshot baselines?

QA Wolf runs visual checks inside the same end-to-end flows that test behavior, with one screenshot assertion across web, mobile and desktop apps, as QA Wolf's docs describe:

• One assertion on every platform. Web and mobile web flows use Playwright's toHaveScreenshot(). Native iOS and Android flows use expect(driver).toHaveScreenshot(), and Windows desktop flows take the same options.
• Every screenshot gets a name and a tolerance. The docs recommend maxDiffPixelRatio: 0.01 on web and maxMisMatchPercentage: 1 on native apps, where it's a 0–100 percentage. An unnamed check is hard to trace to its baseline, and one with no tolerance fails on minor rendering differences.
• The first run sets the baseline. That first screenshot is saved as the expected image, to be reviewed before anyone relies on it. Each later run compares pixel by pixel, and a failure saves a diff image of exactly which pixels changed.
• Diffs are reviewed side by side. The editor's Image Diff panel shows Diff, Expected, Actual and Compare views for each named screenshot.
• An intended change becomes the new baseline. Promote the new screenshot to baseline in the editor, or from a coding agent through the QA Wolf MCP. Future runs compare against the new image.
• Baselines match the device. iOS baselines are named by screen size, so an iPhone 15 baseline won't match an iPhone 15 Plus. On Windows, qawolf:windowSize pins the app window, because a baseline doesn't record window size.
• Screenshots stay off moving content. QA Wolf's docs steer screenshot checks away from timestamps, live data and animations, and toward functional assertions there.

When a screenshot check fails, QA Wolf's agents investigate it like any other failure. Here's the Image Diff panel on two of them, a regression the agent reports as a bug and a redesign it flags as intended for you to promote. Switch between the views, and in Compare, try Overlay, which turns the pixels that differ red, and Flicker, which swaps the two screenshots in place:

Hirevue is a perfect example. Its CTO weighed visual testing tools and an in-house move to Playwright, and found that each left his engineers owning the tests. Hirevue chose QA Wolf and cut its release cycle in half. Start now and ask the agent to add screenshot checks to your critical flows, or talk to sales.