Playwright testing: the complete guide

Playwright testing is automated testing of web apps with Playwright, Microsoft's free, open-source framework that drives Chromium, Firefox and WebKit through one API. You can write tests in TypeScript, JavaScript, Python, Java or .NET. In JavaScript and TypeScript, Playwright's own runner, Playwright Test, handles waiting, parallel runs, retries and debugging traces for you.

Hirevue looked at migrating to Playwright in-house and chose QA Wolf instead, because every in-house option left its engineers dealing with flaky tests and maintaining a framework. It has since doubled its release pace, from every two weeks to every week. QA Wolf's Playwright automation tool writes, runs and maintains real Playwright tests for you. Book a demo to see it on your own app.

What is Playwright?

Playwright is a free, open-source framework from Microsoft for automating and testing web browsers. It drives Chromium, Firefox and WebKit through a single API, on Windows, Linux and macOS, with or without a visible browser window. Microsoft published the first versions in early 2020 and released Playwright 1.0 in May 2020, according to npm's release history, and the project is licensed under Apache 2.0.

Playwright comes in several pieces, all from the same project:

• Playwright Test, a test runner for end-to-end tests in JavaScript and TypeScript.
• The Playwright library, for browser scripts with no test runner, such as screenshots, PDFs and data collection.
• Playwright MCP and the Playwright CLI, which let AI assistants and coding agents drive a browser.
• A VS Code extension for running, debugging and recording tests from the editor.
• Versions for Python, Java and .NET, which share the same browser automation underneath.

What is Playwright testing used for?

Playwright testing is mostly used for end-to-end testing: tests that open your web app in a real browser and click, type and navigate the way a user would, to check whole flows such as sign-up, checkout or account settings. One test can drive several isolated browser contexts, so flows that open a new tab or need two signed-in users at once fit too.

The same suite can take on other kinds of testing:

• Cross-browser testing in Chromium, Firefox and WebKit, the engine behind Safari, from one set of tests.
• Playwright visual regression testing, where toHaveScreenshot() compares a page with a saved baseline image and fails on unexpected differences.
• Playwright accessibility testing, where the @axe-core/playwright package runs the axe engine inside your tests to catch problems such as low color contrast and form fields without labels. Playwright's docs note that automated checks find only some accessibility problems.
• API testing, through Playwright's built-in request context, which sends HTTP requests to test endpoints directly or to set up data before a browser test.
• Mobile web testing, by emulating phones and tablets (screen size, user agent and touch) in Chromium and WebKit.
• Playwright load testing, which runs real browser journeys as many concurrent user sessions to see how the app holds up under traffic.

Playwright can't test native iOS or Android apps. For those, teams use one of the mobile testing frameworks built for them, such as Appium, Espresso or XCUITest.

What is Playwright automation?

Playwright automation means using Playwright to control a browser from code. In QA, it usually means Playwright test automation: a suite of automated Playwright tests that runs on every change.

Outside testing, the same library automates any browser task, such as taking screenshots, generating PDFs, filling in forms or collecting data from pages. It also gives AI agents a browser. Playwright MCP connects AI assistants such as Claude Desktop, Cursor and VS Code to a browser through the Model Context Protocol, reading each page as an accessibility tree instead of a screenshot, and the Playwright CLI does the same job for coding agents from the terminal.

AI is changing who writes the tests, too. QA Wolf's AI agents write and repair Playwright tests from plain-language descriptions. They're trained on testing work, which puts them ahead of other AI testing tools and general-purpose coding agents: across 1,567 evals on web, iOS and Android apps, QA Wolf passed 93%, against 78% for Claude Code and 74% for Codex.

What features does Playwright have for testing?

Playwright Test bundles what a browser test suite needs: cross-browser support, auto-waiting, user-facing locators, a fresh browser context for every test, parallel workers, retries, reports and traces.

Which browsers does Playwright test?

Playwright tests Chromium, Firefox and WebKit, on Windows, Linux and macOS, and downloads its own build of each. It can also run branded Google Chrome and Microsoft Edge through a channel setting. It can't run branded Firefox or Safari, because its Firefox and WebKit builds rely on patches, so WebKit runs are how Playwright covers Safari's engine. For phones and tablets, a built-in device registry emulates screen size, user agent and touch.

Which languages does Playwright support?

Playwright supports TypeScript and JavaScript, Python, Java and .NET. Every language gets the same browser automation, but the test tooling differs: Playwright Test is for JavaScript and TypeScript, Python uses Playwright's official pytest plugin, Java works with JUnit or TestNG, and .NET ships base classes for MSTest, NUnit and xUnit. Playwright's docs suggest picking the language your team already knows.

How does Playwright's auto-waiting work?

Before every action, Playwright checks that the target element is ready and waits until it is. Before a click, for example, it confirms the locator matches exactly one element that is visible, stable (not animating), able to receive events and enabled. Assertions written with expect, such as toBeVisible() or toHaveText(), retry until they pass or time out. Between them, they replace most of the fixed sleeps that older browser tests relied on.

What are Playwright locators?

Locators are how a Playwright test finds elements on the page, and Playwright recommends finding them the way a user would: by role, text, label or placeholder, with a test ID as a stable fallback. A locator like page.getByRole('button', { name: 'Sign in' }) keeps working when a class name or the page structure changes, which is what breaks long CSS and XPath selectors. Playwright looks the element up again every time a locator is used, so a page that re-renders doesn't leave a test holding a stale element.

What does the Playwright Test runner do?

Playwright Test runs your tests and manages everything around them. Each test gets a fresh browser context, the equivalent of a new browser profile, so tests can't leak cookies or storage into each other. Fixtures such as page and request hand each test what it needs, projects run the same suite against several browsers or devices, and the HTML report lets you filter results by passed, failed, flaky and skipped. Retries are off by default. Turn them on and Playwright labels a test that fails and then passes as flaky.

How does Playwright run tests in parallel?

Playwright Test runs tests in parallel worker processes, each with its own browser. By default it starts as many workers as half your machine's logical CPU cores, runs test files in parallel and runs the tests inside one file in order. Setting fullyParallel: true runs every test in parallel. To go beyond one machine, sharding splits the suite into parts, such as --shard=1/4, that run as separate CI jobs, and npx playwright merge-reports combines their blob reports into one.

How do you debug Playwright tests?

You debug Playwright tests with traces, UI mode and Codegen. A trace records every action with DOM snapshots, network requests and console messages, and Trace Viewer lets you step back and forward through a failed run, including one from CI. Playwright's recommended setup records a trace on the first retry of a failing test, since tracing every test is expensive. UI mode (npx playwright test --ui) adds watch mode and time-travel debugging, and Codegen (npx playwright codegen) records your clicks as test code and picks locators for you.

How do you set up Playwright?

Run npm init playwright@latest in a new or existing project. It asks whether you want TypeScript or JavaScript, where to put your tests, whether to add a GitHub Actions workflow and whether to install browsers, then creates:

• playwright.config.ts, which sets browsers, retries, workers, reporters and tracing.
• tests/example.spec.ts, a starter test.
• .github/workflows/playwright.yml, if you chose the GitHub Actions workflow.

You need a current Node.js release; Playwright's system requirements list the versions it supports. Run the example with npx playwright test: by default it runs headless in Chromium, Firefox and WebKit, and npx playwright show-report opens the HTML report. In Python, install Playwright's pytest plugin with pip install pytest-playwright, then download the browsers with playwright install.

How do you write your first Playwright test?

A Playwright test is an async function that receives a page, acts on it through locators and checks the result with expect. This one runs against Playwright's public TodoMVC demo app, so you can drop it into the tests folder of the project you just created and run it without an app of your own:

import { test, expect } from '@playwright/test';

test('a new todo can be completed', async ({ page }) => {
  // Arrange: open the app
  await page.goto('https://demo.playwright.dev/todomvc/');

  // Act: add a todo, then tick it off
  const newTodo = page.getByPlaceholder('What needs to be done?');
  await newTodo.fill('Write a Playwright test');
  await newTodo.press('Enter');
  const todo = page.getByTestId('todo-item');
  await todo.getByRole('checkbox').check();

  // Assert: it's marked done and the counter updated
  await expect(todo).toHaveText('Write a Playwright test');
  await expect(todo).toHaveClass(/completed/);
  await expect(page.getByTestId('todo-count')).toHaveText('0 items left');
});

What each part does:

• test and page: test declares the test, and the page fixture gives it a fresh page in its own browser context.
• Locators: getByPlaceholder, getByTestId and getByRole find elements by what a user sees or by a stable test ID.
• No waits: fill, press and check wait for their element to be ready, and each expect retries until it passes or times out.
• Arrange-Act-Assert: the comments split the test into setup, action and checks, the structure QA Wolf uses to outline every test before it's automated.

Then run it:

npx playwright test

Add --headed to watch the browser, or --ui to step through the test in UI mode.

How do you run Playwright tests in CI?

Install your dependencies and Playwright's browsers on the CI runner, then run npx playwright test on every push and pull request. Playwright tests run this way on any CI provider.

If you let npm init playwright@latest add a GitHub Actions workflow, it already does this. The generated workflow checks out your code, installs Node.js and your dependencies, installs the browsers with npx playwright install --with-deps, runs the tests and uploads the HTML report as a build artifact.

The config it generates also sets a few things up for CI:

• Retries: two on CI and none locally, so a test that fails once and then passes is reported as flaky.
• Workers: one on CI, which runs your tests one at a time until you raise it.
• forbidOnly: fails the build if someone commits a test.only.
• Traces: recorded on the first retry, so every test that fails or flakes in CI comes with a trace to open in Trace Viewer.

To make CI faster, split the suite across several jobs with --shard and merge their reports into one.

Where do teams struggle with Playwright testing at scale?

At scale, the hard work is running the suite fast enough for every pull request, keeping it stable as the app changes, and deciding what each failure means. All three grow with the number of tests and how often you ship.

Why do Playwright tests get flaky?

A flaky test passes and fails on the same code. Auto-waiting removes most timing problems inside the page, so Playwright flakes usually come from outside it:

• Shared state: tests that reuse the same account or data collide when they run in parallel, or depend on the order they run in.
• Overloaded environments: a test environment or CI runner that slows down under parallel load makes tests time out.
• Third-party services: payment sandboxes, email providers and other services that respond slowly or differently from one run to the next.
• Unstable assertions: checks on values that change on their own, such as timestamps, animations or the order of results.

Test execution is processor-bound, and QA Wolf's parallel infrastructure runs each test in its own container because resource contention is a major cause of flaky tests.

Retries hide flakes more than they fix them. By default, a test that fails and then passes on retry is marked flaky without failing the build (the --fail-on-flaky-tests flag changes that), so flakes pile up until nobody trusts a red run.

Why does Playwright test maintenance keep growing?

Every change to your UI can break a locator, a step or an assertion, so maintenance grows with the size of the suite and the pace of your releases. QA Wolf benchmarks show that half of end-to-end tests need to be changed or removed within six weeks without ongoing maintenance. Before AI agents took on the work, QA Wolf measured how long each stage took its QA engineers across 3.5 million tests in more than 100 apps. Investigating a failure took about 17 minutes and fixing a broken test about 45, and that work came back every time a test failed. AI agents do it faster now, but investigation and maintenance still take more of the work than writing the tests.

Role-based locators, test IDs and small, independent tests slow the growth. Someone still has to read each failure and decide whether it's a bug, a broken test or a flake.

What infrastructure do parallel Playwright runs need?

Parallel Playwright runs need more than one machine once the suite outgrows a single CI runner. Playwright's workers share one machine's CPU, and the config npm init playwright generates runs a single worker on CI. Going wider means sharding, and sharding brings its own work:

• Each shard is its own CI job that installs browsers, runs its part of the suite and uploads a report to merge with the others.
• Shards are only as even as your files: without fullyParallel, Playwright splits the suite by file, so one large file makes one slow shard.
• Your test environment has to take the load: every worker hits the same app, database and third-party sandboxes at once.
• Tests have to run in isolation: each one needs its own data and accounts, or parallel runs collide.

With a fixed number of shards, run time still grows with the suite, and the slowest shard sets the pace. Parallel testing is fastest when every test runs at once in an environment of its own.

How does QA Wolf run and maintain Playwright tests?

QA Wolf's managed service writes, runs and maintains your Playwright tests and investigates every failure, so your team gets end-to-end coverage without owning the framework, the infrastructure or the triage.

• Real Playwright code your team owns. QA Wolf's AI agents write tests as standard Playwright code for web apps, and Appium code for iOS, Android and Windows desktop apps. Your team can read, change, version and export them. QA Wolf runs vanilla open-source Playwright, so existing Playwright tests can come with you.
• A test plan before any code. Onboarding starts with a test plan that outlines each flow in Arrange-Act-Assert steps for your approval. QA Wolf then automates your highest-priority and most complex flows first.
• Every test in its own container. QA Wolf runs each test in its own pre-booted container on Kubernetes, so a full run finishes in the time of its longest test, whether the suite has 200 tests or 20,000. Running every test in parallel costs nothing extra.
• Runs on every pull request, deploy or schedule. Suites run on the triggers you choose, runs are unlimited, and every run keeps videos, logs and Playwright traces.
• Every failure investigated. QA Wolf retries a failing test up to three times to rule out a temporary environment issue, its AI agents start investigating while the rest of the suite is still running, and reproduce the failure. Real bugs are reported in your Slack or Teams and your Jira or Linear, and broken tests get fixed. When an agent needs a question answered, it asks your team.
• Coverage in your contract, zero flakes. The flows you choose to cover go in your contract, and QA Wolf guarantees zero flakes: only real bugs reach your team. Maintenance is unlimited.

On a fixed number of machines, run time grows with the suite. With every test in its own container, it doesn't:

QA Wolf

0:10:00

Every test at once, each in its own container

Suite done

10 parallel nodes

2:33:00

10 tests at a time, 20 on each node

Suite done

200 tests that take 5 to 10 minutes each, run two ways. QA Wolf runs every test at once, each in its own container, so the suite is done in 10 minutes, the time of its longest test. On 10 parallel nodes, each node runs 20 of the tests one after another, so the suite takes about 2 and a half hours.

Salesloft runs 2,000 automated tests this way, with 300+ running in parallel on every pull request.

How does Playwright compare with Selenium, Cypress and Puppeteer?

All four automate browsers. They differ in how they control the browser, which browsers and languages they cover, and how much testing tooling comes built in.

• Playwright vs Selenium. Selenium is the older project, built on the W3C WebDriver standard, with official bindings for Java, Python, C#, Ruby and JavaScript and support for all major browsers. It automates the browser and leaves the test runner, waiting strategy and reporting to you, where Playwright Test bundles them. Playwright isn't built on Selenium.
• Playwright vs Cypress. Cypress runs inside the browser, next to your app, and its tests are JavaScript or TypeScript only. Playwright controls the browser from outside, which makes flows across several tabs, users or domains easier to test, one of the reasons QA Wolf chose Playwright.
• Playwright vs Puppeteer. Puppeteer is a JavaScript library that controls Chrome or Firefox over the DevTools Protocol or WebDriver BiDi. Playwright's license notice says it contains code derived from Puppeteer. On top of that, Playwright adds WebKit, versions for Python, Java and .NET, and a full test runner.