Key takeaways
Run automated tests at every stage of your CI/CD pipeline, and let each result decide whether the change moves on.
A commit, pull request, merge and deploy each get a pass or fail before the next stage starts.
Run each kind of test at the earliest stage that can run it.
Unit tests need only the code, so they run on every commit, while end-to-end tests need a deployed build and smoke tests need the environment that just deployed.
Run the full end-to-end suite in parallel against every pull request's preview deploy before it merges.
With every test running at once, the suite takes about as long as its longest test, so the change gets checked against everything a user can do without slowing the merge.
Retry failures automatically and give every test its own data, so the pipeline stays fast and trustworthy enough that nobody skips it.
A pipeline that fails at random teaches people to re-run it until it's green and stop believing red checks.
Declare an order only for flows that must wait for another, and run everything else at once.
A sanity check or a multi-user flow may need to go first, and a rules file keeps that order without turning the suite into one long script.
Continuous testing is running automated tests at every stage of a CI/CD pipeline, so each change gets a pass or fail before it moves to the next stage. A commit runs unit tests, a pull request runs end-to-end tests against its own preview deploy, and every deploy runs smoke tests before users see it.
Most of the work is keeping the pipeline fast and reliable enough that people trust it. If the tests take an hour, people merge without waiting for them, and if the tests fail at random, people re-run them until they pass and stop believing red checks.
What is continuous testing?
Continuous testing is the practice of running automated tests from the delivery pipeline itself, every time code changes. The results decide whether the change moves on, and a failing check stops a merge or a release until someone fixes the problem or decides it isn't blocking.
Continuous integration, continuous delivery and continuous deployment each lean on it at a different point.
- Continuous integration merges every developer's work into the main branch often, and runs automated tests on each change before it merges.
- Continuous delivery deploys every change that passes to a production-like environment and runs end-to-end tests there, so the build is always ready to release. A person still decides when it ships.
- Continuous deployment releases every change that passes to production automatically. Reaching continuous deployment depends on tests the team trusts enough to ship on.
In DevOps, continuous testing is how QA becomes part of every step of delivery. Automated testing supports DevOps by giving each step a fast pass or fail, so developers find problems while the change is still fresh.
How is continuous testing different from automated testing?
Continuous testing is automated testing that the pipeline runs on every change and acts on, by blocking a merge or holding a release. A team can have a large automated suite and still not test continuously, if someone runs it by hand before each release.
Which tests run at each stage of a CI/CD pipeline?
Each kind of test runs at the earliest stage that can run it, so tests that need only the code run first and tests that need the whole app running wait for a deployed build.
| Stage | What runs | What a failure does |
|---|---|---|
| Before a commit | Linters, type checks and unit tests for the code that changed, on the developer's machine. | Stops the commit. |
| Pull request | All unit and integration tests in CI, then the full end-to-end suite in parallel against a preview deploy. | Blocks the merge. |
| After merge | The full regression suite again, against staging or a release candidate, on every browser and device you support. | Holds the release. |
| Deploy | Smoke tests of the critical flows against the environment that just deployed. | Stops the rollout or triggers a rollback. |
| Production | Scheduled checks of critical flows, plus monitoring and alerts. | Alerts whoever is on call. |
The stage follows from what each kind of test needs to run.
- Unit tests check one piece of code with no app running. They can run on every commit and every push.
- Integration tests check that modules, services and APIs work together. They need a database or stubbed services, so they usually run in CI on the pull request.
- End-to-end tests check that a user can finish a flow in the real app. They catch bugs that only show up when the pieces run together, such as a checkout that fails while every unit test passes, and they need a deployed build, so they run against a preview deploy or staging.
- The full end-to-end suite runs on every pull request and again on the release candidate. Anything a user can do should have a test, and with every test running in parallel, the whole suite takes about as long as its longest test. Checking a small subset per pull request made sense when suites ran one test after another and people maintained every test by hand. The release candidate gets its own full run, with cross-browser testing, because it's the combination of changes that actually ships.
What is shift-left testing, and how does it relate to continuous testing?
Shift-left testing means testing earlier in development, closer to when the code is written. Continuous testing is how teams put it into practice, with tests running on each commit and pull request so a bug turns up while its author still remembers the change.
Shift-left doesn't replace the later stages. End-to-end tests still need a deployed app, and some problems only show up in production, so teams also "shift right," with scheduled checks and monitoring after release.
How do you keep a CI/CD pipeline fast?
Run tests in parallel, keep flaky tests from blocking the pipeline, and give every test its own data, so a run finishes fast enough that nobody skips it.
Run every test in parallel
Parallel testing runs tests at the same time instead of one after another, and how much time it saves depends on how the work is split. QA Wolf's parallel testing guide uses 200 tests that each take five minutes.
- Sequential: the suite takes as long as all its tests added together, more than 16 hours.
- Sharding: the suite is split into a fixed number of groups, each on its own machine, and each group still runs its tests in a row. On 10 shards, the 200 tests take 100 minutes, and the run gets longer with every test you add.
- Full parallelization: every test gets its own browser or device and they all start at once, so the suite takes as long as its longest test, five minutes in this example.
Full parallelization works only when the tests are written to run in isolation, with no shared state and no dependence on the order they run in, and when the environment under test can handle every test hitting it at once.
Keep flaky tests from blocking the pipeline
A flaky test passes and fails on the same code, so in a pipeline that gates merges, one flake can block a change that did nothing wrong. Handle flaky tests inside the pipeline:
- Retry failures automatically on the same commit and environment, and cap the retries so they don't hide real problems.
- Record which tests needed a retry, so repeat flakes show up as patterns you can fix.
- Take a test that keeps flaking out of the required checks until it's fixed, then fix the cause: timing, shared data, an unstable environment or a slow third-party service.
The more often a suite runs, the more flakes it turns up, so a team moving its end-to-end suite onto every pull request should fix its flakes first.
Give every test its own data
Give every test its own data. Tests that run at the same time and share an account or a record overwrite each other's work, and the failures look random.
- Create each test's account and records through an API at the start of the test, and clean them up at the end.
- Use unique logins per run, and prefix generated data with a run ID or a timestamp.
- Use separate tenants where the app supports them.
- Don't deploy to the test environment while a run is in progress.
- Use ephemeral environments where you can. A fresh preview per pull request starts every run from a clean state.
How do you test pull requests against preview deploys?
Deploy each pull request to its own preview environment, run the full end-to-end suite against its URL in parallel, and report the result as a check on the pull request that has to pass before it merges.
- Create a preview deploy per pull request, with its own URL. Hosts such as Vercel and Render do this, and teams on their own infrastructure script it with tools like Terraform.
- Tell the test system the deploy is live, with its URL, from the pipeline's deploy step.
- Run the full end-to-end suite against the preview, every test at once, so the check comes back in about the time of the longest test.
- Post the result on the pull request as a status check, and make it required in branch protection, so a failing check blocks the merge.
- Tear the preview down when the pull request closes.
Preview deploys catch the bugs unit and integration tests miss while the author still has the change in mind. Each preview needs its own seeded data and working third-party services, and the release candidate still gets a full run, because merged changes can break each other.
QA Wolf's PR testing follows these steps. It runs your flows against each pull request's preview deploy and posts a QA Wolf Test Results check on GitHub, or a commit status on GitLab, that branch protection can require.
How do you run tests that depend on each other?
Keep tests independent wherever you can. Where a flow really does need another to run first, declare that dependency, and let every other test run in parallel.
Some flows can't be independent:
- A quick sanity check that should pass before a long suite starts.
- Multi-user flows, such as a rider requesting a trip and a driver accepting it, where one flow's output is another's input.
- An environment that can't take the whole suite at once.
Writing these as one long script in a fixed order makes actors that should act at the same time run one after another. Fixed waits stand in for real timing, state leaks from one user to the next, and one failure re-runs the whole sequence.
QA Wolf's Run Rules declare the order instead. A rules.yaml file in the flows repository says which flows run after, before, with, or never alongside which others, by flow name or by tag. By default, when an earlier flow fails, the flows that wait on it are marked failed without running, which keeps the failure list on the root cause, and every flow without a rule still runs in parallel.
How does QA Wolf run continuous testing?
QA Wolf's AI agents write, run, investigate, fix and maintain end-to-end tests, started from your own pipeline against every pull request's preview and every deploy you report, with every test running at once. When an agent needs an answer about how your product should behave, it asks your team. On 1,567 evals across web, iOS and Android apps, QA Wolf passed 93%, against 78% for Claude Code and 74% for Codex.
- Runs start from your pipeline. A deployment trigger runs the flows you choose when your pipeline reports a deploy, through GitHub or GitLab deployments, a GitHub Action or QA Wolf's Node.js SDK, or on a schedule.
- Every pull request can get the full suite. PR testing runs your flows against the pull request's preview deploy and posts the result as a check the merge can require.
- Every test runs at once. Each flow runs in its own container, in parallel by default, so there are no shards to balance and a run takes about as long as its longest test. QA Wolf's infrastructure for web app testing, mobile app testing and desktop app testing gives every test a clean browser, device or machine.
- QA Wolf retries a failed flow to rule out a flake. Each flow gets up to three attempts, first with the whole suite, then in batches of five, then on its own, which isolates environment problems.
- Failures are investigated while the suite still runs. QA Wolf's agents start investigating failures before the run finishes, fix tests that broke because the app changed, and report real bugs to you. In QA Wolf's data on 127 recurring suites, the median time from every test having run to every failure having a conclusion fell from 2.3 hours with human investigation to 21 minutes with AI investigation (data through Oct 15, 2026).
- Dependent flows run in order through Run Rules.
- A release can wait for the verdict. QA Wolf's GitHub Action and Node.js SDK hold a release job until the verdict comes back. The verdict passes when QA Wolf finds no open blocking bug after triaging the run, so a failure triaged as non-blocking, or as a test that needs maintenance, doesn't hold the release.
QA Wolf tests its own releases on the same platform. Its engineering team deploys to production every day, and each day's release candidate clears the full suite of about 400 end-to-end flows across web, AI, Android and iOS. A failing test runs two more times under different conditions before it counts as a bug, and blocking bugs stop the release.
GUIDEcx is a perfect example of a team testing this way. QA Wolf's tests run on every one of its builds, so each build is fully tested on its way to staging, before it can reach production.
Start now and ask the agent to build the end-to-end tests your pipeline runs on every pull request, or talk to sales.
Frequently asked questions
What is continuous testing in DevOps?
In DevOps, continuous testing is automated testing built into the CI/CD pipeline, so every change is tested at each stage on its way to production. It tells developers a change broke something while the pipeline is still running, and lets the team release often without a separate test phase at the end.
How do you implement continuous testing?
Make unit and integration tests required checks on every pull request. Then add end-to-end tests against a deployed build, starting with your critical user flows and getting them stable before you expand to everything a user can do, and run the whole suite in parallel on every pull request. QA Wolf's AI agents write and maintain those end-to-end tests and run them from your pipeline.
What is the difference between continuous integration, continuous delivery and continuous deployment?
Continuous integration merges code into the main branch often and tests each change automatically. Continuous delivery also deploys each passing change to a production-like environment and tests it there, so it's always ready to release, but a person decides when. Continuous deployment releases every change that passes the tests to production automatically.
What are the stages of a CI/CD pipeline?
A typical CI/CD pipeline builds the code and runs unit and integration tests, deploys the build to a test environment, runs end-to-end tests against it, and releases it to production. Each stage runs only if the one before it passed.
How do you add end-to-end tests to a CI/CD pipeline?
Run them against a deployed build, such as a preview deploy for each pull request or staging after a merge. Have the deploy step tell the test system the build is live, run the full suite in parallel, and make the result a required check on the pull request or a gate before release. QA Wolf's PR testing and GitHub Action do this with tests its AI agents write and maintain.
What tools are used for continuous testing?
A CI system such as GitHub Actions, GitLab CI/CD, Jenkins or CircleCI starts the runs. Jest, pytest or JUnit run unit tests, and Playwright, Selenium or Appium run end-to-end tests. QA Wolf's AI agents write and maintain end-to-end tests in open-source Playwright and Appium, standard code you own and can take anywhere, and run every test in parallel from your pipeline.