QA Wolf logo

QA Wolf

/

Star

2.8k

Run Tests on Netlify Deployment

In this tutorial, we'll set up our tests to run on Netlify deployments. This helps ensure our critical workflows don't break when we make changes.

QA Wolf maintains a Netlify Build Plugin that runs your tests on deploy previews or production deployments. If you have a different workflow, please reach out for help.

Create Deployment Trigger

First we'll create a trigger to run our tests on Netlify deployments. QA Wolf uses triggers to determine when to run your tests.

To manage your team's triggers, use the "Triggers" button in the sidebar of the dashboard.

Add trigger from dashboard

Click this button to open a popup with options to create a trigger.

Select the "Deployment" tab on the right and choose Netlify as your deploy service. You can optionally rename your trigger.

Create deploy trigger

Install QA Wolf Build Plugin

The next part of the modal provides one-time set up instructions for integrating QA Wolf with your Netlify project. If you have already completed these steps, you can skip this section.

Netlify set up

First we need to install the QA Wolf Build Plugin. There are two ways to do this: from the Netlify UI or with npm.

A. Install from the Netlify UI

Click this link to install the QA Wolf Build Plugin. You will be redirected to Netlify and asked to choose your site.

Install plugin

You can also find the QA Wolf Build Plugin listed in the Netlify Plugins directory.

B. Install with npm

You can also use npm to install the QA Wolf Build Plugin.

Run the following command in your repository to install the plugin as a dev dependency:

npm install --save-dev netlify-plugin-qawolf

Then update the list of plugins in your netlify.toml file to include the QA Wolf plugin:

[[plugins]]
# runs QA Wolf tests against the deployed URL
package = "netlify-plugin-qawolf"

Set QAWOLF_API_KEY Environment Variable in Netlify

The other one-time step we need to do is to set the QAWOLF_API_KEY environment variable in Netlify. This variable identifies your QA Wolf team in the Netlify Build Plugin.

If you have already done this, you can skip this section. Otherwise, follow these instructions to create an environment variable in Netlify.

Specifically, create an environment variable named QAWOLF_API_KEY with the value provided in the modal.

Netlify set up API key
Netlify set environment variable

Choose Deploy Context

QA Wolf supports running tests on two Netlify deploy contexts: deploy-preview and production.

Production deployments happen when you deploy to your production branch (example: main). Preview deployments correspond to the preview URLs for pull requests.

QA Wolf allows you to run your tests on all deployments (preview and production), just preview deployments, or just production deployments. Choose which deployments you want to test in the popup. In our example, we choose to only test preview deployments.

Netlify deploy context

Optional: Install GitHub App

You can optionally connect your GitHub repository (repo) to QA Wolf. This allows you to see status checks on commits.

Connect GitHub repository

A new tab will open and visit GitHub. You will be asked to choose the repo(s) you want QA Wolf to access.

Install GitHub App

Choose the repo(s) you want to test and click the green "Install" button. After the installation succeeds, the new tab will close.

Confirm the repo you want to test is now selected in the dropdown.

Choose GitHub repo

Choose Tags

You can choose to run all your tests, or only tests with specific tags.

Choose tags

Choose Environment

If you have created any environments, choose the one to use when your tests run.

Choose environment

Press the "Create trigger" button to save your new trigger. 🎉

Assign Deployment Trigger to Test

Update Tests to Use URL from Netlify

Because a Netlify deployment URL isn't known in advance, QA Wolf passes it as an environment variable (process.env.URL) when the deployment succeeds.

We need to update our tests to use this environment variable when specified.

Replace the URL in your tests with process.env.URL. For example, let's say our test currently starts with:

const { context } = await launch();
const page = await context.newPage();
await page.goto("https://default.netlify.app/");

Let's update the page.goto call to go to process.env.URL if it exists, and our default URL otherwise:

const { context } = await launch();
const page = await context.newPage();
// go to process.env.URL if specified, and the default URL otherwise
await page.goto(process.env.URL || "https://default.netlify.app/");

See QA Wolf Results in GitHub

If you installed the QA Wolf GitHub app, you will be able to see QA Wolf status checks in GitHub.

See QA Wolf results in GitHub

QA Wolf will comment on a pull request with the most recent test suite. As your tests run, the comment is updated to report the latest results.

GitHub pull request comment

View Run History

🎉 Congratulations - now your tests will run on Netlify deployments in 100% parallel!

You can see your test runs in the "Run history" page of the dashboard.

Run history