With Padlet's test coverage up, bugs and distractions are down

September 2022
By offloading end-to-end testing onto QA Wolf, Padlet’s team went from 10% test coverage to 95% in 12 weeks — a goal that would have been unattainable with the no-code tools they were using. Today the app is observably more stable than it’s ever been, and the team is saving hours to days every week not having to triage flaky tests.
Impact

Company performance, speed, and focus

  • Automated test coverage went from less than 10% to over 95%
  • No more noise from flaky tests, only verified bugs 
  • Developers are more productive now that QA is handled for them

Meet Padlet

Defying a one-line description, Padlet is something like a digital notebook meets a website builder. The infinitely customizable boards—padlets—are especially popular with educators and students for creating, organizing, and presenting basically any kind of content that’s on the internet.

50M
monthly active users
20
developers
250
test cases
Problem

Testing an open-ended product for a global user base 

Padlet’s easy UI is deceptively complicated to test in the traditional sense where clearly defined workflows allow for neat and orderly test scripts. On Padlet, users create and manage highly customizable and infinitely extensible containers for just about any type of content imaginable. There’s also autosaving, dynamic layouts, drag-and-drop, permissioning, real-time synchronization, and it’s used in 42 different languages on a global range of device types.

So while it’s delightfully simple for humans to use, the no-code tools being used for automated testing weren’t up to the task. Creating tests took a lot of time, even for technical members of the team; and the flaky tests and false positives required constant attention. While the existence of any tests — no matter how few or unreliable — gave the team a false sense of security that let bugs get to production. 

“What we were paying for [our no-code tool] doesn’t capture the true cost when you look at the time and focus it took away from people on the team. And because we had such low coverage, it might have done more harm than good because it would tell us that things were ready to go when they really weren’t.”
—Collin Palmer, Product Manager

With such unreliable test coverage, the team resorted to slow, and error-prone manual testing but bugs were still getting out and fixing them meant more distractions and more delays.

Results

By providing full test coverage QA Wolf was able to take testing completely off Padlet’s plate, giving their team more time to focus and improving overall product quality 

95% end-to-end test coverage
When QA Wolf came on board, Padlet was doing most of their regression testing manually. Within 8 weeks, we had automated more than 70 tests or 80% of their test cases. As Padlet has added test cases, QA Wolf expanded their coverage and today there are 175 tests running several times a day as they deploy.

15% more time every week
Product leadership was spending 5–6 hours a week triaging the no-code tool they were using — which was only covering 5–10% of their application. By managing their test suite and only reporting verified bugs, Padlet’s leadership can focus on innovating and making an impact.

Rapid reduction in bug reports
A surge in bug reports at the start followed by a rapid decrease showed that old bugs Padlet wasn’t even aware of were being fixed, and new bugs were being caught before they ever got out.

Beyond the time and focus that was freed up, the real benefit of QA Wolf is that we can test far more than we were able to test before — and more than we realistically ever would have been able to do — which has led to a much more stable and reliable application.
—Collin Palmer, Product Manager
a wolf crawling inside a computer while another wolf uses it
Ship Faster with Fewer Bugs
We use cookies to
improve your experience.