Guiding GUIDEcx to 80% coverage saved $642K/yr

July 2023

Doubling GUIDEcx’s existing E2E test coverage saved more than $642K/year 

  • Support ticket volume is down 19% saving $98,000/year
  • A 75% drop in bug tickets saved $144,000/year in lost engineering time
  • QA Wolf replaced the need to hire 6+ QA engineers, saving GUIDEcx $400,000/year.
We get as much value, or more, from QA Wolf at the cost of one headcount than if we hired several QA engineers or created a new team to do the same amount of work.
—Ben Bolton, Head of QA Operations

Meet GUIDEcx

GUIDEcx is a workflow management platform focused on client onboarding and implementation. The platform supports multiple user roles, task automation, reminders and notifications — all of which means lots of individualized configurations and a huge surface area for bugs.

product engineers
QA Wolf tests

Gaps in test coverage let bugs through, hurting velocity and overwhelming customer support

The engineering team at GUIDEcx had built a suite of Cypress tests, but they were unreliable, frequently broken, and ran outside of the deployment process. Which meant that product engineers would ship new code without the ability to fully test for regressions and would release bugs to Production. 

The frequency and severity of bugs were causing problems across the entire company. On the customer service team, a 4,500-ticket backlog meant that customers were waiting longer and longer for responses. The falling NPS scores were an indicator that customers might churn, creating challenges for the sales team. And the product team was struggling to release new features when half their sprint was consumed by fixes and rework

The QA engineers were primarily focused on keeping the (limited) test suite up and running, but the maintenance burden prevented them from expanding the test suite or building the DevOps infrastructure that would allow product engineers to run the tests themselves. 

GUIDEcx was at a pivotal point in their growth, shifting from start-up to enterprise-level customers and needed a product that would be able to compete. The frequent bugs and rollbacks were hampering their ability to sell larger contracts and threatened the company’s long-term goals. When a single release caused 17 separate P0 and P1 bugs, engineering and product leadership decided to get serious about quality with more attention to automated testing.

When I came in they had a small set of Cypress tests that were flaky and unreliable and we spent some time trying to bring that up, but it can be a daunting task. And the maintenance alone is more than one person can really handle.
—Ben Bolton, Head of QA Operations

Doubled test coverage to 80% in 4 months, including complex (and critical) workflows like email delivery 

The QA Wolf platform expands on Microsoft Playwright’s native capabilities to give teams automated test coverage for workflows that Cypress doesn’t support. For GUIDEcx, email delivery is mission-critical functionality that used to be tested manually and QA Wolf now checks each time a developer merges to Staging or Production. 

QA Wolf also maintains test coverage for features under a feature flag, which allows developers to work faster and avoid conflicts between squads. 

Making the jump to QA Wolf has taken a huge burden off. Knowing that the QA Wolf tests are running on every build, that they’re reliable, that they’re maintained — it’s been very positive. And there have been a couple dozen or more bugs caught by QA Wolf. Some of them have been very critical.
—Ben Bolton, Head of QA Operations

Increased engineering capacity by more than 50%

By expanding their automated test coverage and integrating into their CI/CD, GUIDEcx has seen a 75% reduction in new bugs each week, and the amount of rework that the team has to do each sprint. The additional engineering capacity is equivalent to more than 500 person-days per year. 

Automated testing is now fully embedded into the deployment pipeline

Without the burden of test maintenance to contend with, GUIDEcx’s QA engineers were able to build out the DevOps infrastructure for continuous testing throughout the deployment process. Builds are fully tested as they move to Staging so they never reach production or slow down the engineering team.

Ship faster with fewer bugs
Schedule a demo
We use cookies to
improve your experience.