With 20% more engineering capacity, Xima doesn’t wait on hold

February 2023
Impact

A more efficient team with room to grow

  • 20% more time spent on feature engineering 
  • Avoided hiring 3 additional QA engineers
  • Manual testers took on new roles and responsibilities

Meet Xima 

Xima’s contact center application handles customer communication over phone, email, and chat. When a customer calls customer service, Xima’s software — integrated with physical phone systems — lets agents transfer the call to the best agent, manages call backs, and provides reporting for supervisors.

211
tests
18
developers
95%
user flow coverage
Challenge

Highly configurable software created a huge surface area for testing

Xima’s contact center software integrates with dozens of third-party applications, office phone systems, and physical devices. Because the platform is so configurable, it can help almost any business manage their customer support operations. But the configurability also makes it extremely complex and time consuming to test without the right automation infrastructure in place.

Rapid feedback is critical. Until you have a full system running through end-to-end workflows, it's hard to know for sure that you haven't broken anything. Having QA Wolf automatically fire off a run and then knowing within minutes if there was an issue gives developers a chance to work on it before they lose context.
—Ed St. Louis, VP of Engineering 

Building automated tests pulled resources away from product engineering 

As the platform continued to grow and manual testing became more and more impractical, the decision was made around January 2021 to automate their suite of manual regression tests. To get it done quickly, two full dev teams had to pause new work for six weeks to build 200+ Cypress tests from scratch. 

And that was just the beginning of the investment.

When the developers finally got back to feature development, they had more work to do than ever before. For each feature they built, there was a backlog of automated tests to go with it. Building these tests each sprint while finishing the core product work on schedule was an uphill battle.

We would get near the end of the sprint and realize we still had tickets to cover the new feature with tests. And it would take a couple of days to do that with most hands on deck. I would say a good 10–20% of our time was spent on writing tests. Having QA Wolf build tests as we build features basically adds an extra day to our sprints.
—Ed St. Louis, VP of Engineering 

Running and maintaining tests in-house took as much effort as manual testing 

It turned out that building the tests was actually the easy part. The real challenge was running and maintaining all of them. 

Xima assigned one full-time developer to babysit and maintain the tests as they would start to break. An experienced QA engineer can realistically maintain about 30 automated tests on their own. So it’s not surprising that this engineer struggled to keep up with the team’s release velocity. 

Without the infrastructure to run tests in parallel, the whole suite would take several hours to run; and it would take several more hours to track down the cause of every failure. 

We invested for over a year in having a full-time person just maintaining those tests. And we never got to a point where we could rely on them all the time. It would take our QA engineer hours to run these tests, and then he'd have a whole bunch of flaky tests to sort through. And so it lost its value quickly. And that's really why QA Wolf stood out, because they were selling the service, not the technology.
—Ed St. Louis, VP of Engineering 
Results

Xima finally has the QA process they always wanted 

Converted 200+ tests — 95% of workflows — from Cypress to Playwright in 4 months
Which avoided any lapse in QA and allowed the engineering team to keep working as they always did as QA Wolf got up and running. 

Gave developers 20% more time each week
Developers that were spending 1–2 days each sprint writing automated tests for the next release can now focus on the product — work they actually want to do.

Replaced the need for any manual QA
Many of Xima’s manual testers transitioned to other roles in the company where they can provide new value that the company needed but didn’t have before. 

I would definitely recommend QA Wolf for any software that has multiple modules that have to integrate together or have to work with third-party APIs. Because our experience is going well so far we have planned on QA Wolf helping us grow into the future.
—Ed St. Louis, VP of Engineering
Ship faster with fewer bugs
Schedule a demo
We use cookies to
improve your experience.