Too many tests, not enough paws.
Meow Wolf’s iOS and Android apps do a lot. From critical revenue-generating tasks like selling tickets to delivering content like games, videos, and comics. All of which need to work for the 2.4M guests that flow through Meow Wolf’s exhibits each year. In all, the app contains some 1,300 regression tests that the team was doing manually.
Regression cycles would take weeks, especially when repeated across multiple devices.
“It took so long to finish testing in time for a release. To do a complete regression would take a week. It pulled me away from supporting other teams. It pulled engineers away from building features, and it even pulled product and project managers into testing because there was so much to test before each release. It wasn't sustainable. ”
—Deborah Garnel, Senior QA Engineer
By far the most complex workflows that need testing involve the “psychic sensor.”
Psychic Sensors use the Apple iBeacon standard, a Bluetooth Low Energy specification that works as a proximity sensor. If a Meow Wolf guest is within a GPS geo-fence and proximity to an iBeacon, the Meow Wolf app displays additional content on the customer’s phone.
The combination of GPS and Bluetooth made manual testing difficult—QA engineers and developers would need to be on-site at one of the locations to physically validate that the combination of signals (GPS and Bluetooth) was activating the correct content on the device.
“I don’t live near a Meow Wolf facility, and neither do many of our mobile engineers. If we wanted to test an iBeacon, we could hard-code GPS coordinates into our Test Flight app but if I wanted to test a disconnect then I’d have to leave my house to verify the geo-fence was working. It could take 2–4 hours to validate that these flows were working.”
—Deborah Garnel, Senior QA Engineer
Manual testing took about 10% of each person’s time every sprint, pulling engineers, product managers, and others away from their core work. If teammates weren’t brought in to help, a full regression would have taken Deborah an entire week to complete—delaying releases and leaving less time for new development.
“We needed to be able to have tests running against the app as often as possible. We needed automation yesterday.”
—Deborah Garnel, Senior QA Engineer
.svg){kind=logo}
