When automated tests are in working order they provide engineers with rapid feedback on features that are still in development. When they're flaky — when they repeatedly fail without finding a bug — they lose any value to developers, and they're a liability to the team. But maintaining complicated and brittle end-to-end tests pulls developers off their roadmap. When teams off-load testing to QA Wolf, they get all of the signal and none of the noise: only human-verified bug reports get passed on, while flaky tests are triaged and fixed in the background.
Engineering leaders often tell us that developers should write and maintain end-to-end test suites because developers are responsible for code quality. We completely agree when it comes to whitebox tests, because whitebox tests influence how a developer writes and structures their code. But when developers have to own blackbox testing as well, productivity is usually much lower and coverage levels are far below average. In this post we talk about how whitebox testing improves code quality, and why effective teams offload blackbox end-to-end tests to dedicated experts.
Starting QA Wolf, we didn’t expect to pioneer a whole new business category. But our evolution from an open source command line interface to a full-service QA solution actually happened very naturally as we looked deeper and deeper into the problem at the heart of test automation.
When teams have high end-to-end test coverage they can deliver more value to customers, capture more of the market, and solve problems more efficiently. The value of E2E test coverage isn’t just in spotting regressions—it’s in the safety, security, and confidence to make the big moves that drive successful companies forward.
What it means, why people like it, when it happens, who does it, how to do it, and where things tend to go sideways
Since 2009, the Test Pyramid has given engineering teams an excuse to under-invest in end-to-end coverage. The feeling was that only a few critical areas were worth the time, effort, or expense. But new services and technologies have lowered those barriers, and the teams that take advantage of them ship faster, provide a better experience, and ultimately are more competitive.
The big idea in “shifting left” is moving QA earlier in your development process to find bugs before they impact the roadmap or the customer experience. Many companies take that to mean developers should write E2E tests—but we can tell you that shifting QA onto your developers ends up costing more in the end.
If you don't have automated tests yet, getting started can feel daunting. In this guide we explore three common pitfalls companies face, and provide solutions for how to avoid them.
