Teams that want to refactor to pay down tech debt frequently get caught in a doom loop. Maintenance tasks get shoved aside when the team needs to catch up on feature work. The more that happens, the more the maintenance accumulates and the more daunting the task of tackling it becomes.
The outcome of tech debt is production escapes, and the work involved with fixing them slows down the team’s ability to release features, let alone fix the root cause of the problem. Eventually, the team spends more time fighting fires than releasing new features.
If this sounds familiar, don’t worry; there’s hope from a surprising source: automated end-to-end regression testing. A set of comprehensive automated end-to-end suites based on a solid coverage plan can assist your developers in conquering their fear of refactoring.
Teams rationalize their avoidance of tech debt by saying things like “the offending code is going to get replaced eventually,” so it doesn’t make sense to refactor, or “the team has too many features to complete” and needs to hire more people before they can invest in improvements.
But, the primary reason teams avoid refactoring is a rational fear of the unknown. Refactoring subjects developers to the butterfly effect, in which one small change can have huge impacts later. So any refactoring effort, whether intended to repair a design flaw, upgrade a deprecated library, or simply clean up kruft, could have unknown consequences.
The key to successful refactoring is developer confidence, they need to know that customers are shielded from the negative repercussions of any ongoing improvements to the code base.
We’ve found that to refactor without fear, teams need a comprehensive automated end-to-end regression test suite, and the starting point for that suite is the coverage plan. Once that’s in place, they can use that plan to build regression test suites that assure them their product is releasable.
The butterfly effect paralyzes developers even when they reasonably understand their application and its dependencies. There is always the potential for a bug no one thought was possible (e.g., solar flares). A good coverage plan alleviates that fear by documenting the existing application’s behavior, making all workflows visible, and revealing coverage gaps.
If you're a developer inheriting code you had no hand in designing or building, you know the chance of getting complete and intelligible documentation is nearly nil.
A good coverage plan consists of test cases everyone on the team can read and understand. Teams should author test cases in a standardized manner, such as by using the Arrange, Act, Assert (AAA) framework. Such test cases serve as documentation for how things are supposed to work and act as a guidepost during refactoring efforts.
Even when developers have well-documented code that they developed and designed, they usually have little insight into their dependencies. It is rare for a software developer to be able to reliably anticipate how a given code change will affect the entire system. Microservices and organization silos compound the fact that modern systems consisting of anything more than a handful of applications are simply too complex for a single developer to understand comprehensively.
A good coverage plan makes the team aware of every possible workflow, assuming the team has agreed on the critical workflows that must be covered. Of course, once the team has identified all the workflows, they should automate at least 80% of them. That way, developers can gain confidence that customers will remain safe when they change the underlying application. We’ll get back to that in a moment.
The butterfly effect justifies the hesitancy to modify the system even when developers reasonably understand their application and its dependencies. It means there is always the potential for something no one on the team thought was possible to cause a bug in the system.
A good coverage plan helps teams identify edge cases and gaps in unit and component integration coverage. Simply thinking about how a user traverses various scenarios helps developers concretize which lines of code are getting exercised and whether or not that code is covered.
Like we said above, a good coverage plan alone can’t quell the fear of refactoring — you also need automation. All those test cases aren’t going to run themselves, and once your application grows past about 20 workflows, manual testers stop being your best option.
A solid automated regression test suite acts as a safety net so developers can make significant changes to the code without fear, much like buildings under renovation are surrounded by scaffolding that prevents debris from injuring passing pedestrians.
Often, a team will put refactoring work on hold because they just don’t know what they’ll uncover when they get into it. Is it a one-sprint project? A month? A year? Having comprehensive end-to-end test suites is like having a map of how everything is connected and can show you how much of a system is impacted by a code change. Simply by making the change and running the test suite in a pre-production environment, you will see which systems will be affected.
Of course, you’ll have to investigate each system and scope out fixes to applications within your blast radius, but you’ll be able to get a complete picture of what needs to be done.
The point of testing is to find issues before they affect live customers. Bugs that pop up in staging or a preview environment don’t matter — that’s what’s supposed to happen. As long as you don’t move that code to production, you can keep breaking things and re-testing until you’re done, and the customer will never know.
A comprehensive regression suite, paired with a pre-production environment, is like a time machine that lets you try out and undo changes indefinitely. And as a bonus, it’ll tell you when you’re done because all the tests will show green.
The conventional wisdom is that developers preparing to refactor their applications should start by increasing their unit test coverage. Of course, that assumes that the application actually works as it was designed — which is frequently not the case; and the system is straightforward enough for developers to be able to easily write unit tests for it— which is almost never the case.
E2E tests are the only way to validate that the entire system works together as intended, especially after making changes in a complex environment. But there’s a catch. Test suites with their own tech debt amplify the doom loop and make things even worse.
Investigation and maintenance are the key to gaining confidence in the caliber of the end-to-end test suites. Developers need to know that their teammates are investigating every failure and taking steps to address the root cause. They need assurance that someone on their team is rooting out false positives and that failures mean one thing: there’s a bug in the application. Teams that thoroughly investigate the root cause of failures and file defects where appropriate and proactively maintain their tests before they break are well on their way to building trust in the suites’ results.
QA Wolf solves all that. We provide the test plans, the tests themselves, the running infrastructure, and the maintenance. We’ll get you to 80% test coverage in less than 4 months and maintain it throughout your refactoring — and as long as after that as you want.
QA Wolf is a hybrid platform & service that helps software teams ship better software faster by taking QA completely off their plate.
© {year} QA Wolf, Inc.
