Accessibility testing is not as hard to pull off as people might think, and it doesn’t make sense that most teams aren’t doing it. The lion’s share of accessibility issues are due to just a handful of causes. Furthermore, modern frameworks have made accessibility testing easier to automate, and AI-powered tools continue to get better at identifying issues.
According to research by WebAIM in 2023, 96.8% of home pages violate the WCAG 2.1 standard. And that’s down only 1.5% from 2022. What’s clear from these numbers is that teams don’t address accessibility issues until they get that dreaded email from their legal department.
Frankly, there’s little excuse for such a state of affairs, given the benefits to be had from adopting a sensible accessibility testing strategy. By making their site more accessible, teams can increase customer satisfaction, loyalty, and reputation. What’s more, accessibility can improve search engine rankings and increase the number of loyal customers you have. And yes, one could argue that it’s simply the right thing to do because everyone deserves to be able to use the web.
With a bit of education and just a pinch of elbow grease, your team can be well on its way to providing an inclusive experience for all users. So, let’s proceed with the educational portion of our program.
All too often the tech industry focuses on accessibility concerns for the visually impaired, deaf, or hard of hearing while overlooking the following:
We don’t have exact numbers on how many people are impacted by inaccessible websites, but we can make some generalizations. People with disabilities spend $490 billion annually in the US, and that is nothing to sneeze at. 70% of people with disabilities surveyed said they simply leave a website if the content is inaccessible to them. Furthermore, according to Pew Research, 15% of disabled people say they “never” go online, compared to 5% of those without a disability who say the same. Simply put, by making your content accessible, you increase the size of your target audience and, thus, your opportunity to make some money. Not to mention the legal and ethical considerations!
Add that to the fact that the number of ADA lawsuits against websites is rising rapidly. That alone should be enough of a rationale to justify a robust accessibility testing strategy.
The gold standard for accessibility guidelines is the Web Content Accessibility Guidelines (WCAG), which is a W3C standard. The WCAG criteria are testable statements that confirm that a given website content is accessible to a wide range of people with disabilities (PWD). Some versions of WCAG guidelines are used by the regulatory bodies of many countries, so it makes sense for anyone developing web content to conform with the highest of those standards, currently WCAG 2.1 A and AA. By doing so, your site will automatically conform with the Americans with Disabilities Act and Section 508 in the US, Canada’s ACA, and the European Union’s EN 301 549.
There are three levels of conformance with WCAG: A (lowest), AA, and AAA (highest). While government regulations may require only the lowest or medium (AA) levels of conformance, your company may still want to target the highest level to broaden its customer appeal.
The principles for the WCAG standard are perceivability, operability, understandability, and robustness (POUR). What that means is as follows:
Perceivable – The site must be presentable to users in ways they can perceive. For example, users need to be able to change the presentation to suit their needs, such as large print, braille, speech, translations, or more straightforward language.
Operable – The site must not require interaction that a user cannot perform. For example, a site cannot require mouse-only inputs. It also needs keyboard shortcuts for people who can’t move a mouse and must provide people with enough time to complete actions. It cannot cause seizures. It must be navigable so that PWDs understand where they are and know how to get to where they are going.
Understandable – Users must be able to comprehend any information and understand how to use the interface. For example, labels and instructions should provide a clear context for the associated page element (e.g., sign-up or log in buttons). If this seems subjective, that’s because it is. This subject will come up again when we talk about manual auditing.
Robust – As in able to weather changes in technologies and user agents through the strictest interpretation of web standards.
There aren’t any technical barriers to implementing an accessibility strategy, so one might think more organizations would be doing it. There is no shortage of free and affordable tools to help teams start their journey. Because it’s so low effort, accessibility testing is one of those places where individual team members can act first and apologize later if so inclined. Except they won’t have to apologize for increasing revenue.
Other guides on this topic will tell you to examine your analytics, generate personas, and create a bunch of test cases, but it's much more straightforward than all of that. Set your target at the latest standard of WCAG 2.2 A and AA standards at a minimum, and if you can get away with AAA, throw that in as well. If your team has to comply with a regulation that falls outside this standard, let that be a factor in selecting your tools.
To be clear, there is a difference between targeting the WCAG standard and being certified. Certification is an excellent way to protect your organization against claims that it disregarded an accessibility issue. Still, it’s an involved process, and your organization needs to hire a third-party auditor to achieve this. The auditor will identify all problems and re-audit once your team has remediated those issues.
At present, a solely automated strategy will identify about 80% of the issues for which your team would need to be WCAG certified. That’s because determining the effectiveness of content presentation and interaction for people with various learning disabilities requires human judgment.
We’re just going to come out and say this. We use the Deque Axe library to power our accessibility testing, and we think you should, too. Axe is compatible with a bunch of different frameworks and has bindings for many of the major programming languages. And it’s open-source. But it’s not the only way to implement a solid accessibility testing strategy, so read on.
You have a decision to make: You can run a static analyzer on the source code or run it as an end-to-end test as God intended; we clearly do the latter because a rendered page more accurately reflects how users, particularly those with disabilities, perceive and interact with it. We use Playwright, and here’s a great page on implementing this.
Accessibility tools, Axe included, typically allow you to configure based on levels of severity, similar to logging levels. If you are running e2e tests, too, one neat trick is that you can set both the WCAG standard you want to conform with and the severity that causes test failures. That way, you can ratchet down the severity as your developers make your site more accessible. Cool, eh?
Either way, there’s some light lifting to instrument the test so the tool can work. We also recommend that you turn these tests on in your CICD pipeline in the e2e regression testing phase so that you and your team can get constant feedback on how accessible your site is. Another neat thing about Axe is that you have fine-grained control over which rules your test runs. If, for some reason, you don’t like a few, you can turn them off with just a little configuration code.
Unfortunately, as stated above, you can’t rely on just your pipeline to identify all your site’s accessibility issues. If your goal is WCAG certification, the fact is that some of these issues require human discernment to detect. For that, you will need to perform occasional manual accessibility audits. And you’ll need to file some paperwork, Wazowski.
But there’s good news. Manual audits used to be the sole expertise of (expensive) consultants, but now tools can walk your team through the audit process and help you find 100% of the violations by yourself. Our friends at Deque make one of them, as do the also reputable people at AccessibilityWeb, but there are several others to choose from.
Even if your organization isn’t shooting for certification, manual audit tools can still be extremely valuable in identifying the last 20% of problems that automated tools can’t yet catch. Not to be a broken record, but every little bit helps when you are trying to expand your userbase.
We think most teams know what happens when your organization gets sued for violating accessibility regulations, but human beings (being what they are — human) are awfully good at convincing themselves it’s not going to happen to them. And if constantly attempting to scare developers convinced them to increase their website’s accessibility, then there wouldn’t be 40% of PWDs feeling that the internet is unavailable to them. So something’s not working.
Sure, you could try to get your team to go to classes, and there are reputable organizations that offer them. We think a better approach is to sneak up on them by dropping truth bombs in whatever forum is appropriate. Try using the following links to get some conversations started.
At QA Wolf, we can write tests to identify the accessibility issues. We can also help your team understand how to put together an accessibility testing strategy customized to your organization’s needs.
We don’t claim to be experts in shifting culture, but we don’t think that’s necessary to get your team to start considering a robust accessibility testing strategy. We build the world's best test coverage engine and see holes in coverage where teams lack information on their application. We make it easy to get said information to your team so they can start doing something about it.
QA Wolf is a hybrid platform & service that helps software teams ship better software faster by taking QA completely off their plate.
© {year} QA Wolf, Inc.
