Long-time readers of my personal blog and my followers on Twitter are already well aware of my disdain for “overlay” products. These are products that claim to automatically fix a website’s accessibility issues. All you have to do, they claim, is insert their JavaScript snippet on your website and your accessibility problems will be repaired. These incredible claims are without basis in fact and I’ve outlined the technical challenges once before. So, recently I decided to go through an exercise of determining the extent to which we could accurately and reliably create an automatic repair for the things Tenon can test for.
The first step in creating an overlay product is detecting problems that need to be repaired. Since Tenon has over 200 accessibility tests, which provide coverage for over 3000 distinct failure conditions, a good first step is to go through our tests and see what can be fixed out of all of the things that we find. It wasn’t long before we found out how brittle and unreliable an automated fix could become. In fact, it was the very first test that demonstrated this!
The first test Tenon runs on a page is test ID 03, which looks to see if a page has its language declared. The test is simply this: return $('html').not('html[lang]');. As you can see, it is a very simple test and it returns a result that is clearly pass/ fail: Either the html element has a lang attribute or not. This is the kind of thing that automated testing can find with no problems. This is also an issue that is easy to fix. All you need to do is add the lang attribute to the opening html element and set that attribute’s value to the relevant ISO 639-1 language code for the page. But that’s where things start getting tricky. To set the language, you need to know what the language is. Doing so in an overlay is certainly possible but requires the use of an external API or the kind of privileged access to the browser object that a 3rd party script like an overlay would not have. This, by the way, just to determine what ISO language code should be used on the page to fix only the first of Tenon’s tests.
It only gets worse from there. The more complicated the test, the more difficult it would be to accurately and reliably fix with an overlay. In addition, making the automatic fixes may involve making changes that harm the performance or appearance of the customer’s site. Then, of course, there’s the problem that if you cannot accurately test for it, you cannot accurately fix it, either.
Impossible, improbable, unreliable, and harmful
Similar to the above, there are entire swaths of accessibility problems that can be detected but cannot be fixed, not fixed reliably, or not fixable without a serious performance hit.
- Images without text alternatives: detectable, not reliably fixable & not fixable without performance hit
- Frames & i-frames without titles/ with duplicate titles: detectable, not reliably fixable
- Tables used for layout: detectable, not reliably fixable
- Implicit headings: detectable, not reliably fixable
- Invalid list structures: detectable, not reliably fixable
- Form elements without labels: detectable, not reliably fixable
- Empty/ missing page title: detectable, not reliably fixable
- Duplicate ID values: detectable, not reliably fixable
- Keyboard accessibility: detectable, not reliably fixable
- Implicit headings: detectable, not fixable without risking altering design
- Improper heading structures: detectable, not fixable without risking altering design
- Page width causes horizontal scroll: detectable, not fixable without risking altering design
- Blank heading element: detectable, not fixable without risking altering design
- Insufficient color contrast: detectable, not fixable without risking altering design
- Form field that occurs after the submit button: detectable, not fixable without risking altering design
- Bold text in table cell, mimicking a table header: detectable, not fixable without risking altering design
- Missing
langattribute on page/ parts of page: detectable, not fixable without performance hit
One approach that overlay vendors are purported to use is to cache their “fixed” version of the page as an attempt to circumvent the performance impact. This approach falls on its face, however, any time a site uses a UI framework based on React, Vue, or Angular. Component-based user interface widgets manage their own states and properties in a way that often doesn’t allow for the sort of post-rendering modifications via JavaScript that would happen with an overlay product. This takes us to the next problem.
Custom Overlays Are Brittle And Expensive
Custom Overlays – that is, those overlays that are tailor made for a specific site – are compelling approach that at least show promise when it comes to more reliably fixing more problems than the generic overlays can. Custom Overlays are, as the name implies, custom to the site to be repaired. The vendors of such solutions typically have their own JavaScript libraries that they leverage to make the repairs. The repairs provided are highly specific and often aimed at handling issues that were identified during an audit. Ostensibly this is a rapid way of handling accessibility issues without potentially long and disruptive refactoring. Consumers of such services should exercise caution, as these approaches can be brittle. All it takes is for you to change something on your site and it could break the custom overlay. It is also prudent to consider the costs involved and weigh that cost against the cost of actually fixing the problems in your code without relying on the overlay. Custom overlays are always only a temporary fix.
“Add-ons” don’t add anything
Finally, there’s the matter of the “enhancements” that overlay vendors claim will make things better for users. Some of the features we’ve seen include:
- Adjusting font size
- Changing font family
- Page zoom
- Word spacing
- Letter spacing
- Line height
- Hide images
- Show image description
- Contrast adjustment
Customers who believe that these features are an improvement for users are unfortunately ignorant of accessibility: If a user needs a larger font size, they will need it on all websites and, in fact, all applications on their computer. As a consequence, all operating systems and all browsers already offer this functionality built-in. The same goes for features like page zoom and higher contrast changes. Hiding images or black-and-white modes are arguably a decrease in accessibility or, at least not useful. Other features, such as showing image descriptions, changing of fonts, letter spacing, word spacing, or line-height are so easy to do in the site’s own CSS files that it doesn’t justify the cost of the overlay. Frankly, it is hard to understand the reason that overlay vendors include these features beyond having eye candy to show during sales presentations.
The myth of artificial intelligence
Some overlay vendors have even gone so far as to claim that their products use artificial intelligence. AI is the hottest new thing in the startup world. Overlay vendors who make such claims need to be treated with significant suspicion. Keep in mind that these products purport to fix accessibility problems. It seems that these companies are claiming that their use of AI is, in some way, informing the repairs they perform. This is certainly a compelling concept, but it is unlikely to have any basis in fact.
- Doing so requires astonishingly large amounts of data.
- Doing so requires that data to be accurate
- Doing so requires staff, knowledgeable in accessibility, to train their algorithms not only on what problems there are but how to fix it.
Despite their claims, none of the above things appear to be happening. Furthermore, it isn’t even necessary. Here at Tenon, we’ve logged over 33,000,000 errors on well over 1,000,000 distinct URLs. 4,516,899 of those errors are completely unique. In other words, without artificial intelligence we’re already able to find millions of problems.
If anything, we could spend the time to train machine learning algorithms with this data by combining it with actual user data to determine the likelihood that the issues Tenon finds would impact end users. However, that itself would require a massive amount of data from actual website users the likes of which no overlay vendor has access to.
Conclusion: Just fix it!
Ultimately there’s only one logical conclusion when it comes to the discussion of overlays: Overlays don’t work. If they did, we’d make one! Purchasing an overlay product only delays the inevitable – that you will eventually need to fix your site’s accessibility problems. The use of such products does not mitigate legal risk and does not improve user experience. We’d rather focus on helping customers with reliable, sustainable, and permanent effective fixes for their accessibility challenges.