Introduction
During the months of January and February I created a “Bulk Tester” for Tenon as a proof-of-concept for how one could use Tenon to do testing on a series of known URLs of a website. It has proven pretty useful to us as a research tester and has even demonstrated that Tenon is capable of testing & processing approximately 2 URLs per second on a single server set up. Recently, it was even used by my daughter for her science project on web accessibility. The other day I decided to research the accessibility of the homepages for the Fortune 500.
Disclaimer
This post seeks to describe the only the findings based on automated testing of homepages for Fortune 500 companies. This is not a good measure of the accessibility of these companies sites. There’s only so much automated testing can do and testing multiple pages of each site may paint an entirely different picture. Nevertheless, it was fun and informative.
Summary
The home pages tested have a significantly lower number of issues per page than most pages that Tenon has tested. The average score, globally, for issues per page is 56 whereas the Fortune 500 homepage had an average of 19. In other words, the average page is 3x worse than the Fortune 500.
| Average errors per page | 18 |
| Average warnings per page | 1 |
| Average issues per page | 19 |
| Average issue certainty | 97% |
| Average issue priority | 77% |
Issues by WCAG Level
Each of Tenon’s tests are mapped to at least one WCAG Success Criteria, indicated in the ‘standards’ array of the issue report. 81% of issues discovered on Fortune 500 home pages were Level A issues. This indicates to me that there are likely to be a lot of “quick wins” to be had in terms of improving the accessibility of their sites.
| Level A | 11355 |
| Level AA | 433 |
| Level AAA | 2138 |
Issues by Certainty
Each of Tenon’s test has a “Certainty” score. This is an indication of how certain we are that the issue discovered is really an issue. This helps us to add context to the issue and is one of many efforts at transparency. Each test is given a Certainty score of 0%, 20%, 40%, 60%, 80%, 100%. You can assume that we are 4x as certain of a 100% issue than a 20% issue. 9/10 of the issues logged for the Fortune 500 homepages have 100% certainty.
| Certainty | Num. Instances |
|---|---|
| 20% Certainty | 6 |
| 40% Certainty | 23 |
| 60% Certainty | 677 |
| 80% Certainty | 312 |
| 100% Certainty | 10770 |
Issues by Priority
As part of its response Tenon calculates a normalized “priority” based upon a proprietary algorithm that helps to answer the question “What must get fixed first?” We derived this algorithm on a multi-round Delphi exercise with leaders in the accessibility field and it has been proven effective at Fortune 500 companies with mature accessibility efforts. The below data indicates that 35% of the issues found have 100% priority. This further bolsters the notion that a large number of the issues on these home pages would be easy to fix and that doing so would have a highly positive impact on the accessible user experience.
| Priority | Num. Instances |
|---|---|
| 23% Priority | 22 |
| 33% Priority | 396 |
| 36% Priority | 20 |
| 41% Priority | 4 |
| 44% Priority | 40 |
| 49% Priority | 33 |
| 56% Priority | 3822 |
| 65% Priority | 243 |
| 67% Priority | 2 |
| 70% Priority | 183 |
| 72% Priority | 113 |
| 73% Priority | 216 |
| 76% Priority | 643 |
| 81% Priority | 1588 |
| 84% Priority | 203 |
| 89% Priority | 133 |
| 100% Priority | 4127 |
Conclusion
The data in this post isn’t complete enough to provide many concrete conclusions, but the data does tend to back up my experience as an accessibility consultant – namely that there are a lot of missed opportunities for better accessibility on most websites. I was surprised to see that the average issues per page was so much lower but my gut says that a more thorough assessment of each site would be needed to get a better idea.