Work

Assessing Evaluation Tools through Accessibility Standards in Web Development : With CI/CD Pipelines

Year: 2025
Type: dissertation
Abstract: To help ensure that websites across the internet comply with web accessibility, there are defined standards known as the Web Content Accessibility Guidelines (WCAG). Accessibility evaluation tools exist to assess websites and raise awareness for violations of accessibility guidelines. This project aids further efforts to provide explainability for WCAG 2.2 criteria, by implementing pass/fail examples of specific criteria in different contexts onto a test website, extended from Moser and Högback’s study “Incorporating Web Accessibility into a CI/CD Pipeline”. Additionally, their study assesses the tools: Axe-core, Accessibility Checker, and HTML_CodeSniffer through a CI Pipeline, and we contribute by incorporating the following tools: WebAim’ s WAVE, Pa11y, and Google’ s Lighthouse. For the assessment of the tools’ performance, our metrics are considering their ability to identify violations, based on WCAG criteria, error description, and error location. From our analysis of the accessibility evaluation tools, we were able to determine that overall Pa11y is the current best performing tool out of the ones that were tested, as it achieved a 25% average for identifying violations on the test website, but it also flagged some false positives. This shows that the results capable of being produced by the evaluation tools still remain quite limited. Regardless of their completeness, the tools still prove to be effective considering there are violations that they can successfully identify with descriptive and traceable information. This provides developers with a better understanding of accessibility, but there remains a need for further development on the evaluation tools themselves.
Authors: Kelasidou, Sofia, Pettersson, Nils
Language: English
Open Access status: green
OpenAlex ID: W7055000028
Cites: 3
Cited by:
Related to: 10