What Automated Accessibility Testing Means

An automated accessibility test runs a defined rule against a test subject without asking a person to decide the outcome during that run. Depending on the tool, the test subject may be source code, the Document Object Model, the accessibility tree, computed styles, a rendered page, a component or a collection of pages.

Automation appears in several forms. Browser extensions inspect the current page. Crawlers scan groups of URLs. Linters evaluate source code before it reaches a browser. Component tests run during development. Continuous integration checks can block a build when selected rules fail. Monitoring platforms repeat scans and track trends over time.

Some tools also guide a person through tests that cannot be completed automatically. These are better described as guided or semi-automated tests. The software gathers evidence or presents a question, but a trained person still decides whether the requirement is met. The W3C ACT Rules Format recognises automatic, manual and combined testing modes. [2] [6]

A tool name alone does not define the result. The ruleset, ruleset version, scan configuration, browser, page state and access permissions all affect what is evaluated. Two tools can report different findings on the same page because their rules and assumptions differ.

What Automated Testing Can Find

Automation is most reliable when the required condition is available to the tool and the correct outcome does not depend on subjective meaning. Depending on the technology and ruleset, automated checks can identify issues such as the following.

• A page with no title element or no declared page language.
• Images that have no alt attribute, while leaving the quality and purpose of the alternative for human review.
• Form controls and interactive elements that have no programmatically determinable accessible name.
• Invalid or broken ARIA attributes, references and role combinations that match a documented rule.
• Text contrast failures when the tool can determine the foreground, background, font size and rendered state.
• Missing structural markup, empty headings, empty links and other detectable code patterns.
• Some duplicate, repeated or template-level failures across large numbers of pages.
• The presence or absence of specified media tracks, landmarks, labels and relationships where the rule is objective.

These findings are valuable because one rule can expose a recurring defect across a shared template or component. Automated checks can also run early, before a full user interface is assembled, and repeat after every relevant change. W3C recommends evaluating accessibility early and throughout development because early problems are easier to correct. [1]

Automation also improves traceability when teams record the tool, rule identifier, version and tested build. The ACT Rules Format was designed to make test procedures more transparent and reproducible, although a rule implementation can still produce inaccurate results and needs maintenance as technologies change. [6]

What Automation Cannot Confirm

A machine can often verify that an attribute or relationship exists. It cannot automatically establish that the result communicates the right meaning or supports the intended task. W3C uses alternative text as a clear example: a tool can detect that alt text is missing, but a person must judge whether the supplied alternative is appropriate in context. [3]

Human judgement is normally required to determine whether the following outcomes are achieved.

• Alternative text communicates the purpose of an image, chart, icon or control without adding irrelevant information.
• Headings, labels, instructions and link text describe the content or action clearly.
• Reading order, focus order and visual order remain meaningful across layouts, languages and responsive states.
• Every function can be completed with a keyboard and focus remains visible, predictable and unobscured.
• Custom controls expose the correct name, role, value and state and announce changes at the right time.
• Forms provide enough instruction, identify errors clearly and allow users to recover and complete the task.
• Captions, transcripts and audio descriptions are accurate, complete and synchronised.
• Content reflows correctly when zoom, text size, text spacing, orientation or viewport width changes.
• A multi-step journey such as registration, authentication, booking or payment can be completed independently.
• An apparent technical pass does not create a barrier when used with real browsers and assistive technologies.

Automation can also miss states it never reaches. Authenticated pages, expanded menus, dialogs, validation errors, timeouts, personalised content and third-party payment or identity services may remain outside a crawl unless the scan is configured to operate those paths. A report should identify which states and environments were actually tested.

What Manual Accessibility Testing Includes

Manual testing is a structured evaluation performed by a person. It is not limited to looking at the page, tabbing through a few controls or turning on a screen reader. A professional manual review applies documented procedures to representative pages, components, states and complete processes, then records evidence against the agreed standard. [5] [7]

Expert Review

The evaluator examines structure, content, design and behaviour against the relevant WCAG success criteria. This includes questions that depend on purpose and context, such as whether a heading describes its section, whether an error message explains how to recover and whether an accessible name matches the visible label.

Keyboard Testing

The evaluator completes tasks without a mouse or touch input. Testing follows focus through menus, dialogs, forms, date pickers, data grids and other components. It checks reachability, operation, focus order, visible focus, escape behaviour and whether any part of the interface traps or loses the user.

Assistive Technology Testing

The evaluator uses agreed combinations of browsers, operating systems and assistive technologies such as NVDA, JAWS, VoiceOver or TalkBack. Tests cover navigation, control announcements, dynamic updates, validation, tables and meaningful task completion. The selected combinations should reflect the product audience and accessibility support baseline rather than an undefined promise to test every possible environment.

Responsive and Visual Testing

The interface is reviewed at relevant zoom levels, viewport sizes, text-spacing settings, orientations and colour modes. The evaluator checks whether content becomes clipped, obscured or unavailable and whether controls remain perceivable and operable after the presentation changes.

Content and Language Review

Content is tested for meaningful alternatives, descriptive links, clear instructions, appropriate headings, understandable errors and correct language changes. This work becomes especially important on Arabic and bilingual services because direction, translation and mixed-language content can change the experience independently of the English version.

Manual testing also has limitations. It is slower, usually sample-based and sensitive to evaluator knowledge, fatigue and consistency. A manual review is credible only when its scope, procedure, environments, sample and findings are documented. A quick keyboard check by an untrained reviewer is not equivalent to a professional manual audit.