The W3C Evaluation Method

WCAG-EM 2.0 gives evaluators and organisations a shared structure. Its five main steps are to define the evaluation scope, explore the target product, select a representative sample set, evaluate the sample and report the findings. The method can be used for internal self-assessment or third-party evaluation. [1] [2]

The method assumes solid knowledge of WCAG, accessible design, assistive technologies, evaluation techniques and the barriers experienced by people with different disabilities. A tool does not supply that expertise. W3C also recommends involving people with disabilities because user evaluation can reveal usability barriers that conformance testing alone may not find. [1] [7]

A critical limitation must be understood. WCAG conformance claims cover all pages and complete processes within the stated scope. Most professional audits use a representative sample because testing every page and state is impractical. A sample can provide strong evidence about the evaluated product, but WCAG-EM 2.0 states that a sample alone generally does not justify a conformance claim for an entire website. The evaluation statement must describe the actual scope and outcome accurately. [1]

Not every useful audit needs to claim full use of WCAG-EM 2.0. A targeted launch review, design-system audit or critical-journey assessment can be appropriate when its purpose and limits are transparent. The problem is not a narrow scope. The problem is presenting a narrow assessment as proof that the whole product conforms.

Why an Automated Scan Is Not a Complete Audit

Automated tools are valuable because they can inspect many pages quickly, identify recurring code-level defects and support ongoing monitoring. Depending on the tool and product, they may detect missing accessible names, invalid ARIA relationships, colour contrast failures, missing language attributes, form association errors and structural warnings.

W3C is equally clear about their limits. Tools cannot check every aspect of accessibility, human judgement is required, and results can be false or misleading. A tool may confirm that an image has an alt attribute but cannot reliably decide whether the wording communicates the image’s purpose. It may find a button name but cannot establish that a complete booking or payment journey makes sense to a screen reader user. [5] [8]

Automated testing should normally be used to do the following.

• Scan a broad set of public and authenticated pages where access is technically possible.
• Identify repeated defects in shared templates and components.
• Provide a baseline for comparison after remediation and future releases.
• Support manual evaluation by locating pages, components and patterns that need closer review.
• Integrate repeatable checks into development and publishing workflows.

A scan result should not be converted into a claim such as 92 percent accessible without explaining the rules, coverage, weighting and limitations. WCAG does not define a universal scoring system, and WCAG-EM warns that aggregated scores can be misleading. A professional report should prioritise the barriers and affected tasks, not hide them behind one number. [1]

Defining the Purpose and Scope

The first audit decision is not which tool to run. It is why the evaluation is being commissioned. A launch approval, procurement review, legal response, redesign baseline, supplier acceptance test and ACR preparation may require different depth, evidence and reporting.

A written scope should identify at least the following.

• The product name, version, domains, subdomains and environments being evaluated.
• Whether the scope covers only a website or also a web application, mobile application, kiosk, documents, media and third-party services.
• The supported languages, including Arabic and English versions, and whether each will be tested independently.
• The user roles and authenticated areas that require test accounts and realistic data.
• The target standard, version and level, such as WCAG 2.2 Level AA, plus any legal, procurement or sector requirements.
• The operating systems, browsers, assistive technologies, viewport sizes and device types included in the accessibility support baseline.
• The required deliverables, such as an executive summary, detailed defect register, remediation workshop, ACR or verification report.
• Any exclusions, dependencies, access restrictions, unavailable features and assumptions that could affect the findings.

WCAG 2.2 Level AA is the generally accepted target for current work, and WCAG-EM 2.0 describes Level AA as the recommended target. A law, authority or contract may still name WCAG 2.0, WCAG 2.1, EN 301 549, Section 508 or another requirement. The report should test and describe the requirement that actually applies rather than silently replacing it with a preferred standard. [1] [3]

Exploring the Product and Selecting a Sample

A professional evaluator explores the product before finalising the sample. The objective is to understand how the product is built and used, not to select a convenient list of URLs. This discovery stage identifies common views, essential functions, content variations, technologies, third-party services and sensitive or high-risk tasks. [1]

A representative website sample commonly includes the following.

• Entry points such as the home page, landing pages, login and account creation.
• Shared components such as navigation, search, header, footer, cookie controls and support tools.
• Each major template and content type, including articles, service pages, tables, forms, media and downloads.
• Interactive components such as menus, accordions, tabs, dialogs, carousels, date pickers and data grids.
• Critical and high-risk tasks such as registration, authentication, booking, payment, uploading documents and requesting support.
• Error states, confirmation messages, empty states, timeouts and other content revealed through interaction.
• Different languages, responsive layouts, user roles and technologies when they change the experience.
• Accessibility help, feedback and statement pages where available.

Complete processes require special treatment. If a selected page is part of a multi-step process, the audit should include the pages and states needed to complete the default path and critical branches. Testing the first page of an application form does not establish that validation, document upload, review and submission are accessible. [1] [3]

WCAG-EM 2.0 combines a structured sample with a random sample equal to 10 percent of the structured sample. The random selection is used to check whether the planned sample missed content types or findings. This requirement applies when an evaluation is represented as following WCAG-EM 2.0. Other targeted audits may use a different documented method. [1]

There is no credible universal answer such as ten pages for every website. Sample size depends on the number of templates, components, journeys, roles, technologies, languages, documents and dynamic states, as well as the risk and purpose of the evaluation. Two websites with the same page count can require very different effort.

The Testing Methods a Professional Audit Should Combine

No single method covers the full product. A professional audit combines complementary methods and records which pages, tools, browsers, assistive technologies and procedures were used. [1] [8] [9]

Expert Manual Review

Manual evaluation applies human judgement to every relevant WCAG success criterion at the target level. It checks whether content and interactions meet the intended outcome, not merely whether a code pattern exists. Typical areas include the following.

• Meaningful text alternatives, headings, landmarks, lists, tables and reading order.
• Colour contrast, use of colour, visible focus, reflow, text spacing and content at different zoom levels.
• Keyboard access, focus sequence, shortcuts, pointer alternatives and avoidance of keyboard traps.
• Form labels, instructions, required fields, error identification, suggestions and status messages.
• Accessible names, roles, values and states for native and custom controls.
• Captions, transcripts, audio description and controls for audio, video and animation.
• Timing, motion, flashing content, content on hover or focus and interruptions.
• Authentication, help, repeated entry and target-size requirements added or strengthened in WCAG 2.2.

Keyboard Testing

Keyboard testing follows complete tasks without using a mouse or touch input. The evaluator checks whether every interactive element can be reached and operated, whether focus moves in a meaningful order, whether it remains visible and unobscured, and whether the user can enter and exit components such as dialogs, menus and date pickers.

This testing should cover more than Tab and Enter. Depending on the component, expected operation may also involve Shift plus Tab, Space, arrow keys, Escape and other documented commands. The evaluator should record focus loss, unexpected movement, traps, skipped controls and actions that depend on pointer gestures.

Assistive Technology Testing

Assistive technology testing checks how the product communicates and behaves through the user agents that matter to its audience. The scope should define the combinations rather than making the impossible promise that every assistive technology and browser has been tested. WCAG-EM calls this the accessibility support baseline. [1]

Depending on the product and users, the baseline may include selected combinations such as the following.

• NVDA with a supported Windows browser.
• JAWS with a supported Windows browser where it is relevant to the audience or procurement requirement.
• VoiceOver with Safari on macOS or iOS.
• TalkBack with Chrome on Android for mobile services.
• Screen magnification, forced-colour settings or voice control when required by the product and audit objectives.

The evaluator should complete meaningful tasks, not only listen to a page from top to bottom. Testing should cover navigation by headings and landmarks, control names and states, form completion, validation, dynamic updates, dialogs, tables, account processes and the announcements made after an action.

Responsive Visual and Interaction Testing

A page can appear acceptable at its default desktop size and still fail when text is enlarged, the viewport becomes narrow or interface elements overlap. The audit should check reflow, zoom, text spacing, focus visibility, content revealed on hover or focus, orientation and touch target behaviour where applicable. Mobile web and native mobile applications may require separate scopes because their interaction models and assistive technologies differ.

Content Media and Documents

Accessibility defects are not limited to source code. Content authors control headings, link purpose, instructions, alternative text and plain communication. Media producers control captions, transcripts and audio description. Document authors control tags, reading order, table structure and form fields in PDFs and office documents.

A website audit should state whether downloadable files and embedded media are included. A page that links to an inaccessible application form or policy document can still block the user even when the HTML page itself is accessible. Documents may need their own representative sample or complete evaluation because their structures and testing methods differ from HTML.

Third Party Components

Payment gateways, identity services, chat tools, maps, video players, document viewers and consent platforms can become part of the user’s task. They should be identified during scoping and tested within the integrated journey. If the organisation cannot modify the component, the report should still explain the barrier, user impact, supplier dependency and any accessible alternative or escalation route.

Arabic and Bilingual Testing

For GCC organisations, Arabic testing is not a translation check added after the English audit. Arabic and English versions can use different content, layouts, components, documents and publishing workflows. Each supported language needs representative evaluation.

Arabic and bilingual testing should include the following.

• Correct page and passage language so screen readers use appropriate pronunciation.
• Right-to-left direction, visual order, reading order and keyboard focus order.
• Mixed Arabic and English text, numbers, dates, currency, addresses, telephone numbers and verification codes.
• Accessible names that match visible Arabic labels and communicate the same purpose.
• Arabic instructions, validation messages, status updates, alternatives, captions and support content.
• Mirrored or directional icons whose meaning changes in a right-to-left layout.
• Arabic PDFs, forms and tables, including tags, headings and reading order.
• Assistive technology behaviour in the actual language, browser and operating-system combinations in scope.

Passing the English interface does not establish that the Arabic interface passes. Direction changes can alter focus behaviour and component logic, while translated labels and error messages can create separate content failures. Automated tools are also unable to judge whether Arabic wording communicates the intended meaning.

What the Audit Report Should Contain

The report is the main working deliverable. It should allow an independent reader to understand what was evaluated, reproduce a defect, assess its impact and plan a correction. W3C emphasises transparency and replicability, while Section508.gov recommends detailed scope, methods, environments, outcomes, severity, evidence and remediation information. [1] [9]

A complete report should contain the following sections.

• Executive summary with the main risks, recurring patterns and recommended priorities.
• Product name, version, evaluation dates, evaluator and organisation responsible for the assessment.
• Evaluation purpose, exact scope, exclusions, assumptions and known access limitations.
• Target standard, version and conformance level, plus any additional legal or contractual criteria.
• Representative sample, critical journeys, selection method and technologies relied upon.
• Operating systems, browsers, tools, assistive technologies and evaluation methods used.
• Results for the relevant success criteria, including pass, fail and not applicable outcomes where required by the reporting method.
• A detailed issue register with clear evidence and prioritised guidance.
• A statement of limitations that prevents the findings from being interpreted beyond the tested scope.

Each reported defect should give the delivery team enough information to act. Useful fields include the following.

• A unique issue identifier and concise defect title.
• The affected page, view, component, user journey and language.
• The applicable WCAG success criterion and conformance outcome.
• A plain-language explanation of the barrier and the users likely to be affected.
• Steps to reproduce, including required account state, input and interaction.
• Expected behaviour and the behaviour observed during testing.
• Severity, frequency and whether the issue is isolated or systemic.
• Screenshots, video, DOM or code evidence where they improve understanding.
• Remediation guidance that explains the required accessible outcome.
• Retest status, date and evidence after the issue is addressed.

The audit report itself should be accessible. Headings, tables, links, colour contrast and reading order should be structured correctly, and any spreadsheet or PDF deliverables should be tested in their final format. A report about accessibility loses credibility when the intended readers cannot use it. [1] [9]

Severity Remediation and Retesting

Severity Is Not the WCAG Level

WCAG levels A, AA and AAA classify conformance requirements. They do not tell a product team which individual defect to fix first. An audit therefore needs a separate, documented severity method. A Level A failure may block a critical task, while a different failure may affect a narrower situation without preventing task completion.

Severity should consider factors such as the following.

• Whether the barrier prevents, seriously delays or merely inconveniences task completion.
• Which disability groups and interaction methods are affected.
• Whether an effective workaround exists and can be discovered independently.
• How often the component or pattern appears across the product.
• Whether the issue affects a high-risk journey such as authentication, payment, healthcare, education or employment.
• Whether one fix in a template or design-system component can remove the barrier across many pages.

Remediation Guidance

Useful remediation guidance explains the accessible outcome and, where appropriate, identifies a relevant code, design or content pattern. It should be specific enough for the delivery team to act without pretending that one generic code snippet is correct for every framework and component.

Findings should be grouped so teams can address root causes. If the same missing accessible name appears in a shared component across 200 pages, the report should identify the systemic component and representative instances. This supports one sustainable correction instead of 200 disconnected tickets.

A handover or triage workshop can clarify the highest-risk barriers, assign owners and separate design, development, content and third-party actions. The agreed outputs should be moved into the organisation’s normal issue-tracking system so accessibility work is planned alongside other product defects.

Verification of Fixes

Retesting is a separate quality step. The evaluator should verify the correction in the agreed environment, repeat the relevant interaction with the appropriate tools and assistive technologies, and check whether the change created a regression. Closing a ticket based only on a developer note or screenshot is not verification.

The agreement should distinguish between verification of listed findings and a full repeat audit. A focused retest confirms whether reported issues were resolved. It does not automatically cover newly added pages, components or workflows. Significant product changes may require an updated sample or a new evaluation. [1]