What the scan checks, criterion by criterion
Our ruleset has 15 rules, and they touch 13 of the 50 success criteria in WCAG 2.1 at levels A and AA. Here is which ones, and what is left for a person on every single criterion, including the ones we do check.
The number, and what it is not
Of 30 criteria at level A, our rules touch 10. Of 20 at level AA, they touch 3. That is the same order of magnitude every honest automated tool reports. In the UK government accessibility tool audit, automated tools found between 13 and 40 per cent of the deliberately introduced barriers; axe found around 30 per cent. The rest needs a person.
Read "touch", not "cover". A rule checks one concrete case of a criterion, not the criterion itself. Our rule for page titles sees an empty title; it cannot see that the title says Shop on all forty pages. That is why the right-hand column below exists for checked criteria too.
A scan that returns nothing means nothing was found in the part that can be found by machine. It never means the shop is accessible, and no report of ours will say otherwise.
Touched by a rule: 13 criteria
| Criterion | EN 301 549 | Rules | What the rules do not see |
|---|---|---|---|
| 1.1.1 Non-text Content (A) | 9.1.1.1 | a11y:image-alt | A person has to read each alternative text and judge whether it describes the product. The rule only sees an image with no alternative at all, and it cannot see text baked into an SVG, a CSS background or a canvas. |
| 1.3.1 Info and Relationships (A) | 9.1.3.1 | a11y:heading-order | The rule sees a skipped heading level. A person still has to check tables, lists and grouped checkout fields, where the structure can look right on screen and be absent from the markup. |
| 1.3.5 Identify Input Purpose (AA) | 9.1.3.5 | woo:checkout-field-autocomplete | The rule sees a known checkout field with no autocomplete. A person still has to check the fields your theme renamed, and confirm each token matches what the field actually asks for. |
| 2.1.1 Keyboard (A) | 9.2.1.1 | woo:variation-swatch-keyboard | The rule sees variation swatches the keyboard cannot reach. Every other custom control has to be tried by hand: the mini cart, the filter dropdown, the gallery, the accordion. |
| 2.4.1 Bypass Blocks (A) | 9.2.4.1 | a11y:skip-link | The rule sees a missing skip link and a missing main landmark. A person still has to follow the link and confirm it lands where it says it will. |
| 2.4.2 Page Titled (A) | 9.2.4.2 | a11y:page-title | The rule sees an empty title. A person has to judge whether it describes the page: the word Shop on every page passes the rule and fails the buyer. |
| 2.4.4 Link Purpose (In Context) (A) | 9.2.4.4 | a11y:link-name | The rule sees a link with no text at all. A person has to judge whether the text that is there says where the link goes, which read more never does. |
| 2.4.6 Headings and Labels (AA) | 9.2.4.6 | woo:quantity-stepper-name | The rule sees a quantity control with no name. A person has to judge whether the headings and labels on the page describe what they actually head and label. |
| 3.1.1 Language of Page (A) | 9.3.1.1 | a11y:html-lang | The rule sees a missing language. A person has to confirm the declared language is the one the page is actually written in. |
| 3.3.1 Error Identification (A) | 9.3.3.1 | woo:payment-error-not-announced | The rule sees a checkout error container that is never announced. A person has to submit the form wrongly and confirm the message says which field is wrong and why. |
| 3.3.2 Labels or Instructions (A) | 9.3.3.2 | woo:checkout-field-label, woo:required-field-not-marked | The rule sees fields with no label and required fields marked only by a theme class. A person has to judge whether the instructions are enough: the format of a postcode, the reason a phone number is asked for. |
| 4.1.2 Name, Role, Value (A) | 9.4.1.2 | woo:checkout-field-label, woo:variation-swatch-keyboard, woo:quantity-stepper-name, a11y:link-name, a11y:button-name | The rule sees controls with no accessible name. A person has to confirm role and state as well: a chosen swatch that never announces it is chosen passes the rule. |
| 4.1.3 Status Messages (AA) | 9.4.1.3 | woo:payment-error-not-announced, woo:cart-update-not-announced, woo:coupon-status-not-announced | The rule sees whether the page is able to announce anything at all. A person has to confirm the right message is announced at the right moment, when an item goes into the cart, when a coupon is applied, when a payment fails. |
Not checked by the scan: 37 criteria
These need a person. Some of them could be measured by machine on a rendered page and are on our roadmap; most cannot be measured at all, because they ask whether something makes sense, not whether an attribute is present.
| Criterion | EN 301 549 | What a person has to do |
|---|---|---|
| 1.2.1 Audio-only and Video-only (Prerecorded) (A) | 9.1.2.1 | A person has to watch and listen to every video, compare the captions against the audio and confirm a description of what happens on screen exists. Nothing about this can be measured by a scan. |
| 1.2.2 Captions (Prerecorded) (A) | 9.1.2.2 | A person has to watch and listen to every video, compare the captions against the audio and confirm a description of what happens on screen exists. Nothing about this can be measured by a scan. |
| 1.2.3 Audio Description or Media Alternative (Prerecorded) (A) | 9.1.2.3 | A person has to watch and listen to every video, compare the captions against the audio and confirm a description of what happens on screen exists. Nothing about this can be measured by a scan. |
| 1.2.4 Captions (Live) (AA) | 9.1.2.4 | A person has to watch and listen to every video, compare the captions against the audio and confirm a description of what happens on screen exists. Nothing about this can be measured by a scan. |
| 1.2.5 Audio Description (Prerecorded) (AA) | 9.1.2.5 | A person has to watch and listen to every video, compare the captions against the audio and confirm a description of what happens on screen exists. Nothing about this can be measured by a scan. |
| 1.3.2 Meaningful Sequence (A) | 9.1.3.2 | A person has to read the page with the stylesheet switched off and confirm the order it reads in still makes sense. |
| 1.3.3 Sensory Characteristics (A) | 9.1.3.3 | A person has to look for instructions that rely on shape, size or position, such as asking the buyer to press the round button on the right. |
| 1.3.4 Orientation (AA) | 9.1.3.4 | A person has to turn a phone sideways and confirm the shop still works in both orientations. |
| 1.4.1 Use of Color (A) | 9.1.4.1 | A person has to look for meaning carried by colour alone: a sale price, an out-of-stock label, a field that turns red and says nothing. |
| 1.4.2 Audio Control (A) | 9.1.4.2 | A person has to open the page and listen for audio that starts playing on its own. |
| 1.4.3 Contrast (Minimum) (AA) | 9.1.4.3 | Contrast has to be measured on the rendered page, and text over photographs, gradients and hover states has to be sampled by a person because the worst case is not the average case. |
| 1.4.4 Resize Text (AA) | 9.1.4.4 | A person has to zoom the text to 200 per cent, narrow the window to 320 pixels and increase letter and line spacing, then look for what is cut off, overlapped or pushed off screen. |
| 1.4.5 Images of Text (AA) | 9.1.4.5 | A person has to spot text that is really a picture: banners, price badges, shipping notices exported from a design tool. |
| 1.4.10 Reflow (AA) | 9.1.4.10 | A person has to zoom the text to 200 per cent, narrow the window to 320 pixels and increase letter and line spacing, then look for what is cut off, overlapped or pushed off screen. |
| 1.4.11 Non-text Contrast (AA) | 9.1.4.11 | Contrast has to be measured on the rendered page, and text over photographs, gradients and hover states has to be sampled by a person because the worst case is not the average case. |
| 1.4.12 Text Spacing (AA) | 9.1.4.12 | A person has to zoom the text to 200 per cent, narrow the window to 320 pixels and increase letter and line spacing, then look for what is cut off, overlapped or pushed off screen. |
| 1.4.13 Content on Hover or Focus (AA) | 9.1.4.13 | A person has to hover and focus every tooltip, dropdown and size chart, and confirm it can be dismissed and does not vanish when the pointer moves onto it. |
| 2.1.2 No Keyboard Trap (A) | 9.2.1.2 | A person has to tab through the page and confirm the focus can always get out again, above all out of the payment frame. |
| 2.1.4 Character Key Shortcuts (A) | 9.2.1.4 | A person has to check whether a single key does something on its own, such as a search box that swallows the letter s. |
| 2.2.1 Timing Adjustable (A) | 9.2.2.1 | A person has to check whether a session, a stock reservation or a countdown expires without a way to ask for more time. |
| 2.2.2 Pause, Stop, Hide (A) | 9.2.2.2 | A person has to find every carousel, ticker and auto-playing banner and confirm it can be stopped with the keyboard, not only by holding a mouse over it. |
| 2.3.1 Three Flashes or Below Threshold (A) | 9.2.3.1 | A person has to watch for content that flashes, and count the flashes if it does. |
| 2.4.3 Focus Order (A) | 9.2.4.3 | A person has to tab through the page and confirm the focus moves in the order the page reads. |
| 2.4.5 Multiple Ways (AA) | 9.2.4.5 | A person has to confirm a product can be found in more than one way: through search, through the menu, through a sitemap. |
| 2.4.7 Focus Visible (AA) | 9.2.4.7 | A person has to tab through the whole buying path and watch for the focus ring, including the one element where the theme removed it. |
| 2.5.1 Pointer Gestures (A) | 9.2.5.1 | A person has to work the gallery, the slider and the mobile menu with one finger, confirm an action can still be aborted by moving away before releasing, and check that nothing depends on tilting or shaking the device. |
| 2.5.2 Pointer Cancellation (A) | 9.2.5.2 | A person has to work the gallery, the slider and the mobile menu with one finger, confirm an action can still be aborted by moving away before releasing, and check that nothing depends on tilting or shaking the device. |
| 2.5.3 Label in Name (A) | 9.2.5.3 | A person has to compare the label on screen with the name a screen reader announces, so that voice control can say what is written on the button. |
| 2.5.4 Motion Actuation (A) | 9.2.5.4 | A person has to work the gallery, the slider and the mobile menu with one finger, confirm an action can still be aborted by moving away before releasing, and check that nothing depends on tilting or shaking the device. |
| 3.1.2 Language of Parts (AA) | 9.3.1.2 | A person has to find passages in another language, such as an English payment notice inside a German page, and confirm they are marked as such. |
| 3.2.1 On Focus (A) | 9.3.2.1 | A person has to focus and fill each field and confirm nothing submits, jumps or reloads on its own before they asked for it. |
| 3.2.2 On Input (A) | 9.3.2.2 | A person has to focus and fill each field and confirm nothing submits, jumps or reloads on its own before they asked for it. |
| 3.2.3 Consistent Navigation (AA) | 9.3.2.3 | These two compare pages against each other, which a single-page check cannot do: a person has to confirm the menu, the cart and the search sit in the same place and are named the same way throughout. |
| 3.2.4 Consistent Identification (AA) | 9.3.2.4 | These two compare pages against each other, which a single-page check cannot do: a person has to confirm the menu, the cart and the search sit in the same place and are named the same way throughout. |
| 3.3.3 Error Suggestion (AA) | 9.3.3.3 | A person has to submit a wrong postcode, a declined card and an empty required field, and confirm the shop suggests how to fix each one. |
| 3.3.4 Error Prevention (Legal, Financial, Data) (AA) | 9.3.3.4 | A person has to place a test order and confirm it can be reviewed, corrected and confirmed before money moves, and undone afterwards. |
| 4.1.1 Parsing (A) | 9.4.1.1 | A person or a validator has to confirm the markup of the page parses as the document it claims to be. |
Why we publish this
Because the alternative is a score. A score of 94 per cent tells a shop owner they are nearly done, and it is the single most expensive sentence in this industry: the remaining barriers are usually the ones that stop a purchase.
Because a supervisor asks what you checked, not what you scored. A list of what was not checked, with what you did about it, is a defensible answer. A percentage is not.
This table is generated from the ruleset itself when the page is built, so it cannot drift away from what the software actually does.