dima/supabase

0 Fork 0

mirror of https://github.com/supabase/supabase.git synced 2026-10-05 09:25:06 +03:00

Files

T

Miranda LimonczenkoandClaude Opus 5 777c02c205 test(docs): scan changed pages for WCAG 2.1 A/AA in warn mode (#48727 )

Closes DOCS-1233

## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Test coverage. The docs accessibility check now covers the full WCAG 2.1
A/AA rule set instead of two rules.

**Note:** This PR tests _only_ the main article of changed pages
(meaning, the content itself). A follow-up Linear issue is to address
scanning the pieces outside of that: header, navigation, and interactive
elements.

## What is the current behavior?

The `@a11y` test in `e2e/docs` runs two axe rules against each in-scope
page, `heading-order` and `page-has-heading-one`. Both already pass
everywhere, so the check only guards a result we have. Nothing else in
WCAG A/AA is checked.

## What is the new behavior?

The same test runs the full WCAG 2.1 A/AA rule set.

- **Existing debt does not block PRs.** Only the two heading rules fail.
Everything else reports.
- **The check stays fast.** It scans the article only and skips nine
rules that cannot fire there. Scan time drops from 2405ms to 981ms.
- **Findings belong to us.** Legacy mode excludes cross-origin frames.
YouTube embeds were counting against us, 11 of 15 violations on one
page.
- **A pass carries meaning.** A 404 reports as a load failure, not an
a11y bug. A page scanned before it hydrates warns instead of quietly
reporting clean.

## How the findings appear

The test is named `has no blocking accessibility violations`, so a
failure listed by CI is always something to fix. It is not named for the
full rule set, because a green check would then claim more than the
check verifies.

| | Rules | Where you see it |
| --- | --- | --- |
| Blocking | `heading-order`, `page-has-heading-one` | Test failure, so
the runner reports it on the PR |
| Reported | Everything else in WCAG A/AA | `::warning` annotation on
the run |

An annotation looks like this, on a run that still passes:

```
::warning title=Accessibility::/docs/guides/database/functions has 1 non-blocking accessibility finding(s): frame-title (4)
```

The full axe result for each page is attached to the report as
`axe-results.json`.

## Matching the Studio ratchet

This follows the ESLint ratchet in `apps/studio`. That pattern warns on
pre-existing debt rather than blocking on it, surfaces findings as
annotations rather than PR comments, and promotes a rule to an error
once its violations reach zero.

The mechanism here is `ENFORCED_RULES` in `utils/axe-helpers.ts`. The
two heading rules are on it because the heading-hierarchy work drove
them to zero site-wide.

The intent is to migrate rules into that list one at a time. Pick a
rule, fix its violations, then move it into `ENFORCED_RULES` so it
cannot come back. An exhaustive scan of the site groups the current
backlog by root cause to sequence that work, and two fixes cover 99.1%
of it.

Studio keeps per-file baseline counts, which this does not. A whole-rule
list is coarser, and it works here because docs violations reach zero
across the site rather than per file.

## Manual testing

Install the browser once, then run each step from the repo root. Every
command scans production, so you do not need a local docs server.

```bash
pnpm -C e2e/docs exec playwright install chromium
```

1. Confirm a reported finding does not fail the check.

```bash
DOCS_E2E_PAGE_PATHS=/docs/guides/database/functions
PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs:a11y
```

Expect `1 passed`, and the `::warning` annotation above in the output.

2. Confirm the scan finds that violation. Same page, now failing on
every rule.

```bash
A11Y_ENFORCE_ALL=1 DOCS_E2E_PAGE_PATHS=/docs/guides/database/functions
PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs:a11y
```

Expect `1 failed`, reporting `frame-title (serious, 4 node(s))`. Steps 1
and 2 together are the point of this PR.

3. Confirm the skipped rules stay skipped.

```bash
A11Y_ENFORCE_ALL=1
DOCS_E2E_PAGE_PATHS=/docs/guides/getting-started/quickstarts/nextjs
PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs:a11y
```

Expect `button-name (critical, 2 node(s))` and `label (critical, 2
node(s))`, and no `color-contrast`.

4. Confirm a page that does not load reports a load failure.

```bash
DOCS_E2E_PAGE_PATHS=/docs/guides/does-not-exist-xyz
PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs:a11y
```

Expect `Expected a successful response for
/docs/guides/does-not-exist-xyz, got 404`, and no axe assertion.

5. Confirm the link checker still passes alongside the a11y test.

```bash
DOCS_E2E_PAGE_PATHS=/docs/guides/auth/passwords
PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs
```

Expect `3 passed`.

## Known gaps

- `/docs/reference/*` is not scanned. Those routes render client-side
into tens of thousands of elements, where axe exceeds its timeout and
results depend on whether the scan caught the page mid-render.
- Shared chrome is outside the article scope, so nav, sidebar, footer,
menus, and drawers are not covered.
- axe catches roughly 30-40% of WCAG issues. Keyboard navigation, focus
management, and screen reader behavior still need manual testing.

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

2026-08-07 10:41:23 -07:00

3.0 KiB

Raw Blame History

Developing Supabase Docs

Getting started

Thanks for your interest in Supabase docs and for wanting to contribute! Before you begin, read the code of conduct and check out the existing issues. This document describes how to set up your development environment to contribute to Supabase docs.

For a complete run-down on how all of our tools work together, see the main DEVELOPERS.md. That readme describes how to get set up locally in lots of detail, including minimum requirements, our Turborepo setup, installing packages, sharing components across projects, and more. This readme deals specifically with the docs site.

Tip

If you work at Supabase, branch this repo directly to make PRs. Don't use a fork. This lets the CI checks auto-run and speeds up review.

Local setup

supabase.com/docs is a Next.js site. You can get setup by following the same steps for all of our other Next.js projects:

Follow the steps outlined in the Local Development section of the main DEVELOPERS.md
If you work at Supabase, from apps/docs run pnpm run dev:secrets:pull to write internal env vars to .env.local. If you're a community member, create apps/docs/.env.local and add this line: NEXT_PUBLIC_IS_PLATFORM=false
Start the local docs site by navigating to /apps/docs and running pnpm run dev
Visit http://localhost:3001/docs in your browser - don't forget to append the /docs to the end
Your local site should look exactly like https://supabase.com/docs

AI friendly documentation

This project generates Markdown files for each page under /docs/guides/.. path.

To test locally, within the apps/docs directory:

Run pnpm build:guides-markdown
Run pnpm dev

This creates Markdown files for all routes under the public/markdown/guides directory, ignored by Git.

For production this setup runs as a prebuild task to allow Vercel to bundle these files with middleware and functions.

Accessibility checks

Docs pages are scanned for WCAG 2.1 A/AA issues with axe-core, as part of the Playwright suite in e2e/docs. Pull requests scan the pages your change affects, limited to the main article.

To scan the pages your current branch changes:

PLAYWRIGHT_BASE_URL=https://supabase.com pnpm e2e:docs:a11y

That resolves which pages to scan from your branch, but reads them from production, so it won't see your edits and will 404 on a page you just added. Point PLAYWRIGHT_BASE_URL at your pull request's preview to scan your own content.

See e2e/docs/README.md for coverage and skipped rules.

Contributing

For repo organization and style guide, see the contributing guide.

3.0 KiB Raw Blame History