mirror of
https://github.com/supabase/supabase.git
synced 2026-10-05 17:35:10 +03:00
Miranda LimonczenkoandClaude Opus 5
21265b2e59
Ref DOCS-1274 Follow-up to #49017, now merged. This is the go-to-green piece: everything aimed at the three failing eval checks, and nothing else. Technical corrections follow in the PR stacked on this one. ## Problem `build-docs-002-rls-guide` points an agent at this guide with a vibe-coder prompt that never says RLS, policy, role, or test. Grants, policies, access probes, indexes, and security-definer placement all pass. Three checks fail, and have failed on every recorded run: | Check | What it measures | Why it failed | | --- | --- | --- | | `pgTAP test file(s) written under supabase/tests/` | Any `.sql` file exists | The agent never wrote one. | | `supabase test db runs at least 8 assertions and all pass` | Suite runs, ≥8 assertions, none failing | Nothing to run. The only example was `plan(4)`, under the floor even if copied perfectly. | | `tests assert allow and deny per operation … for anon and authenticated` | LLM judge on coverage | Never reached the judge: "no test files to review". | The guide already had a `Test your policies` section, so this isn't a strength problem. Agents don't read the page. They fetch it through an LLM extraction guided by their own query, and that query asked for enabling RLS, policy syntax, `auth.uid()`, indexes, and security definer functions. It never mentioned tests. A section about testing never enters the extract, so more testing prose cannot reach the agent. There was also a plain documentation bug underneath it: `Secure a table with RLS` said a table isn't secured until the suite passes, but the procedure beneath it ran 1–3 and ended on `grant`. A reader following the numbered steps finished without ever being told to write a test. ## Solution Put the tests where the procedure and the examples already are. - **`Secure a table with RLS`** opens with the four steps that finish a table, ending on `supabase test db`. Until the suite passes, you don't know whether the policies do what you intended. - **`Enable RLS and set the grants` gains step 4** — `supabase test new <table>_rls.test`, then `supabase test db`. The procedure ends on a passing suite instead of a grant. - **The public-read example** gains its policy and `announcements_rls.test.sql`, so a test file rides along in the enable-RLS extract. - **The four policy examples** are followed immediately by `profiles_rls.test.sql`, so one rides along in the `create policy` extract too. - **`Run the test suite` shrinks** to creating and running the files. It no longer carries content that has to survive extraction. - Each file leads with its own path as a comment, so it survives if the fence metadata is dropped. ### How that maps to the three checks | Check | Addressed by | | --- | --- | | Test files written | A complete test file now sits inside both extracts an agent's own query pulls, and step 4 of the procedure names the command that creates one. | | ≥8 assertions, all passing | `announcements_rls.test.sql` is `plan(10)`, `profiles_rls.test.sql` is `plan(14)`. Either alone clears the floor; together, 24. | | Coverage judge | `profiles` asserts allow **and** deny for all four operations. Allowed writes use `returning` + `results_eq`, proving state changed rather than that nothing raised. `using`-filtered denials use `is_empty`, asserting the row is unchanged rather than that an error was raised — the case the rubric explicitly fails suites for getting wrong. Both files switch role with `set local role` and identity with `set local request.jwt.claim.sub`, and cover `anon` as well as `authenticated`. | ## Manual testing 1. Open the [Row Level Security guide](https://docs-git-docs-rls-tests-in-procedure-supabase.vercel.app/docs/guides/database/postgres/row-level-security) on the preview. `Secure a table with RLS` opens with a four-step definition of done ending on `supabase test db`. 2. Read `Enable RLS and set the grants`. The procedure runs 1–4 and ends on writing and running the test, not on the grant. 3. Scroll to `DELETE policies`. The four policies are followed immediately by `profiles_rls.test.sql`, not a pointer to a later section. 4. Open the [markdown version](https://docs-git-docs-rls-tests-in-procedure-supabase.vercel.app/docs/guides/database/postgres/row-level-security.md), which is what agents fetch. Both test files are present, each leading with its path. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Documentation - Updated database security guidance for enabling row-level security and configuring grants. - Added per-table pgTAP testing requirements and revised `supabase test db` examples. - Expanded examples for permitted and denied access across public and authenticated roles. - Added dedicated guidance for profile testing and security-definer member/non-member cases. - Documented recursive-policy `42P17` failures and the security-definer workaround. - Clarified indexing, denial diagnosis, returned-row verification, and table-hardening links. - Streamlined the general policy-testing guidance. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Reference Docs
Supabase Reference Docs
Maintainers
If you are a maintainer of any tools in the Supabase ecosystem, you can use this site to provide documentation for the tools & libraries that you maintain.
DocSpec
We use documentation specifications which can be used to generate human-readable docs.
- OpenAPI: for documenting API endpoints.
- SDKSpec (custom to Supabase): for SDKs and client libraries.
- ConfigSpec (custom to Supabase): for configuration options.
- CLISpec (custom to Supabase): for CLI commands and usage.
The benefit of using custom specifications is that we can generate many other types from a strict schema (eg, HTML and manpages). It also means that we can switch to any documentation system we want. On this site we use Next.js, but on Supabase's official website, we use a custom React site and expose only a subset of the available API for each tool.
Contributing
To contribute to docs, see the developers' guide and contributing guide.