mirror of
https://github.com/supabase/supabase.git
synced 2026-10-05 09:25:06 +03:00
e357ec8f9f
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Docs update. ## What is the current behavior? Linear: [CLI-1618](https://linear.app/supabase/issue/CLI-1618/update-cli-workflow-docs-for-pg-delta-default-diffing) Four docs pages lag the shipped CLI behavior now that `pg-delta` is the default diff engine for projects created by a recent `supabase init`: - **CLI workflows** claims `db diff` compares `supabase/schemas/` against migrations. Under `pg-delta`, declarative files are never the `db diff` baseline (and `[db.migrations].schema_paths` no longer changes it) — the declarative flow goes through `supabase db schema declarative sync`. The cleanup guidance describes `migra`-era output. - **Declarative database schemas** teaches the old `db diff -f` + `schema_paths` flow throughout, and its known-caveats list is the `migra` issue list. - **Managing environments** still presents `--use-migra` as an "experimental flag" for a "more concise" diff — inverted now. - **Backup and restore (migrating within Supabase)** and the CLI workflows guide both steer users to `db diff`/`db pull` with `--schema auth,storage`. Under `pg-delta`, `--schema` layers an extra exclude policy on top of the Supabase profile: it can only narrow a diff, never re-include managed schemas, and managed-schema selections can even fail closed (e.g. `--schema auth` when a trigger function lives in `public`). Unfiltered diffs are the supported path. ## What is the new behavior? All claims verified against the CLI source at current `develop` — including supabase/cli#6300, which upgraded the engine to `@supabase/pg-delta` 1.0.0-alpha.46 — against the pinned pg-delta package source (profile rules, format defaults, coverage doc), and against a live dogfood run of the documented workflows on `develop` `38f31b4` (two OSS corpus projects, warm shadow cache). - **`cli-workflows.mdx`**: adds a "Which diff engine you're on" note (`pg-delta` for new `supabase init` projects, `migra` for existing ones until they opt in by adding `[experimental.pgdelta] enabled = true`; per-run fallbacks `--use-migra` on `db diff` / `--diff-engine migra` on `db pull`); corrects `db pull` and `db diff` mechanics (shadow built from migrations vs. live database; the baseline history record is offered, not unconditional); switches the declarative flow to `supabase db schema declarative sync`; reworks the cleanup section around pg-delta output (uppercase keywords at max width 180, `format_options`, per-unit migration files with numeric segment suffixes, the `-- pg-delta: transaction=false` directive on genuinely non-transactional files, engine-neutral grant/revoke review guidance, coverage warnings + `--strict-coverage`); documents what pg-delta captures in managed schemas (user triggers, RLS policies on `auth` tables and on `storage.objects`/`storage.buckets`/`realtime.messages`) versus what it doesn't; adds key-command rows for the declarative commands and troubleshooting entries (`db pull` non-zero exit when in sync, the `schema_paths` warning, `PGDELTA_DEBUG=1` bundles under `supabase/.temp/pgdelta/v2/debug/`). - **`declarative-database-schemas.mdx`**: swaps `db diff -f` for `db schema declarative sync -f` throughout; replaces lexicographic/`schema_paths` ordering guidance with automatic dependency ordering and the `generate` export layout (`_cluster/`, reserved `_custom/`); bootstraps from production via `db schema declarative generate --linked` (explicit target + `--overwrite` in scripts) and refreshes via `db pull --declarative`; rewrites known caveats for pg-delta (DML including storage buckets, untracked object kinds + the `_custom/` escape hatch, managed schemas, extension-managed objects, and the two gates when adopting an existing schema tree: `[experimental.webhooks]` for `pg_net` migrations and declaring the tree's extensions) keeping the `migra` workflow and issue list under a legacy section for projects that haven't enabled it. - **`managing-environments.mdx`**: frames the verbose grant sample as legacy-engine output, notes that generated migrations can include grant statements on any engine, describes `--use-migra` as a single-run fallback, and adds a `db diff --strict-coverage` CI step. - **`backup-restore.mdx`**: replaces `db diff --linked --schema auth,storage` with a plain `db diff --linked` on `pg-delta` (keeping the `--schema auth,storage` form for the legacy engine) and explains what the engine includes (user triggers on managed tables, user RLS policies on `auth`, `storage.objects`/`storage.buckets`/`realtime.messages`) and what must be recreated manually. - **New `diff-engines.mdx` page** (from #49889): the single home for how the engine is selected, a behavior matrix for `pg-delta` versus `migra`, the per-command fallback flags, a procedure for switching an existing project (the first `db pull` after enabling may write a catch-up migration), and how to go back with `enabled = false`. Registered in navigation. A shared `diff_engine_check` partial replaces the inline engine parentheticals across seven pages, and a `managed_schemas_diff_capture` partial carries the managed-schema capture rules. - **CLI reference (`cli_v1_commands.yaml`, `cli_v1_config.yaml`)**: `db pull`, `db schema declarative sync`/`generate` flags and descriptions, `experimental.pgdelta.*` and `db.migrations.schema_paths` config keys, and the `db diff` description updated to describe both engines. Note that `cli_v1_commands.yaml` is generated from the CLI repo; [supabase/cli#6557](https://github.com/supabase/cli/pull/6557) carries the matching `db pull` example and overlay text so the next publish keeps it. - **`examples/prompts/declarative-database-schema.md`**: rewritten for the `db schema declarative sync` flow, with the `[experimental.pgdelta]` prerequisite. ## Additional context The first draft was written against pg-delta 1.0.0-alpha.42. supabase/cli#6300 (engine upgrade to alpha.46) then changed two documented behaviors, both reflected here: generated SQL now defaults to uppercase pretty-printed keywords, and user RLS policies on `storage.objects`/`storage.buckets`/`realtime.messages` are included via the engine's `SUPABASE_USER_POLICY_SURFACES` allowlist. A follow-up dogfood run on `develop` `38f31b4` then falsified three more claims (pg-delta emits no grant noise, `_schema_changes`/`_after_enum_values` multi-file names, directive on every split file), all corrected in the last commit. **Update (Sep 14 to 17):** [#49889](https://github.com/supabase/supabase/pull/49889) and [#50220](https://github.com/supabase/supabase/pull/50220) were merged into this branch, so this PR now carries the full stack. #50220 corrected the `schema_paths` warning wording (the CLI warns only when the setting lists paths), added `auth` RLS policies to the managed-schema partial, and described the migra initial pull accurately (the `pg_dump` skips managed schemas and the migra diff pass that follows appends the trigger and policy changes). It also reframed `pg-delta` as the default for every project ahead of supabase/cli#6391. That plan changed: no breaking default flip before Select, so [#50332](https://github.com/supabase/supabase/pull/50332) restores the opt-in framing (`pg-delta` requires `[experimental.pgdelta] enabled = true`, which `supabase init` writes for new projects) and also resolves the four CodeRabbit findings from the latest review round. Two claims are pending confirmation from the owning teams: that branching runs every migration in a transaction and ignores the `-- pg-delta: transaction=false` directive, and the `--db-url` pooler-versus-direct connection advice, which currently disagrees with the CLI's own `db pull` docs. Stale spots found in the CLI repo's own docs while verifying (out of scope here, worth follow-ups): four `SIDE_EFFECTS.md` files still claim lowercase output, `docs/supabase/db/diff.md` still lists `migra`-era "known failure cases" that alpha.46 fully models, the `supabase init` template's commented `format_options` example shows `maxWidth: 80` against an actual default of 180, and the CLI upgrade recipe appends `--experimental` even when the config already enables pg-delta. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01SUuaVmXLRbV6tZjzhka3cp <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified `pg-delta` and legacy `migra` behavior, configuration, and switching guidance. * Expanded declarative schema workflows, including synchronization, migration generation, baselines, deployment, and legacy-engine support. * Documented managed schemas, permissions, extensions, transaction handling, dependency ordering, and troubleshooting. * Added guidance for strict coverage checks, output directories, non-interactive workflows, and declarative pull modes. * Added a dedicated diff engines guide and updated CLI navigation, backup and restore, branching, deployment, and CI documentation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Wen Bo Xie <wenbox323@gmail.com>
90 lines
12 KiB
Plaintext
90 lines
12 KiB
Plaintext
---
|
|
id: 'diff-engines'
|
|
title: 'Diff engines: pg-delta and migra'
|
|
description: 'How the Supabase CLI generates migrations, which engine your project uses, and how to switch.'
|
|
subtitle: 'How the Supabase CLI generates migrations, which engine your project uses, and how to switch.'
|
|
---
|
|
|
|
A diff engine compares two database states and generates the SQL that turns one into the other. It powers `supabase db diff`, `supabase db pull`, and the `supabase db schema declarative` commands. The CLI ships two engines, and the same command can behave differently depending on which one your project uses. This page explains how to tell which engine you're on, what differs between them, and how to move an existing project from one to the other.
|
|
|
|
[`pg-delta`](https://github.com/supabase/pg-toolbelt/tree/main/packages/pg-delta) is Supabase's open source engine and the default for projects created with a recent `supabase init`. Rather than parsing SQL to understand your schema, it loads each state into a real Postgres instance and reads the result from the catalog. It models entities the legacy engine missed, such as comments, domains, roles, publication membership, and the security invoker setting on views, and it supports Postgres 14 through 18. It ships under the `[experimental.pgdelta]` config namespace and remains pre-1.0.
|
|
|
|
[`migra`](https://github.com/djrobstep/migra) is `djrobstep`'s open source Python library for diffing Postgres schemas, and the CLI has shelled out to it since before `pg-delta` existed. It compares two live Postgres databases and generates the SQL that turns one into the other, the mechanism behind `supabase db diff`. Projects that haven't opted in to `pg-delta` continue to use it, and every command they relied on keeps working.
|
|
|
|
## Which engine your project uses
|
|
|
|
<$Partial path="diff_engine_check.mdx" />
|
|
|
|
Projects created with a recent `supabase init` use `pg-delta` because `init` writes this into `config.toml`:
|
|
|
|
```toml name=supabase/config.toml
|
|
[experimental.pgdelta]
|
|
enabled = true
|
|
```
|
|
|
|
Existing projects without that section keep the legacy `migra` engine until they add it. See [Switch an existing project to `pg-delta`](#switch-an-existing-project-to-pg-delta).
|
|
|
|
To use `migra` for a single run without changing `config.toml`, the flag depends on the command:
|
|
|
|
| Command | Flag to fall back to `migra` |
|
|
| --------- | ---------------------------- |
|
|
| `db diff` | `--use-migra` |
|
|
| `db pull` | `--diff-engine migra` |
|
|
|
|
To opt out entirely, set `enabled = false` under `[experimental.pgdelta]`.
|
|
|
|
## What differs between the engines
|
|
|
|
The commands and the migration workflow are the same on both engines. What differs is what each command reads, what it captures, and what it writes.
|
|
|
|
| Behavior | `pg-delta` | Legacy `migra` |
|
|
| -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| What `db diff` compares | Always the live database against a shadow database built from your migrations. Never reads `supabase/schemas/`. | The live database against your migrations, unless `supabase/schemas/` contains declarative files. Then it compares those files against your migrations instead. |
|
|
| Generating migrations from declarative files | `supabase db schema declarative sync` | `supabase db diff`, with the local stack stopped |
|
|
| Exporting a database into declarative files | `supabase db schema declarative generate` | Not available. Use `supabase db dump` and split the output by hand. |
|
|
| Ordering of declarative files | Automatic, based on dependencies between statements. `[db.migrations].schema_paths` is ignored, and the CLI warns when the setting lists any paths. | Lexicographic by default. `[db.migrations].schema_paths` overrides the order. Files in `supabase/schemas/` are used even when the setting is empty. |
|
|
| Initial `db pull` (empty migration history) | Diffs the remote database against an empty shadow database. Your customizations in managed schemas, such as triggers on `auth.users` and RLS policies on `storage.objects`, are included. | Seeds the migration with `pg_dump`, which skips managed schemas, then appends a diff of your triggers and RLS policies in `auth` and `storage`. Older versions excluded those schemas and told you to run `supabase db pull --schema auth,storage`. |
|
|
| Later `db pull` runs | Diffs the remote database against your migrations. | Same. Trigger and RLS policy changes in managed schemas are included. |
|
|
| `db pull --declarative` | Replaces `supabase/schemas/` from the selected database without creating a migration. | Runs, but don't use it. The tree it writes is only readable by the `db schema declarative` commands. `db diff` on the legacy engine then tries to load that tree as its baseline and fails. |
|
|
| Objects the engine doesn't track | Reported as warnings and never silently dropped. Pass `--strict-coverage` to turn the warnings into failures. | Silently omitted. `--strict-coverage` has no effect. |
|
|
| Generated SQL | Uppercase keywords wrapped at 180 characters. Configurable with `[experimental.pgdelta] format_options`. | Engine default formatting. |
|
|
| Migration files per run | Usually one. When a change crosses a transaction boundary, it may write one ordered file per unit with `_1`, `_2` suffixes. Files whose statements can't run in a transaction start with `-- pg-delta: transaction=false`. | One file. |
|
|
| Known limitations | See [Known caveats](/docs/guides/local-development/declarative-database-schemas#known-caveats). | See [Limitations of the legacy engine](/docs/guides/local-development/declarative-database-schemas#limitations-of-the-legacy-engine). |
|
|
|
|
For a walkthrough of the day-to-day commands on `pg-delta`, see [Local development workflow](/docs/guides/local-development/cli-workflows). For the declarative workflow on each engine, see [Declarative database schemas](/docs/guides/local-development/declarative-database-schemas).
|
|
|
|
## Switch an existing project to `pg-delta` [#switch-an-existing-project-to-pg-delta]
|
|
|
|
Switching is a one-time change to `config.toml` followed by a catch-up migration. Do it on a branch so you can review everything before it reaches your team.
|
|
|
|
1. Add the engine setting to `config.toml`:
|
|
|
|
```toml name=supabase/config.toml
|
|
[experimental.pgdelta]
|
|
enabled = true
|
|
```
|
|
|
|
2. If your project uses declarative schemas, remove `[db.migrations].schema_paths`. Ordering is automatic on `pg-delta`, and the CLI warns when the setting lists any paths.
|
|
|
|
3. Pull once against your linked project to create a catch-up migration:
|
|
|
|
```bash
|
|
supabase db pull
|
|
```
|
|
|
|
If your history was built from legacy diffs, it doesn't mention objects the legacy engine didn't track, such as comments, domains, roles, and publication membership. This first pull captures them in a single catch-up migration. Your remote database already has these objects, so accept the prompt to record the migration as applied and review the file like any other generated migration. If your baseline came from the legacy engine's `pg_dump` path, it already contains those objects, and this pull reports "No schema changes found". There is nothing to commit in that case. See [Cleaning up generated migrations](/docs/guides/local-development/cli-workflows#cleaning-up-generated-migrations).
|
|
|
|
4. Replace `supabase db diff` with `supabase db schema declarative sync` wherever you generate migrations from declarative files, including shell scripts, CI jobs, and AI prompt files such as [`examples/prompts/declarative-database-schema.md`](https://github.com/supabase/supabase/blob/master/examples/prompts/declarative-database-schema.md). You no longer need to run `supabase stop` before generating. If the CLI didn't generate your schema tree, see [Adopting an existing schema tree](/docs/guides/local-development/declarative-database-schemas#adopting-an-existing-schema-tree) for two checks that run on the first `sync`.
|
|
|
|
5. Verify the full migration chain locally:
|
|
|
|
```bash
|
|
supabase db reset
|
|
```
|
|
|
|
6. Optionally, add `supabase db diff --strict-coverage` to CI so untracked objects fail the job instead of producing warnings. See [Managing environments](/docs/guides/deployment/managing-environments).
|
|
|
|
If you deploy through the [GitHub integration](/docs/guides/deployment/branching/github-integration), branching runs every migration inside a transaction and doesn't honor the `-- pg-delta: transaction=false` directive. A migration that carries it applies with `supabase db push` but fails when branching deploys it.
|
|
|
|
To go back, set `enabled = false` under `[experimental.pgdelta]`, or use the per-command flags above for a single run. Migrations already generated by `pg-delta` are plain SQL and keep working on either engine.
|