mirror of
https://github.com/supabase/supabase.git
synced 2026-10-05 09:25:06 +03:00
docs(cli): update local development workflow docs for pg-delta default diffing (#49280)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Docs update. ## What is the current behavior? Linear: [CLI-1618](https://linear.app/supabase/issue/CLI-1618/update-cli-workflow-docs-for-pg-delta-default-diffing) Four docs pages lag the shipped CLI behavior now that `pg-delta` is the default diff engine for projects created by a recent `supabase init`: - **CLI workflows** claims `db diff` compares `supabase/schemas/` against migrations. Under `pg-delta`, declarative files are never the `db diff` baseline (and `[db.migrations].schema_paths` no longer changes it) — the declarative flow goes through `supabase db schema declarative sync`. The cleanup guidance describes `migra`-era output. - **Declarative database schemas** teaches the old `db diff -f` + `schema_paths` flow throughout, and its known-caveats list is the `migra` issue list. - **Managing environments** still presents `--use-migra` as an "experimental flag" for a "more concise" diff — inverted now. - **Backup and restore (migrating within Supabase)** and the CLI workflows guide both steer users to `db diff`/`db pull` with `--schema auth,storage`. Under `pg-delta`, `--schema` layers an extra exclude policy on top of the Supabase profile: it can only narrow a diff, never re-include managed schemas, and managed-schema selections can even fail closed (e.g. `--schema auth` when a trigger function lives in `public`). Unfiltered diffs are the supported path. ## What is the new behavior? All claims verified against the CLI source at current `develop` — including supabase/cli#6300, which upgraded the engine to `@supabase/pg-delta` 1.0.0-alpha.46 — against the pinned pg-delta package source (profile rules, format defaults, coverage doc), and against a live dogfood run of the documented workflows on `develop` `38f31b4` (two OSS corpus projects, warm shadow cache). - **`cli-workflows.mdx`**: adds a "Which diff engine you're on" note (`pg-delta` for new `supabase init` projects, `migra` for existing ones until they opt in by adding `[experimental.pgdelta] enabled = true`; per-run fallbacks `--use-migra` on `db diff` / `--diff-engine migra` on `db pull`); corrects `db pull` and `db diff` mechanics (shadow built from migrations vs. live database; the baseline history record is offered, not unconditional); switches the declarative flow to `supabase db schema declarative sync`; reworks the cleanup section around pg-delta output (uppercase keywords at max width 180, `format_options`, per-unit migration files with numeric segment suffixes, the `-- pg-delta: transaction=false` directive on genuinely non-transactional files, engine-neutral grant/revoke review guidance, coverage warnings + `--strict-coverage`); documents what pg-delta captures in managed schemas (user triggers, RLS policies on `auth` tables and on `storage.objects`/`storage.buckets`/`realtime.messages`) versus what it doesn't; adds key-command rows for the declarative commands and troubleshooting entries (`db pull` non-zero exit when in sync, the `schema_paths` warning, `PGDELTA_DEBUG=1` bundles under `supabase/.temp/pgdelta/v2/debug/`). - **`declarative-database-schemas.mdx`**: swaps `db diff -f` for `db schema declarative sync -f` throughout; replaces lexicographic/`schema_paths` ordering guidance with automatic dependency ordering and the `generate` export layout (`_cluster/`, reserved `_custom/`); bootstraps from production via `db schema declarative generate --linked` (explicit target + `--overwrite` in scripts) and refreshes via `db pull --declarative`; rewrites known caveats for pg-delta (DML including storage buckets, untracked object kinds + the `_custom/` escape hatch, managed schemas, extension-managed objects, and the two gates when adopting an existing schema tree: `[experimental.webhooks]` for `pg_net` migrations and declaring the tree's extensions) keeping the `migra` workflow and issue list under a legacy section for projects that haven't enabled it. - **`managing-environments.mdx`**: frames the verbose grant sample as legacy-engine output, notes that generated migrations can include grant statements on any engine, describes `--use-migra` as a single-run fallback, and adds a `db diff --strict-coverage` CI step. - **`backup-restore.mdx`**: replaces `db diff --linked --schema auth,storage` with a plain `db diff --linked` on `pg-delta` (keeping the `--schema auth,storage` form for the legacy engine) and explains what the engine includes (user triggers on managed tables, user RLS policies on `auth`, `storage.objects`/`storage.buckets`/`realtime.messages`) and what must be recreated manually. - **New `diff-engines.mdx` page** (from #49889): the single home for how the engine is selected, a behavior matrix for `pg-delta` versus `migra`, the per-command fallback flags, a procedure for switching an existing project (the first `db pull` after enabling may write a catch-up migration), and how to go back with `enabled = false`. Registered in navigation. A shared `diff_engine_check` partial replaces the inline engine parentheticals across seven pages, and a `managed_schemas_diff_capture` partial carries the managed-schema capture rules. - **CLI reference (`cli_v1_commands.yaml`, `cli_v1_config.yaml`)**: `db pull`, `db schema declarative sync`/`generate` flags and descriptions, `experimental.pgdelta.*` and `db.migrations.schema_paths` config keys, and the `db diff` description updated to describe both engines. Note that `cli_v1_commands.yaml` is generated from the CLI repo; [supabase/cli#6557](https://github.com/supabase/cli/pull/6557) carries the matching `db pull` example and overlay text so the next publish keeps it. - **`examples/prompts/declarative-database-schema.md`**: rewritten for the `db schema declarative sync` flow, with the `[experimental.pgdelta]` prerequisite. ## Additional context The first draft was written against pg-delta 1.0.0-alpha.42. supabase/cli#6300 (engine upgrade to alpha.46) then changed two documented behaviors, both reflected here: generated SQL now defaults to uppercase pretty-printed keywords, and user RLS policies on `storage.objects`/`storage.buckets`/`realtime.messages` are included via the engine's `SUPABASE_USER_POLICY_SURFACES` allowlist. A follow-up dogfood run on `develop` `38f31b4` then falsified three more claims (pg-delta emits no grant noise, `_schema_changes`/`_after_enum_values` multi-file names, directive on every split file), all corrected in the last commit. **Update (Sep 14 to 17):** [#49889](https://github.com/supabase/supabase/pull/49889) and [#50220](https://github.com/supabase/supabase/pull/50220) were merged into this branch, so this PR now carries the full stack. #50220 corrected the `schema_paths` warning wording (the CLI warns only when the setting lists paths), added `auth` RLS policies to the managed-schema partial, and described the migra initial pull accurately (the `pg_dump` skips managed schemas and the migra diff pass that follows appends the trigger and policy changes). It also reframed `pg-delta` as the default for every project ahead of supabase/cli#6391. That plan changed: no breaking default flip before Select, so [#50332](https://github.com/supabase/supabase/pull/50332) restores the opt-in framing (`pg-delta` requires `[experimental.pgdelta] enabled = true`, which `supabase init` writes for new projects) and also resolves the four CodeRabbit findings from the latest review round. Two claims are pending confirmation from the owning teams: that branching runs every migration in a transaction and ignores the `-- pg-delta: transaction=false` directive, and the `--db-url` pooler-versus-direct connection advice, which currently disagrees with the CLI's own `db pull` docs. Stale spots found in the CLI repo's own docs while verifying (out of scope here, worth follow-ups): four `SIDE_EFFECTS.md` files still claim lowercase output, `docs/supabase/db/diff.md` still lists `migra`-era "known failure cases" that alpha.46 fully models, the `supabase init` template's commented `format_options` example shows `maxWidth: 80` against an actual default of 180, and the CLI upgrade recipe appends `--experimental` even when the config already enables pg-delta. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01SUuaVmXLRbV6tZjzhka3cp <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Clarified `pg-delta` and legacy `migra` behavior, configuration, and switching guidance. * Expanded declarative schema workflows, including synchronization, migration generation, baselines, deployment, and legacy-engine support. * Documented managed schemas, permissions, extensions, transaction handling, dependency ordering, and troubleshooting. * Added guidance for strict coverage checks, output directories, non-interactive workflows, and declarative pull modes. * Added a dedicated diff engines guide and updated CLI navigation, backup and restore, branching, deployment, and CI documentation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Wen Bo Xie <wenbox323@gmail.com>
This commit is contained in:
19 files changed
+556
-190
No files matched your search
@@ -2961,10 +2961,20 @@ commands:
|
||||
name: --name <string>
|
||||
description: Name for the generated migration file.
|
||||
default_value: ''
|
||||
- id: no-apply
|
||||
name: --no-apply
|
||||
description: |
|
||||
Generate the migration file without prompting or applying it to the local database.
|
||||
default_value: 'false'
|
||||
- id: schema
|
||||
name: -s, --schema <strings>
|
||||
description: Comma separated list of schema to include.
|
||||
default_value: '[]'
|
||||
- id: strict-coverage
|
||||
name: --strict-coverage
|
||||
description: |
|
||||
Fail when bundled pg-delta finds schema objects it cannot manage instead of leaving them unmanaged.
|
||||
default_value: 'false'
|
||||
- id: experimental
|
||||
name: --experimental
|
||||
description: enable experimental features
|
||||
@@ -3002,6 +3012,11 @@ commands:
|
||||
name: --local
|
||||
description: Generates declarative schema from the local database.
|
||||
default_value: 'false'
|
||||
- id: output-dir
|
||||
name: --output-dir <string>
|
||||
description: |
|
||||
Write declarative schema files to this directory instead of the configured declarative schema directory.
|
||||
default_value: ''
|
||||
- id: overwrite
|
||||
name: --overwrite
|
||||
description: Overwrite declarative schema files without confirmation.
|
||||
@@ -3019,6 +3034,11 @@ commands:
|
||||
name: -s, --schema <strings>
|
||||
description: Comma separated list of schema to include.
|
||||
default_value: '[]'
|
||||
- id: strict-coverage
|
||||
name: --strict-coverage
|
||||
description: |
|
||||
Fail when bundled pg-delta finds schema objects it cannot manage instead of leaving them unmanaged.
|
||||
default_value: 'false'
|
||||
- id: experimental
|
||||
name: --experimental
|
||||
description: enable experimental features
|
||||
@@ -3212,9 +3232,11 @@ commands:
|
||||
|
||||
Optionally, a new row can be inserted into the migration history table to reflect the current state of the remote database.
|
||||
|
||||
If no entries exist in the migration history table, `pg_dump` will be used to capture all contents of the remote schemas you have created. Otherwise, this command will only diff schema changes against the remote database, similar to running `db diff --linked`.
|
||||
If no entries exist in the migration history table, the behavior depends on the diff engine. Under `pg-delta`, the remote database is diffed against an empty shadow database. Under the legacy `migra` engine, the CLI uses `pg_dump` to capture all contents of the remote schemas you have created. Otherwise, this command only diffs schema changes against the remote database, similar to running `db diff --linked`.
|
||||
|
||||
Pass `--diff-engine pg-delta` to keep the migration-file `db pull` workflow while using pg-delta for the shadow diff step. Pass `--use-pg-delta` to switch to the declarative pg-delta export workflow instead.
|
||||
When `[experimental.pgdelta] enabled = true`, the migration-file `db pull` workflow uses pg-delta for the shadow diff step; pass `--diff-engine migra` to fall back for a single run. Projects without that setting use migra; pass `--diff-engine pg-delta` to use pg-delta for a single run. Pass `--declarative` to replace the declarative schema tree (`supabase/schemas` by default, configurable with `experimental.pgdelta.declarative_schema_path`) from the selected database instead of creating a migration.
|
||||
|
||||
Under the pg-delta engine, your own customizations in Supabase-managed schemas are captured automatically, such as triggers on managed tables like `auth.users` and RLS policies on `storage.objects` and `storage.buckets`. Under the legacy migra engine, the initial pull's `pg_dump` skips managed schemas, but the diff pass that follows appends your trigger and RLS policy changes there to the same migration. If an older CLI reports that `auth` and `storage` were excluded, capture existing customizations there with `--schema auth,storage`.
|
||||
examples:
|
||||
- id: basic-usage
|
||||
name: Basic usage
|
||||
@@ -3225,28 +3247,29 @@ commands:
|
||||
Update remote migration history table? [Y/n]
|
||||
Repaired migration history: [20240414044403] => applied
|
||||
Finished supabase db pull.
|
||||
The auth and storage schemas are excluded. Run supabase db pull --schema auth,storage again to diff them.
|
||||
- id: local-studio
|
||||
name: Local studio
|
||||
code: supabase db pull --local
|
||||
response: |
|
||||
Connecting to local database...
|
||||
Setting up initial schema....
|
||||
Creating custom roles supabase/roles.sql...
|
||||
Creating shadow database...
|
||||
Seeding globals from roles.sql...
|
||||
Applying migration 20240414044403_remote_schema.sql...
|
||||
Diffing schemas...
|
||||
No schema changes found
|
||||
The auth and storage schemas are excluded. Run supabase db pull --schema auth,storage again to diff them.
|
||||
The remote database is already in sync with your local migrations — nothing to pull.
|
||||
exit status 1
|
||||
- id: custom-schemas
|
||||
name: Custom schemas
|
||||
code: supabase db pull --schema auth,storage
|
||||
code: supabase db pull --schema public,extensions
|
||||
response: |
|
||||
Connecting to remote database...
|
||||
Setting up initial schema....
|
||||
Creating custom roles supabase/roles.sql...
|
||||
Creating shadow database...
|
||||
Seeding globals from roles.sql...
|
||||
Applying migration 20240414044403_remote_schema.sql...
|
||||
Diffing schemas: public,extensions
|
||||
No schema changes found
|
||||
Try rerunning the command with --debug to troubleshoot the error.
|
||||
The remote database is already in sync with your local migrations — nothing to pull.
|
||||
exit status 1
|
||||
tags: []
|
||||
links: []
|
||||
@@ -3258,6 +3281,11 @@ commands:
|
||||
description: |
|
||||
Pulls from the database specified by the connection string (must be percent-encoded).
|
||||
default_value: ''
|
||||
- id: declarative
|
||||
name: --declarative
|
||||
description: |
|
||||
Replace the declarative schema tree from the selected database instead of creating a migration. Migration history is not updated. Only use this on projects with `[experimental.pgdelta] enabled = true`. On the legacy migra engine, `db diff` tries to load the generated tree as its baseline and fails.
|
||||
default_value: 'false'
|
||||
- id: diff-engine
|
||||
name: --diff-engine <[ migra | pg-delta ]>
|
||||
description: Diff engine to use for migration-style db pull.
|
||||
@@ -3285,9 +3313,15 @@ commands:
|
||||
name: -s, --schema <strings>
|
||||
description: Comma separated list of schema to include.
|
||||
default_value: '[]'
|
||||
- id: strict-coverage
|
||||
name: --strict-coverage
|
||||
description: |
|
||||
Fail when bundled pg-delta finds schema objects it cannot manage instead of leaving them unmanaged.
|
||||
default_value: 'false'
|
||||
- id: use-pg-delta
|
||||
name: --use-pg-delta
|
||||
description: Use pg-delta to pull declarative schema.
|
||||
description: |
|
||||
Use pg-delta to pull declarative schema. Deprecated: use `--declarative` instead.
|
||||
default_value: 'false'
|
||||
- id: supabase-db-lint
|
||||
title: supabase db lint
|
||||
@@ -3484,11 +3518,13 @@ commands:
|
||||
|
||||
Requires the local development stack to be running when diffing against the local database. To diff against a remote or self-hosted database, specify the `--linked` or `--db-url` flag respectively.
|
||||
|
||||
Runs [djrobstep/migra](https://github.com/djrobstep/migra) in a container to compare schema differences between the target database and a shadow database. The shadow database is created by applying migrations in local `supabase/migrations` directory in a separate container. Output is written to stdout by default. For convenience, you can also save the schema diff as a new migration file by passing in `-f` flag.
|
||||
Compares schema differences between the target database and a shadow database. Projects with `[experimental.pgdelta] enabled = true` use the bundled pg-delta engine; other projects run the legacy [djrobstep/migra](https://github.com/djrobstep/migra) engine in a container. The shadow database is created by applying migrations in local `supabase/migrations` directory in a separate container. Output is written to stdout by default. For convenience, you can also save the schema diff as a new migration file by passing in `-f` flag.
|
||||
|
||||
By default, all schemas in the target database are diffed. Use the `--schema public,extensions` flag to restrict diffing to a subset of schemas.
|
||||
|
||||
While the diff command is able to capture most schema changes, there are cases where it is known to fail. Currently, this could happen if you schema contains:
|
||||
Projects created by a recent `supabase init` default to pg-delta (`[experimental.pgdelta] enabled = true` in `config.toml`). Existing projects keep using migra unless they opt in. To fall back to migra for a single run on a pg-delta project, pass `--use-migra`.
|
||||
|
||||
Under the legacy migra engine, the diff is known to miss some changes. This could happen if your schema contains:
|
||||
|
||||
- Changes to publication
|
||||
- Changes to storage buckets
|
||||
@@ -3560,6 +3596,11 @@ commands:
|
||||
name: -s, --schema <strings>
|
||||
description: Comma separated list of schema to include.
|
||||
default_value: '[]'
|
||||
- id: strict-coverage
|
||||
name: --strict-coverage
|
||||
description: |
|
||||
Fail when bundled pg-delta finds schema objects it cannot manage instead of leaving them unmanaged.
|
||||
default_value: 'false'
|
||||
- id: to
|
||||
name: --to <string>
|
||||
description: Diff to local, linked, migrations, or a Postgres URL.
|
||||
|
||||
@@ -449,6 +449,18 @@ parameters:
|
||||
- name: 'PgBouncer Configuration'
|
||||
link: https://www.pgbouncer.org/config.html#max_client_conn
|
||||
|
||||
- id: 'db.migrations.schema_paths'
|
||||
title: 'db.migrations.schema_paths'
|
||||
tags: ['database']
|
||||
required: false
|
||||
default: '[]'
|
||||
description: |
|
||||
An ordered list of schema files, directories, or glob patterns describing your database, relative to the `supabase` directory. For example: `["./schemas/*.sql"]`.
|
||||
Only the legacy `migra` diff engine uses this setting, where it controls the order in which declarative schema files are applied. The `pg-delta` diff engine (see `experimental.pgdelta.enabled`) ignores it and orders declarative files automatically based on the dependencies between statements.
|
||||
links:
|
||||
- name: 'Declarative database schemas'
|
||||
link: 'https://supabase.com/docs/guides/local-development/declarative-database-schemas'
|
||||
|
||||
- id: 'db.seed.enabled'
|
||||
title: 'db.seed.enabled'
|
||||
tags: ['database']
|
||||
@@ -1799,6 +1811,42 @@ parameters:
|
||||
Note: This is an experimental feature and may change in future releases.
|
||||
links: []
|
||||
|
||||
- id: 'experimental.pgdelta.enabled'
|
||||
title: 'experimental.pgdelta.enabled'
|
||||
tags: ['experimental']
|
||||
required: false
|
||||
default: 'false'
|
||||
description: |
|
||||
Enables the `pg-delta` diff engine for `db diff`, `db pull`, and the `db schema declarative` commands.
|
||||
Projects created with a recent `supabase init` have this set to `true` in the generated `config.toml`. Existing projects keep the legacy `migra` engine until they opt in by adding this setting. Set to `false` to fall back to `migra`.
|
||||
Note: This is an experimental feature and may change in future releases.
|
||||
links:
|
||||
- name: 'Declarative database schemas'
|
||||
link: 'https://supabase.com/docs/guides/local-development/declarative-database-schemas'
|
||||
|
||||
- id: 'experimental.pgdelta.declarative_schema_path'
|
||||
title: 'experimental.pgdelta.declarative_schema_path'
|
||||
tags: ['experimental']
|
||||
required: false
|
||||
default: './schemas'
|
||||
description: |
|
||||
The directory where the `pg-delta` engine reads and writes declarative schema files. Relative paths resolve from the `supabase` directory.
|
||||
Note: This is an experimental feature and may change in future releases.
|
||||
links:
|
||||
- name: 'Declarative database schemas'
|
||||
link: 'https://supabase.com/docs/guides/local-development/declarative-database-schemas'
|
||||
|
||||
- id: 'experimental.pgdelta.format_options'
|
||||
title: 'experimental.pgdelta.format_options'
|
||||
tags: ['experimental']
|
||||
required: false
|
||||
description: |
|
||||
A JSON string of formatting options applied to SQL generated by the `pg-delta` engine. When unset, generated SQL uses uppercase keywords wrapped at a maximum line width of 180 characters.
|
||||
Supported keys: `keywordCase` (`"upper"`, `"lower"`, or `"preserve"`), `commaStyle` (`"trailing"` or `"leading"`), `indent` (number), `maxWidth` (number), and the booleans `alignColumns`, `alignKeyValues`, `preserveRoutineBodies`, `preserveViewBodies`, and `preserveRuleBodies`. Omitted keys keep their defaults.
|
||||
For example: `format_options = "{\"keywordCase\":\"lower\",\"maxWidth\":80}"`. Set `format_options = "null"` to emit raw statements with no formatting applied.
|
||||
Note: This is an experimental feature and may change in future releases.
|
||||
links: []
|
||||
|
||||
- id: 'remotes.branch_name.project_id'
|
||||
title: 'remotes.<branch_name>.project_id'
|
||||
tags: ['branching']
|
||||
|
||||
@@ -160,6 +160,30 @@
|
||||
"title": "Start only the local database",
|
||||
"slug": "supabase-db-start",
|
||||
"type": "cli-command"
|
||||
},
|
||||
{
|
||||
"id": "supabase-db-schema",
|
||||
"title": "Manage database schema",
|
||||
"slug": "supabase-db-schema",
|
||||
"type": "cli-command"
|
||||
},
|
||||
{
|
||||
"id": "supabase-db-schema-declarative",
|
||||
"title": "Manage declarative database schemas",
|
||||
"slug": "supabase-db-schema-declarative",
|
||||
"type": "cli-command"
|
||||
},
|
||||
{
|
||||
"id": "supabase-db-schema-declarative-sync",
|
||||
"title": "Generate a new migration from declarative schema",
|
||||
"slug": "supabase-db-schema-declarative-sync",
|
||||
"type": "cli-command"
|
||||
},
|
||||
{
|
||||
"id": "supabase-db-schema-declarative-generate",
|
||||
"title": "Generate declarative schema from a database",
|
||||
"slug": "supabase-db-schema-declarative-generate",
|
||||
"type": "cli-command"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
Reference in new issue
Block a user