mirror of
https://github.com/supabase/supabase.git
synced 2026-10-10 20:05:06 +03:00
Miranda Limonczenko
abbf667084
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## Problem The Docs E2E link checker found broken links throughout docs, starting with `phone-login.mdx` pointing to `/docs/guides/cli/config` (404). Old links like `/docs/guides/cli/config` still work on the live site because `supabase.com` has redirects set up for them, but these links break on the docs preview site, which is what the E2E check tests against. These issues look clean on the live site, and I didn't catch them in my first pass because I was testing production instead of the preview. The E2E check only tests the ~20 pages a given PR happens to touch, so fixing the pages it flagged kept exposing more of the same problem one page at a time as each fix pulled in a new file. To stop chasing this incrementally, I cross-referenced every `/docs/guides/*` and `/docs/reference/*` redirect source in `apps/www/lib/redirects.js` against actual usage across all of `apps/docs`, and verified each candidate against the live preview. ## Solution Rather than updating the Docs E2E link checker, this PR resolves the links. **Why:** we own these docs, so keeping the links clean without redirects is keeping the house maintained. See [Broken Window Theory](https://blog.codinghorror.com/the-broken-window-theory/). Updated every link still using an old path to point straight at the current page instead of relying on a redirect. This covers old links like: - `/docs/guides/cli/config` → `/docs/guides/local-development/cli/config` - `/docs/guides/cli/getting-started` → `/docs/guides/local-development/cli/getting-started` - `/docs/guides/cli/local-development` → `/docs/guides/local-development/database-migrations` - `/docs/guides/cli/managing-environments` → `/docs/guides/deployment/managing-environments` - `/docs/guides/cli/seeding-your-database` → `/docs/guides/local-development/seeding-your-database` - bare `/docs/guides/cli` → `/docs/guides/local-development` - `/docs/guides/platform/compute-add-ons` → `/docs/guides/platform/compute-and-disk` - `/docs/guides/platform/shared-responsibility-model` → `/docs/guides/deployment/shared-responsibility-model` - `/docs/guides/database` → `/docs/guides/database/overview` - `/docs/reference/javascript`, `/docs/reference/dart`, `/docs/reference/kotlin`, `/docs/reference/python`, `/docs/reference/csharp` → their `/introduction` pages (the redirect's own destination, `/start`, turned out to be dead even on production — a separate bug in `redirects.js` I didn't touch here) - and about 35 more of the same pattern, listed in the commit messages Also fixed a handful of dead heading anchors found along the way (links that resolve to the right page but point at a `#section` that got renamed or moved), including the original `#bigquery` anchor and a few in `connecting-to-postgres.mdx` where content moved to its own dedicated page. Left alone on purpose: - `content/guides/cli.mdx` — this page has no route in the docs app at all (no `app/guides/cli/` directory), so it 404s even in production before the `www` redirect ever fires. Fixing its internal link wouldn't change that; it needs an actual routing/content decision, not a link fix. - A few candidates that already resolve fine as-is (`pg_partman`, bare `/docs/reference/api`, bare `/docs/reference/cli`) — confirmed via curl, left untouched. ## Manual testing 1. Confirmed every new link target actually exists by checking the destination file/page and matching heading anchors. 2. Cross-referenced every `/docs/guides/*` and `/docs/reference/*` redirect source in `apps/www/lib/redirects.js` against real usage in `apps/docs`, and curl-verified each old path (404) and new path (200) against the live PR preview before fixing it. 3. Ran the Docs E2E link checker locally against changed pages. 4. Spot-checked the original broken link from CI (`/docs/guides/cli/config`) to confirm it now points to a working page.
107 lines
4.3 KiB
Plaintext
107 lines
4.3 KiB
Plaintext
---
|
|
id: 'manual-replication-setup'
|
|
title: 'Set up manual replication'
|
|
description: 'Configure your own replication using external tools and Postgres logical replication.'
|
|
subtitle: 'Set up replication with Airbyte, Estuary, Fivetran, and other tools.'
|
|
sidebar_label: 'Setting up'
|
|
---
|
|
|
|
This guide covers setting up **manual logical replication** using your own tools. If you prefer a managed solution, read [Set up Pipelines](/docs/guides/database/replication/pipelines) instead.
|
|
|
|
<Admonition type="note">
|
|
|
|
This guide is for replicating data to destination systems using your own tools. For deploying read-only databases across multiple regions, see [read replicas](/docs/guides/platform/read-replicas) instead.
|
|
|
|
</Admonition>
|
|
|
|
### Prerequisites
|
|
|
|
To set up replication, the following is recommended:
|
|
|
|
- Instance size of XL or greater
|
|
- [IPv4 add-on](/docs/guides/platform/ipv4-address) enabled
|
|
|
|
To create a replication slot, you will need to use the `postgres` user and follow the instructions in the [logical replication example](/docs/guides/database/postgres/setup-replication-external).
|
|
|
|
<Admonition type="note">
|
|
|
|
If you are running Postgres 17 or higher, you can create a new user and grant them replication permissions with the `postgres` user. For versions below 17, you will need to use the `postgres` user.
|
|
|
|
</Admonition>
|
|
|
|
If you are replicating to a destination system and using any of the tools below, check their documentation first. Additional information is provided where the setup with Supabase can vary.
|
|
|
|
<Tabs
|
|
scrollable
|
|
size="small"
|
|
type="underlined"
|
|
defaultActiveId="estuary"
|
|
queryGroup="tool"
|
|
>
|
|
|
|
<TabPanel id="airbyte" label="Airbyte">
|
|
|
|
Airbyte has the following [documentation](https://docs.airbyte.com/integrations/sources/postgres/) for setting up Postgres as a source, either in their cloud offering or by self-hosting.
|
|
|
|
You can follow those steps with the following modifications:
|
|
|
|
1. Use the `postgres` user
|
|
2. Select `logical replication` as the replication method (`xmin` is possible, but not recommended)
|
|
|
|
</TabPanel>
|
|
|
|
<TabPanel id="estuary" label="Estuary">
|
|
Estuary has the following [documentation](https://docs.estuary.dev/reference/Connectors/capture-connectors/PostgreSQL/Supabase/) for setting up Postgres as a source.
|
|
|
|
</TabPanel>
|
|
|
|
<TabPanel id="fivetran" label="Fivetran">
|
|
|
|
Fivetran has the following [documentation](https://fivetran.com/docs/connectors/databases/postgresql/setup-guide) for setting up Postgres as a source.
|
|
|
|
You can follow those steps with the following modifications:
|
|
|
|
1. In Step 2, choose `logical replication` as the sync mechanism
|
|
2. In Step 3, do not create a user and use the existing `postgres` user for replication
|
|
3. In Step 5, no need to modify any WAL settings as this has been configured
|
|
|
|
</TabPanel>
|
|
|
|
<TabPanel id="materialize" label="Materialize">
|
|
|
|
Materialize has the following [documentation](https://materialize.com/docs/sql/create-source/postgres/) on setting up Postgres as a source.
|
|
|
|
You can follow those steps with the following modifications:
|
|
|
|
1. Follow the steps in the [logical replication example](/docs/guides/database/postgres/setup-replication-external) to create a publication slot
|
|
|
|
</TabPanel>
|
|
|
|
<TabPanel id="stitch" label="Stitch">
|
|
|
|
Stitch has the following [documentation](https://www.stitchdata.com/docs/integrations/databases/postgresql/v2#extract-data) on configuring Postgres as a source.
|
|
|
|
You can follow those steps with the following modifications:
|
|
|
|
1. Use the `postgres` user for replication
|
|
2. Skip step 3
|
|
|
|
</TabPanel>
|
|
|
|
<TabPanel id="dms" label="AWS DMS">
|
|
|
|
AWS DMS has the following [documentation](https://docs.aws.amazon.com/dms/latest/userguide/Welcome.html) on configuring Postgres as a source.
|
|
|
|
DMS is notably useful if you have infrastructure running in AWS and/or you have custom networking. An additional benefit is that DMS is able to replicate schema changes.
|
|
|
|
You can follow those steps with the following modifications:
|
|
|
|
1. Use the `postgres` user for replication (or create a new user with replication permissions: `ALTER USER <user> WITH REPLICATION;`)
|
|
2. Set `pluginname` to `test-decoding`
|
|
3. Depending on your databases the pre-assessment may fail. It is not required.
|
|
4. Increase `wal_sender_timeout` and other settings (`max_wal_size`) using the [Supabase CLI](/docs/guides/database/custom-postgres-config)
|
|
|
|
</TabPanel>
|
|
|
|
</Tabs>
|