diff --git a/apps/docs/content/guides/database/replication/etl.mdx b/apps/docs/content/guides/database/replication/etl.mdx index 52989e8a4eb..177045cb1f5 100644 --- a/apps/docs/content/guides/database/replication/etl.mdx +++ b/apps/docs/content/guides/database/replication/etl.mdx @@ -3,7 +3,7 @@ title: ETL description: Replicate your Supabase database to external destinations --- -`ETL` simplifies the process of syncing your data to an external destination. This could be another database, a data warehouse, or a specialized service such as customer management or business analytics software. +ETL simplifies the process of syncing your data to an external destination. This could be another database, a data warehouse, or a specialized service such as customer management or business analytics software. @@ -53,21 +53,26 @@ Before setting up replication to BigQuery, you need: 1. Navigate to the [Database](https://supabase.com/dashboard/project/_/database/tables) section in your Supabase Dashboard and select the **Replication** tab. 2. Click **Add destination** to open the New Destination panel. -3. Click **Enable**. -4. Configure the replication settings: +3. Configure the replication settings: - **Name**: Enter a descriptive name for your replication destination - **Publication**: Select an existing publication or create a new one - **Type**: Select "BigQuery" from the destination types - **Project Id**: Enter the GCP project ID - **Project's Dataset Id**: Enter the ID of the BigQuery dataset - **Service Account Key**: Enter your GCP service account key -5. Review the [Advanced Settings](#advanced-settings) to change the limits for the replication. +4. Review the [Advanced Settings](#advanced-settings) to change the limits for the replication. -- **Max size**: Changes are sent in batches. Max size limits the size of an individual batch. -- **Max fill seconds**: Number of seconds after which a batch should be sent, even if it hasn't reached the max size. -- **Max staleness**: See BigQuery's [`max_staleness`](https://cloud.google.com/bigquery/docs/change-data-capture#manage_table_staleness) option + - **Max size**: Changes are sent in batches. Max size limits the size of an individual batch. + - **Max fill seconds**: Number of seconds after which a batch should be sent, even if it hasn't reached the max size. + - **Max staleness**: See BigQuery's [`max_staleness`](https://cloud.google.com/bigquery/docs/change-data-capture#manage_table_staleness) option -6. Click **Create** to start the replication process. +5. Click **Create** to start the replication process. + + + +Due to ingestion latency in BigQuery, there may be a delay in the dataflow. + + ## Monitoring replication diff --git a/apps/docs/content/guides/database/replication/manual-replication.mdx b/apps/docs/content/guides/database/replication/manual-replication.mdx index ec900df9288..8c2415901a9 100644 --- a/apps/docs/content/guides/database/replication/manual-replication.mdx +++ b/apps/docs/content/guides/database/replication/manual-replication.mdx @@ -3,7 +3,7 @@ title: Manual Replication subtitle: Set up your own replication pipeline for more control --- -ETL enables you to build replication pipelines from Postgres databases to various analytics-focused destinations such as BigQuery, DuckDB, Iceberg, and other specialized systems optimized for OLAP workloads, unlocking fast, scalable analysis with minimal setup and operational overhead. +Postgres comes with built-in support for replication. If `pg_replicate` doesn't provide a destination that you need, or if you need more control over the replication process, you can set up your own replication pipeline using Postgres primitives. ## Replication and publications