mirror of
https://github.com/supabase/supabase.git
synced 2026-10-13 21:29:08 +03:00
Review
This commit is contained in:
2 files changed
+14
-9
No files matched your search
@@ -3,7 +3,7 @@ title: ETL
|
||||
description: Replicate your Supabase database to external destinations
|
||||
---
|
||||
|
||||
`ETL` simplifies the process of syncing your data to an external destination. This could be another database, a data warehouse, or a specialized service such as customer management or business analytics software.
|
||||
ETL simplifies the process of syncing your data to an external destination. This could be another database, a data warehouse, or a specialized service such as customer management or business analytics software.
|
||||
|
||||
<Admonition type="caution">
|
||||
|
||||
@@ -53,21 +53,26 @@ Before setting up replication to BigQuery, you need:
|
||||
|
||||
1. Navigate to the [Database](https://supabase.com/dashboard/project/_/database/tables) section in your Supabase Dashboard and select the **Replication** tab.
|
||||
2. Click **Add destination** to open the New Destination panel.
|
||||
3. Click **Enable**.
|
||||
4. Configure the replication settings:
|
||||
3. Configure the replication settings:
|
||||
- **Name**: Enter a descriptive name for your replication destination
|
||||
- **Publication**: Select an existing publication or create a new one
|
||||
- **Type**: Select "BigQuery" from the destination types
|
||||
- **Project Id**: Enter the GCP project ID
|
||||
- **Project's Dataset Id**: Enter the ID of the BigQuery dataset
|
||||
- **Service Account Key**: Enter your GCP service account key
|
||||
5. Review the [Advanced Settings](#advanced-settings) to change the limits for the replication.
|
||||
4. Review the [Advanced Settings](#advanced-settings) to change the limits for the replication.
|
||||
|
||||
- **Max size**: Changes are sent in batches. Max size limits the size of an individual batch.
|
||||
- **Max fill seconds**: Number of seconds after which a batch should be sent, even if it hasn't reached the max size.
|
||||
- **Max staleness**: See BigQuery's [`max_staleness`](https://cloud.google.com/bigquery/docs/change-data-capture#manage_table_staleness) option
|
||||
- **Max size**: Changes are sent in batches. Max size limits the size of an individual batch.
|
||||
- **Max fill seconds**: Number of seconds after which a batch should be sent, even if it hasn't reached the max size.
|
||||
- **Max staleness**: See BigQuery's [`max_staleness`](https://cloud.google.com/bigquery/docs/change-data-capture#manage_table_staleness) option
|
||||
|
||||
6. Click **Create** to start the replication process.
|
||||
5. Click **Create** to start the replication process.
|
||||
|
||||
<Admonition type="info">
|
||||
|
||||
Due to ingestion latency in BigQuery, there may be a delay in the dataflow.
|
||||
|
||||
</Admonition>
|
||||
|
||||
## Monitoring replication
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ title: Manual Replication
|
||||
subtitle: Set up your own replication pipeline for more control
|
||||
---
|
||||
|
||||
ETL enables you to build replication pipelines from Postgres databases to various analytics-focused destinations such as BigQuery, DuckDB, Iceberg, and other specialized systems optimized for OLAP workloads, unlocking fast, scalable analysis with minimal setup and operational overhead.
|
||||
Postgres comes with built-in support for replication. If `pg_replicate` doesn't provide a destination that you need, or if you need more control over the replication process, you can set up your own replication pipeline using Postgres primitives.
|
||||
|
||||
## Replication and publications
|
||||
|
||||
|
||||
Reference in new issue
Block a user