supabase/apps/docs/content/guides/database/replication/etl-replication-setup.mdx

---
id: 'etl-replication-setup'
title: 'ETL Replication Setup'
description: 'Set up ETL replication to automatically sync your database to external destinations.'
subtitle: 'Configure publications and destinations for ETL replication.'
sidebar_label: 'Setting up'
---

<Admonition type="caution" label="Private Alpha">

ETL Replication is currently in private alpha. Access is limited and features may change.

</Admonition>

ETL Replication requires two main components: a **publication** (source) and a **destination**. Follow these steps to set up your replication pipeline.

<Admonition type="tip">

If you already have a publication set up, you can skip to [Step 2: Enable ETL Replication](#step-2-enable-etl-replication).

</Admonition>

### Step 1: Create a publication

A publication defines which tables and change types will be replicated. You need to create a publication using SQL.

#### Creating a publication

The following SQL examples assume you have `users` and `orders` tables in your database.

##### Publication for specific tables

```sql
-- Create publication for both tables
create publication pub_users_orders
for table users, orders;
```

This publication will track all changes (INSERT, UPDATE, DELETE) for both the `users` and `orders` tables.

##### Publication for all tables in a schema

```sql
-- Create a publication for all tables in the public schema
create publication pub_all_public for tables in schema public;
```

This will track changes for all existing and future tables in the `public` schema.

##### Publication for all tables

```sql
-- Create a publication for all tables
create publication pub_all_tables for all tables;
```

This will track changes for all tables in your database.

#### Advanced publication options

##### Selecting specific columns

You can replicate only a subset of columns from a table:

```sql
-- Replicate only specific columns from the users table
create publication pub_users_subset
for table users (id, email, created_at);
```

This will only replicate the `id`, `email`, and `created_at` columns from the `users` table.

##### Filtering rows with a predicate

You can filter which rows to replicate using a WHERE clause:

```sql
-- Only replicate active users
create publication pub_active_users
for table users where (status = 'active');

-- Only replicate recent orders
create publication pub_recent_orders
for table orders where (created_at > '2024-01-01');
```

#### Viewing publications in the Dashboard

After creating a publication via SQL, you can view it in the Supabase Dashboard:

1. Navigate to **Database** → [Publications](/dashboard/project/_/database/publications) in your Supabase Dashboard
2. You'll see all your publications listed with their tables

### Step 2: Enable ETL replication

Before adding destinations, you need to enable ETL replication for your project:

1. Navigate to the [Database](/dashboard/project/_/database/etl) section in your Supabase Dashboard
2. Select the **ETL Replication** tab
3. Click **Enable replication** to activate ETL replication for your project

<Image
  alt="Enable ETL Replication"
  src="/docs/img/database/replication/etl-enable-replication.png"
  zoomable
/>

### Step 3: Add a destination

Once replication is enabled and you have a publication, you can add a destination. The destination is where your replicated data will be stored, while the pipeline is the active process that continuously replicates changes from your database to that destination.

#### Available destinations

For a complete list of available destinations and how to choose the right one for your needs, see [ETL Destinations](/docs/guides/database/replication/etl-destinations).

#### Configuration

1. In the ETL Replication tab, click **Add destination**
2. Configure the destination settings:

   **General Settings:**

   - **Destination name**: A name to identify this destination (e.g., "Analytics Warehouse")
   - **Publication**: The publication to replicate data from (created in Step 1)
   - **Destination type**: Choose from available destination types

   **Destination-specific settings:**
   Each destination type requires different configuration. See the [ETL Destinations guide](/docs/guides/database/replication/etl-destinations) for configuration details specific to your chosen destination.

<Image
  alt="Add ETL Destination"
  src="/docs/img/database/replication/etl-add-destination.png"
  zoomable
/>

3. Configure **Advanced Settings** (optional):

   - **Batch wait time (milliseconds)**: How long to wait for more changes before sending a batch. We recommend leaving this at the default value for optimal performance. Setting this too low can result in too much traffic and less efficient batching.

4. Click **Create and start** to begin replication

### Step 4: Monitor your pipeline

After creating a destination, the pipeline will start and appear in the destinations list. You can monitor the pipeline's status and performance from the Dashboard.

<Image
  alt="ETL Destinations List"
  src="/docs/img/database/replication/etl-destinations-list.png"
  zoomable
/>

For comprehensive monitoring instructions including pipeline states, metrics, and logs, see the [ETL Replication Monitoring guide](/docs/guides/database/replication/etl-replication-monitoring).

### Managing your pipeline

You can manage your pipeline from the destinations list using the actions menu.

<Image
  alt="Pipeline Actions"
  src="/docs/img/database/replication/etl-pipeline-actions.png"
  zoomable
/>

Available actions:

- **Start**: Begin replication for a stopped pipeline
- **Stop**: Pause replication (changes will queue up in the WAL)
- **Restart**: Stop and start the pipeline (required after publication changes)
- **Edit destination**: Modify destination settings like credentials or advanced options
- **Delete**: Remove the destination and permanently stop replication

### Adding or removing tables

If you need to modify which tables are replicated after your ETL pipeline is already running, follow these steps:

<Admonition type="note">

If your publication uses `FOR ALL TABLES` or `FOR TABLES IN SCHEMA`, new tables in that scope are automatically included in the publication. However, you still **must restart the ETL pipeline** for the changes to take effect.

</Admonition>

#### Adding tables to replication

1. Add the table to your publication using SQL:

   ```sql
   -- Add a single table to an existing publication
   alter publication pub_users_orders add table products;

   -- Or add multiple tables at once
   alter publication pub_users_orders add table products, categories;
   ```

2. **Restart the ETL pipeline** using the actions menu (see [Managing your pipeline](#managing-your-pipeline)) for the changes to take effect.

#### Removing tables from replication

1. Remove the table from your publication using SQL:

   ```sql
   -- Remove a single table from a publication
   alter publication pub_users_orders drop table orders;

   -- Or remove multiple tables at once
   alter publication pub_users_orders drop table orders, products;
   ```

2. **Restart the ETL pipeline** using the actions menu (see [Managing your pipeline](#managing-your-pipeline)) for the changes to take effect.

### How it works

Once configured, ETL Replication:

1. **Captures** changes from your database using the publication
2. **Loads** the data to your destination in near real-time batches

Changes are sent in batches to optimize performance and reduce costs. The batch size and timing can be adjusted using the advanced settings.

<Admonition type="note">

ETL Replication currently performs data extraction and loading only, without transformation. Your data is replicated as-is to the destination.

</Admonition>

### Troubleshooting

If you encounter issues during setup:

- **Publication not appearing**: Ensure you created the publication via SQL and refresh the dashboard
- **Tables not showing in publication**: Verify your tables have primary keys (required for replication)
- **Pipeline failed to start**: Check the error message in the status view for specific details
- **No data being replicated**: Verify your publication includes the correct tables and event types

For more troubleshooting help, see the [ETL Replication FAQ](/docs/guides/database/replication/etl-replication-faq).

### Limitations

ETL Replication has the following limitations that apply to all destinations:

- **Primary keys required**: Tables must have primary keys
- **Custom data types**: Not supported
- **Schema changes**: Not automatically handled
- **No data transformation**: Data is replicated as-is without transformation
- **Data duplicates**: Duplicates can occur when stopping a pipeline if your database has transactions that take longer than a few minutes to complete. See [Can data duplicates occur?](/docs/guides/database/replication/etl-replication-faq#can-data-duplicates-occur-during-pipeline-operations) for details

Destination-specific limitations may also apply. See the [Iceberg](/docs/guides/database/replication/etl-iceberg#limitations) destination page for details.

### Future work

ETL Replication is actively being developed. Planned improvements include:

- **DDL support**: Automatic handling of schema changes (ALTER TABLE, ADD COLUMN, etc.)
- **Additional destinations**: Support for more data warehouses and analytics platforms

We don't have public timelines for these features, but they represent our roadmap for making ETL Replication more robust and flexible.

### Next steps

- [Monitor ETL Replication](/docs/guides/database/replication/etl-replication-monitoring)
- [View ETL Replication FAQ](/docs/guides/database/replication/etl-replication-faq)