## Summary
Make pipeline actions and status feedback reliable while requests are
running or fail. Let the backend coordinate table resets and restarts,
keep stopped pipelines stopped after resets or settings changes, and
refresh the UI from confirmed backend state.
## Pipeline actions and recovery
- Reset one table, all errored tables, or all tables through the
rollback endpoint without separate frontend stop/start requests. Explain
which destination data is deleted, which rows are copied again, initial
sync charges, and the skip-initial-sync setting.
- Keep pending feedback until the action and a fresh status read finish,
including across navigation and polling errors. Prevent overlapping
actions and disable start/stop controls when status is unavailable or
transitioning.
- Close the creation form once the pipeline is created. If its initial
start fails, users can retry Start on the existing pipeline without
creating a duplicate.
- Wait for confirmed shutdown before deletion; a shutdown error or
timeout leaves deletion retryable. Keep failed version updates open and
avoid reporting success.
- Clarify recovery guidance and pending labels, suppress duplicate error
toasts, and hide stale table errors during transitions.
## Status updates and shared UI
- Poll pipeline status and table metrics one second after each response,
share in-flight reads, pause dashboard polling in background tabs, and
respect rate-limit backoff. The shutdown waiter continues in the
background.
- Refresh metadata after mutations even when an older read is in flight,
while preserving shared polling requests. Refresh affected data after
failures that may follow a committed reset or settings change.
- Move pending request state into the shared, project-keyed
`DatabaseLayout` so the list, detail page, and diagram stay consistent.
The surrounding database-page changes update named imports in both
Next.js and TanStack routes.
- Simplify action, status, and form rendering; announce status changes
to assistive technology; and sort table statuses without mutating cached
data.
---------
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: Danny White <3104761+dnywh@users.noreply.github.com>
## What kind of change does this PR introduce?
Feature. The last remaining piece of this PR's original scope, rebased
onto current `master`.
## What is the current behavior?
A BigQuery pipeline replicates every published table into a flat
destination table. There is no way to say how those tables should be
laid out in BigQuery, so partitioning and clustering have to be applied
by hand after the fact, and are lost whenever a destination table is
reset.
The rest of this PR's original scope has since merged separately:
#49841, #49842, #49843, #49844 and #49845. The branch now carries only
the table layout work, and adds nothing to `packages/api-types`. It
consumes the `table_options` contract that #49844 already brought in.
## What is the new behavior?
The BigQuery destination form gains a "Table layout" section under
Advanced settings. Every table in the selected publication appears as a
row, and expanding one reveals optional partitioning and clustering:
- Time-column partitioning, by hour, day, month or year
- Integer-range partitioning, with start, end and interval
- Ingestion-time partitioning
- Clustering, up to four columns
Rows are always present, so nothing implies a table can be excluded
here. A collapsed row summarises what is applied: `Not configured`,
`Daily by created_at`, `Integer range by id`, plus clustering counts.
`Not configured` is dimmed a step further so configured rows are what
the eye lands on in a long publication.
Notes on behavior:
- Partition columns are resolved from the published table's real
columns, filtered to the types BigQuery accepts for each partition mode.
- A row you expand but leave empty is dropped from the payload rather
than saved half-configured. Choosing a partition mode without a column
is a validation error, not a silent drop.
- Clear returns a row to `Not configured` and keeps it in the list.
Remove is reserved for stale configuration whose table has left the
publication.
- Updates send `null` to clear previously stored table options, since
omitting the property leaves the stored value unchanged.
- Layout applies when a destination table is first created or reset,
matching the backend.
Roughly 40% of the diff is tests. The bulk of the rest is four new files
under `DestinationForm/BigQuery`, which are all one feature. The three
commits are readable in order: types and payload builders, then the
columns query, then the UI.
## To test
Open a project's Database > Replication, then create or edit a BigQuery
pipeline.
1. Expand Advanced settings. "Table layout" lists every table in the
selected publication.
2. Expand a row, set Partition by to Time column and pick a column.
Collapse. The row reads `Daily by <column>`. Hit Clear. The row returns
to `Not configured` and stays in the list.
3. Set Partition by to Time column, leave Partition column empty,
collapse, and Save. The row explains "Select a partition column" in red
rather than saving and silently dropping it.
4. Expand a row and add clustering columns. The fifth is refused.
5. Switch publications and watch the loading state. The row list should
not jump size when it resolves.
6. Narrow the sheet. The integer range Start, End and Interval fields
should reflow rather than stay in three columns.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* BigQuery replication destinations now support per-table layout
settings.
* Configure partitioning by time column, integer range, or ingestion
time.
* Configure up to four clustering columns per table.
* View available columns and validation feedback while editing table
layouts.
* Table settings are preserved when editing destinations and cleaned up
when publications change.
* **Bug Fixes**
* Improved handling of invalid, unavailable, or removed table and column
configurations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Danny White <3104761+dnywh@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
## What kind of change does this PR introduce?
Studio data-layer refactor.
## What is the current behavior?
Pipeline creation, editing, and validation build similar destination and
pipeline payloads separately. The duplicated mappings rely on type
assertions and can drift between actions.
## What is the new behavior?
Uses shared typed builders for create, update, and validation payloads
across the existing destinations. Update payloads continue to omit blank
secrets, while create payloads preserve their current values. This PR
does not add table partitioning configuration.
## To test
This is a data-layer refactor. No visible behaviour should change.
1. Open **Database > Replication** and click **Start a new pipeline**.
2. Select **BigQuery**, or any other enabled destination.
3. Edit a few non-secret fields and expand **Advanced settings**.
4. Confirm the form remains usable and no runtime errors appear.
Create, update, validation, and secret-handling behaviour is covered by
the focused tests and CI. Deploy previews and fresh local projects do
not have the existing destinations or credentials needed to exercise
those paths manually.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved replication destination configuration handling during
creation, updates, and validation.
* Applied consistent configuration mapping across supported destination
types.
* Ensured blank secret values are omitted during updates while retained
when creating destinations.
* Standardized table synchronization defaults when no specific setting
is provided.
* **Tests**
* Added coverage for BigQuery configuration mapping and secret handling.
* Updated DuckLake tests for destination updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
## Summary
Adds initial-copy scoping to Pipelines in Studio. Users can copy all
existing rows, skip all initial copies, copy only selected publication
tables, or skip selected table copies. All publication tables continue
streaming new changes regardless of the initial-copy policy.
The policy now round-trips through create, edit, validation, and the
generated Management API contract. Initial-copy estimates and
table-restart confirmations use the same scope. Edit requests also
preserve redacted credentials and pipeline settings that Studio does not
own.
This completes the Studio layer of the [ETL API
change](https://github.com/supabase/etl/pull/897) and [Management API
change](https://github.com/supabase/platform/pull/35479).
## Screenshots
### Selector
<img width="1153" height="465" alt="image"
src="https://github.com/user-attachments/assets/bf615e82-ee61-4222-979d-a8695a957e82"
/>
### Select certain tables only
<img width="1153" height="465" alt="image"
src="https://github.com/user-attachments/assets/28adaa24-f239-4d1d-8fb8-fdb1988320cd"
/>
### Confirm copy costs
As the final step before the pipeline is created:
<img width="597" height="619" alt="image"
src="https://github.com/user-attachments/assets/a660bd87-bfb8-41c5-8099-4cdbdef943bf"
/>
### Policy-aware initial-copy estimate
#### Copy no table is selected
<img width="407" height="464" alt="image"
src="https://github.com/user-attachments/assets/99d859ec-2ec3-452a-ab69-11924a8db260"
/>
#### Some tables are selected
<img width="407" height="464" alt="image"
src="https://github.com/user-attachments/assets/e68aedf4-66bc-4372-98ef-0dd7fecef324"
/>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Added configurable “initial table copy” policies (copy/skip all and
copy/skip selected) during replication setup, including table-picker
behavior, pruning of stale selections, and updated restart/cost
estimates.
- **Bug Fixes**
- Improved restart flows to consistently use `schema.table` identity and
simplified “errored tables” targeting to match error-state tables.
- Reduced unnecessary loading by gating publication/table fetches to
when panels are visible; improved validation/toast handling when
publication tables are unavailable.
- **Tests**
- Added/expanded coverage for destination form submission, table-copy
selection, restart/cost dialogs, and copy-estimate summarization.
- **Style**
- Refreshed warning/label text for clearer configuration and
confirmation messaging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Adds ClickHouse as a replication destination type in Studio.
- New ClickHouse option in the destination type selector, gated behind
the
`etlEnableClickHousePrivateAlpha` organization feature flag (off by
default).
- ClickHouse settings form: URL, user, password (optional), database,
and
- Client-side URL validation requires HTTPS and rejects URLs targeting
internal addresses (loopback, RFC 1918, link-local, CGNAT, IPv6
loopback/link-local/ULA, and IPv4-mapped/NAT64 forms). Server-side
validation remains authoritative.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## New Features
- Added **ClickHouse** as a replication destination option (private
alpha), including support in destination selection/panel, replication
diagram rendering, and destination icons.
- Introduced a ClickHouse destination form with fields for URL, user,
optional password (masked toggle), database, and engine selection.
- Added ClickHouse destination config handling for create/update flows,
with normalization and engine support.
## Tests
- Expanded unit tests to cover ClickHouse validation and destination
config building/normalization, including HTTPS-only and blocking
localhost/internal targets.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
## What kind of change does this PR introduce?
Add support to use supabase projects as a pg catalog and storage when
adding a ducklake replication.
## What is the current behavior?
Only simple form with raw input text for custom parameters is available.
## What is the new behavior?
Being able to select supabase project to directly use projects in
supabase for the ducklake.
I also fixed a warning we had in the console for this form (cf
screenshot)
## Additional context
[API Changes ](https://github.com/supabase/platform/pull/34282)
https://github.com/user-attachments/assets/4ff9ee65-6ba4-4f17-9ea1-9aebad34171c
<img width="862" height="228" alt="Capture d’écran 2026-06-18 à 09 58
50"
src="https://github.com/user-attachments/assets/1592c3be-807e-426f-9a5a-84979e05d93c"
/>
### Test scenario
Follow the screencast, go to your supabase project (better if it's in
ap-southeast-1)
Create a test table with 1 row for example
-> Database -> Replication -> New destination -> Select ducklake and use
supabase option
-> Keep the same current supabase project selected for both catalog and
storage
-> Create destination -> You'll get a warning about the storage and
credentials
-> Confirm creation
-> Wait until it's in status Running, if it's runing then it works
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added DuckLake replication destination with **Use Supabase** and
**Custom parameters** modes.
* Added DuckLake bucket selection with a **“New bucket”** creation
dialog.
* Added/expanded BigQuery, Analytics Bucket, and Snowflake destination
configuration.
* **Improvements**
* Updated DuckLake create vs edit behavior: mode selection is hidden in
edit mode and configuration is mapped correctly for the selected
variant.
* Enhanced field-level validation (including whitespace-only handling)
and added clearer validation issue messages.
* Added a cross-region warning for DuckLake when catalog and storage
regions differ.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Signed-off-by: Benjamin <5719034+bnjjj@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
## Details of change
Adds Snowflake to the Studio replication destination flow:
- destination selection and display
- create/edit form fields
- validate/create/update payload serialization
- generated Platform API types
Snowflake remains gated behind `etlEnableSnowflakePrivateAlpha`.
**Note:** I have configured `etlEnableSnowflakePrivateAlpha` in
ConfigCat ("all" in staging and tied to my own org id in prod).
## Details of Verification Process
- Studio focused Vitest coverage for form serialization and diagram
mapping
- Studio typecheck
- ESLint on changed Studio replication files
- Local `mise fullstack:dev` smoke test to confirm the Snowflake form
renders ok.
<img width="937" height="569" alt="image"
src="https://github.com/user-attachments/assets/8d6b3a87-1f9d-4a59-91da-be719714ea49"
/>
Full create/validate E2E depends on the Platform PR and ETL runtime
rollout.
## Review Requests
Please check the Snowflake wire payload matches the Platform/ETL
contract and that gating/edit/display behavior follows the existing ETL
destination patterns.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Snowflake added as a supported replication destination (private-alpha
gated), including UI for selecting and configuring connection and auth
(account, user, database, schema, role, private key, optional
passphrase).
* **Validation**
* Form validation and submission now handle Snowflake-specific
required/optional fields.
* **Tests**
* Unit tests added for Snowflake form behavior and replication-type
detection.
* **API**
* Destination create/update/validate flows extended to accept Snowflake
payloads.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Remove useless expire snapshot configuration
> This is an alpha feature not already deployed to any of our customers
so breaking changes are not an issue.
Signed-off-by: Benjamin <5719034+bnjjj@users.noreply.github.com>
* Add custom types for queries, mutations and infinite queries.
* Migrate all queries to use the new type.
* Migrate all infinite queries to useCustomInfiniteQueryOptions.
* Migrate all mutations to use useCustomMutationOptions.
* Add type to all imports in `types` folder.
* Migrate all uses of invalidateQueries to use object syntax.
* Migrate the remainder of useInfiniteQuery.
* Migrate all setQueriesData.
* Migrate all fetchQuery uses.
* Migrate some leftover functions from RQ.
* Fix issues found by Charis.
