Commit Graph
11 Commits
Author SHA1 Message Date
edec85d1ca fix(pipelines): Make pipeline actions and status updates reliable (#50085)
## Summary

Make pipeline actions and status feedback reliable while requests are
running or fail. Let the backend coordinate table resets and restarts,
keep stopped pipelines stopped after resets or settings changes, and
refresh the UI from confirmed backend state.

## Pipeline actions and recovery

- Reset one table, all errored tables, or all tables through the
rollback endpoint without separate frontend stop/start requests. Explain
which destination data is deleted, which rows are copied again, initial
sync charges, and the skip-initial-sync setting.
- Keep pending feedback until the action and a fresh status read finish,
including across navigation and polling errors. Prevent overlapping
actions and disable start/stop controls when status is unavailable or
transitioning.
- Close the creation form once the pipeline is created. If its initial
start fails, users can retry Start on the existing pipeline without
creating a duplicate.
- Wait for confirmed shutdown before deletion; a shutdown error or
timeout leaves deletion retryable. Keep failed version updates open and
avoid reporting success.
- Clarify recovery guidance and pending labels, suppress duplicate error
toasts, and hide stale table errors during transitions.

## Status updates and shared UI

- Poll pipeline status and table metrics one second after each response,
share in-flight reads, pause dashboard polling in background tabs, and
respect rate-limit backoff. The shutdown waiter continues in the
background.
- Refresh metadata after mutations even when an older read is in flight,
while preserving shared polling requests. Refresh affected data after
failures that may follow a committed reset or settings change.
- Move pending request state into the shared, project-keyed
`DatabaseLayout` so the list, detail page, and diagram stay consistent.
The surrounding database-page changes update named imports in both
Next.js and TanStack routes.
- Simplify action, status, and form rendering; announce status changes
to assistive technology; and sort table statuses without mutating cached
data.

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: Danny White <3104761+dnywh@users.noreply.github.com>
2026-09-18 11:32:48 +08:00
Danny White 2e435986c9 feat(studio): polish the Replication pipelines list (#50252)
## What kind of change does this PR introduce?

Studio UI and interaction polish. This is the second PR in the Pipelines
review stack and depends on #50251.

## What is the current behaviour?

Pipeline rows require a separate view action, cannot be sorted, and
present lifecycle, lag, and destination terminology inconsistently.

## What is the new behaviour?

Makes rows navigable with link-like mouse and keyboard behaviour, adds
Name and Status sorting, reuses cached status queries, and moves row
actions into the overflow menu. It also clarifies pipeline terminology,
adds Docs and feedback actions, and standardises state, error, lag, and
loading presentation with accessible announcements.

| Before | After |
| --- | --- |
| <img width="1280" height="1323" alt="Replication Database ETL BigTable
ETL Team Supabase"
src="https://github.com/user-attachments/assets/53a62283-0e58-4408-8409-2b87a38af159"
/> | <img width="1280" height="1323" alt="Replication Database Agua
Basket Supabase"
src="https://github.com/user-attachments/assets/ba4de1e0-4a84-43b4-8975-ca05a3056bcf"
/> |

## To test

1. Open `/project/<ref>/database/replication`.
2. Sort by Name and Status, then confirm failed and stopped pipelines
surface first when Status is ascending.
3. Click a row, use Enter or Space, and modifier-click or middle-click
to verify link behaviour.
4. Open the row overflow menu and confirm it does not navigate.
5. Check loading, initial sync, caught up, numeric lag, unavailable lag,
and table-error states where available.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Added sortable pipeline lists with clearer loading, empty, and error
states.
- Pipeline rows now support direct navigation, detail viewing, status
indicators, lag progress, and table error summaries.
- Added initial-sync progress indicators and accessible status
announcements.
  - Added documentation and feedback links.
  - Improved pipeline version update and enable/disable dialogs.

- **Bug Fixes**
  - Prevented right-clicks from triggering navigation.
  - Improved unavailable lag and initial-sync handling.

- **Style**
- Standardized replication terminology and confirmation messaging around
pipelines.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-15 10:07:49 +10:00
Danny WhiteandJoshen Lim 8dd97d75aa refactor(studio): use v2 replication publication APIs (#49844)
## What kind of change does this PR introduce?

Studio data-layer migration.

## What is the current behavior?

Studio loads complete publication details through the original bulk
endpoint and creates publications by executing SQL against the source
database. Publication and source-table data use names where stable table
IDs are available.

## What is the new behavior?

Uses the v2 publication-name, publication-detail, publication mutation,
and source-table endpoints. The existing creation sheet continues to
behave the same, including publishing partition changes through the
parent table by default. Initial-sync selection and Analytics Bucket
associations now consume the selected publication detail. Generated
platform API types and their required nullability updates are included.

The generated Platform contract accounts for roughly 10,000 changed
lines in this PR.

## Dependency

Depends on the v2 source table, table column, and publication endpoints
from
[supabase/platform#37505](https://github.com/supabase/platform/pull/37505),
which are deployed to production.

## To test

1. Open the pipeline creation sheet and select an existing publication.
2. Create a publication with mixed-case schema and table names, then
confirm the table names are shown while stable IDs are submitted.
3. Exercise all four initial-sync policies, including selecting
individual tables.
4. Reopen the publication and table selectors and confirm they refresh
without replacing populated options.
5. Edit and delete a publication.
6. Open an Analytics Bucket associated with a pipeline and confirm its
publication tables resolve correctly.
7. Confirm unlimited WAL retention renders as Unlimited on pipeline
status.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Improved replication publication setup with on-demand table loading,
refresh controls, clearer table labels, and streamlined publication
selection.
- Publication creation and updates now use the latest replication API
and table-based configurations.
- Added clearer handling for tables removed from publications, including
stale-selection warnings.

- **Bug Fixes**
- Prevented table selections from carrying over when switching
publications.
- Improved replication status displays when lag or WAL metrics are
unavailable.
- Updated replication deletion and table management for the latest API
behavior.

- **Tests**
- Expanded coverage for publication creation, table selection, stale
tables, loading states, and replication metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2026-09-08 11:37:59 +10:00
Charis 3b7052b5a9 cleanup: fix import order and prefixes for studio/data (#44501) 2026-04-03 09:15:57 +02:00
Joshen Lim 1536ee1064 Joshen/fe 2287 summary nodegraph in replication page (#42042)
* init

* Init 2

* Finalize

* bit more details in the summary graph

* Moreeee details

* Clean

* 🧹

* FIIIXX

* Fix rabbit

* Simplify long polling logic to only depend on database for destinations UI

* ADdress rabbit

* Address feedback
2026-01-23 17:59:52 +08:00
Riccardo Busetti 1a3ba9f15a ref(etl): Rename ETL Replication to Replication and update docs (#40769)
* ref(etl): Rename ETL Replication to Replication and update docs

* Fix

* Fix

* Fix

* Fix

* Fix
2025-11-25 20:45:50 +08:00
Riccardo BusettiandJoshen Lim 91464b8fe8 ref(etl): Rename Replication to ETL Replication (#40202)
* ref(etl): Rename Replication to ETL Replication

* Fix

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-11-07 12:02:40 +08:00
Ivan Vasilov 8b657165b5 chore: Migrate to use custom type for ReactQuery queries and mutations (#40073)
* Add custom types for queries, mutations and infinite queries.

* Migrate all queries to use the new type.

* Migrate all infinite queries to useCustomInfiniteQueryOptions.

* Migrate all mutations to use useCustomMutationOptions.

* Add type to all imports in `types` folder.
2025-11-03 13:18:13 +01:00
Joshen Lim 64e3e047eb Final final cleaning up barrel files (#40018)
* Final final cleaning up barrel files

* Fix merge conflict
2025-10-31 14:02:59 +08:00
Alaister YoungandJoshen Lim 8855d05803 chore(studio): swap react-query to object syntax (#39842)
* chore(studio): swap react-query to object syntax

* Fix small issues found

* Fix realtime settings

* Nit

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-10-27 09:38:27 +01:00
Raminder SinghandSaxon Fletcher 23ceb9072d Add replication UI in Studio (#30090)
* add dummy sinks and pipelines pages

* update api types

* show empty sources state

* show empty replication state at /replication

* create source when enable replication button is clicked

* improve replication page when replication is enabled

* replace sources page with publications page

* publications table

* show publications in table

* create publication wip

* show toast error instead of throwing an exception

* user can now delete a publication

* show empty sinks page

* create and list sinks

* add ui to delete a sink

* show pipelines on the pipelines page

* add ui to create and delete pipelines

* get pipeline status wip

* show pipeline status wip

* show correct label on action buttons

* start and stop pipelines

* remove a couple of console.logs

* fix error when deleting a pipeline

* only consider replication enabled when a source with name = ref is present

* add source and sink names

* correct colspan for 'no pipelines' row

* hide 'supabase_realtime' publication on ui

* move filtering to fetch query

* show sink name in ui

* show source/sink names on pipelines page

* fix start/stop status shown on ui

* fix prettier formatting

* update api types

* extract pipeline action button as a separate component

* fix a crashing page

* fixed publications page crash

* update to match with changes in api

* add new replication page under database

* hide replication page behind feature flag

* update types

* update api types

* show destinations empty state

* add destinations table

* factor out components from Destinations table

* show status dot

* move pipeline fetch query to parent component

* add ability to enable/disable a pipeline

* show loader when starting or stopping a pipeline

* fix a bug in which loading & empty states were shown together

* fix a bug in which error & empty states were shown together

* wrap in default layout

* add new destination panel

* add type field

* fix a forwardRef error

* fix layout

* create destination

* delete destination

* add ability to create or delete destinations with pipelines

* create source if missing

* show only a single error

* add an enable switch

* new layout

* add subsections

* comment out unused code

* show enabled switch only in the header

* close panel when destination is created

* disable buttons when api requests in flight

* reduce panel size

* remove commented out code

* treat max size and max fill secs as numbers

* use drop down to show publications

* simpler vertical layout

* add separators

* add form validation

* remove publications drop down padding

* add new publication button

* hide advanced settings behind an accordion

* add some margin between icon and text

* show publications panel on clicking new publication button

* add header to new publication panel

* fix validation not running for publication drop down

* create publication in the new publication panel

* add table selector in new publication panel

* update api types

* remove old code

* update platform.d.ts

* update navigation bar utils

* remove a redirect from replication page to publications page

* ask user for confirmation before deleting destination

* edit destination panel

* edit destination panel values fixed

* bug fixes

* fix prettier formatting

* enable/disable pipeline after editing

* rename snake_case params to camelCase

* loading button when editing

* remove merge markers

* update api types

* add max_staleness parameter in sinks for bigquery

* add read replicas flow diagram to replication page

* remove an unused import

* Revert "add read replicas flow diagram to replication page"

This reverts commit 8852d7847b457885603dba786141a8aaf8e99350.

* add panel to warn users about additional cost before creating a destination

* hide replication page contents behind a feature flag

* fix merge conflicts

* styling changes

* revert static flag

* styling updates

* fixes

* fix switch

* copy

* fix layout

---------

Co-authored-by: Saxon Fletcher <saxonafletcher@gmail.com>
2025-04-23 10:52:46 +05:30