Commit Graph
9 Commits
Author SHA1 Message Date
edec85d1ca fix(pipelines): Make pipeline actions and status updates reliable (#50085)
## Summary

Make pipeline actions and status feedback reliable while requests are
running or fail. Let the backend coordinate table resets and restarts,
keep stopped pipelines stopped after resets or settings changes, and
refresh the UI from confirmed backend state.

## Pipeline actions and recovery

- Reset one table, all errored tables, or all tables through the
rollback endpoint without separate frontend stop/start requests. Explain
which destination data is deleted, which rows are copied again, initial
sync charges, and the skip-initial-sync setting.
- Keep pending feedback until the action and a fresh status read finish,
including across navigation and polling errors. Prevent overlapping
actions and disable start/stop controls when status is unavailable or
transitioning.
- Close the creation form once the pipeline is created. If its initial
start fails, users can retry Start on the existing pipeline without
creating a duplicate.
- Wait for confirmed shutdown before deletion; a shutdown error or
timeout leaves deletion retryable. Keep failed version updates open and
avoid reporting success.
- Clarify recovery guidance and pending labels, suppress duplicate error
toasts, and hide stale table errors during transitions.

## Status updates and shared UI

- Poll pipeline status and table metrics one second after each response,
share in-flight reads, pause dashboard polling in background tabs, and
respect rate-limit backoff. The shutdown waiter continues in the
background.
- Refresh metadata after mutations even when an older read is in flight,
while preserving shared polling requests. Refresh affected data after
failures that may follow a committed reset or settings change.
- Move pending request state into the shared, project-keyed
`DatabaseLayout` so the list, detail page, and diagram stay consistent.
The surrounding database-page changes update named imports in both
Next.js and TanStack routes.
- Simplify action, status, and form rendering; announce status changes
to assistive technology; and sort table statuses without mutating cached
data.

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: Danny White <3104761+dnywh@users.noreply.github.com>
2026-09-18 11:32:48 +08:00
Riccardo Busetti a23e673993 ref(etl): Improve UX around table resets (#44843) 2026-04-14 16:21:36 +02:00
Charis 3b7052b5a9 cleanup: fix import order and prefixes for studio/data (#44501) 2026-04-03 09:15:57 +02:00
Riccardo Busetti 1a3ba9f15a ref(etl): Rename ETL Replication to Replication and update docs (#40769)
* ref(etl): Rename ETL Replication to Replication and update docs

* Fix

* Fix

* Fix

* Fix

* Fix
2025-11-25 20:45:50 +08:00
Riccardo BusettiandJoshen Lim 91464b8fe8 ref(etl): Rename Replication to ETL Replication (#40202)
* ref(etl): Rename Replication to ETL Replication

* Fix

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-11-07 12:02:40 +08:00
Ivan Vasilov 8b657165b5 chore: Migrate to use custom type for ReactQuery queries and mutations (#40073)
* Add custom types for queries, mutations and infinite queries.

* Migrate all queries to use the new type.

* Migrate all infinite queries to useCustomInfiniteQueryOptions.

* Migrate all mutations to use useCustomMutationOptions.

* Add type to all imports in `types` folder.
2025-11-03 13:18:13 +01:00
Ivan Vasilov da4a40e308 chore: Migrate RQ functions to use object syntax style (#39895)
* Migrate all uses of invalidateQueries to use object syntax.

* Migrate the remainder of useInfiniteQuery.

* Migrate all setQueriesData.

* Migrate all fetchQuery uses.

* Migrate some leftover functions from RQ.

* Fix issues found by Charis.
2025-10-28 10:43:14 +01:00
Alaister YoungandJoshen Lim 8855d05803 chore(studio): swap react-query to object syntax (#39842)
* chore(studio): swap react-query to object syntax

* Fix small issues found

* Fix realtime settings

* Nit

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-10-27 09:38:27 +01:00
Raminder SinghandSaxon Fletcher 23ceb9072d Add replication UI in Studio (#30090)
* add dummy sinks and pipelines pages

* update api types

* show empty sources state

* show empty replication state at /replication

* create source when enable replication button is clicked

* improve replication page when replication is enabled

* replace sources page with publications page

* publications table

* show publications in table

* create publication wip

* show toast error instead of throwing an exception

* user can now delete a publication

* show empty sinks page

* create and list sinks

* add ui to delete a sink

* show pipelines on the pipelines page

* add ui to create and delete pipelines

* get pipeline status wip

* show pipeline status wip

* show correct label on action buttons

* start and stop pipelines

* remove a couple of console.logs

* fix error when deleting a pipeline

* only consider replication enabled when a source with name = ref is present

* add source and sink names

* correct colspan for 'no pipelines' row

* hide 'supabase_realtime' publication on ui

* move filtering to fetch query

* show sink name in ui

* show source/sink names on pipelines page

* fix start/stop status shown on ui

* fix prettier formatting

* update api types

* extract pipeline action button as a separate component

* fix a crashing page

* fixed publications page crash

* update to match with changes in api

* add new replication page under database

* hide replication page behind feature flag

* update types

* update api types

* show destinations empty state

* add destinations table

* factor out components from Destinations table

* show status dot

* move pipeline fetch query to parent component

* add ability to enable/disable a pipeline

* show loader when starting or stopping a pipeline

* fix a bug in which loading & empty states were shown together

* fix a bug in which error & empty states were shown together

* wrap in default layout

* add new destination panel

* add type field

* fix a forwardRef error

* fix layout

* create destination

* delete destination

* add ability to create or delete destinations with pipelines

* create source if missing

* show only a single error

* add an enable switch

* new layout

* add subsections

* comment out unused code

* show enabled switch only in the header

* close panel when destination is created

* disable buttons when api requests in flight

* reduce panel size

* remove commented out code

* treat max size and max fill secs as numbers

* use drop down to show publications

* simpler vertical layout

* add separators

* add form validation

* remove publications drop down padding

* add new publication button

* hide advanced settings behind an accordion

* add some margin between icon and text

* show publications panel on clicking new publication button

* add header to new publication panel

* fix validation not running for publication drop down

* create publication in the new publication panel

* add table selector in new publication panel

* update api types

* remove old code

* update platform.d.ts

* update navigation bar utils

* remove a redirect from replication page to publications page

* ask user for confirmation before deleting destination

* edit destination panel

* edit destination panel values fixed

* bug fixes

* fix prettier formatting

* enable/disable pipeline after editing

* rename snake_case params to camelCase

* loading button when editing

* remove merge markers

* update api types

* add max_staleness parameter in sinks for bigquery

* add read replicas flow diagram to replication page

* remove an unused import

* Revert "add read replicas flow diagram to replication page"

This reverts commit 8852d7847b457885603dba786141a8aaf8e99350.

* add panel to warn users about additional cost before creating a destination

* hide replication page contents behind a feature flag

* fix merge conflicts

* styling changes

* revert static flag

* styling updates

* fixes

* fix switch

* copy

* fix layout

---------

Co-authored-by: Saxon Fletcher <saxonafletcher@gmail.com>
2025-04-23 10:52:46 +05:30