## What kind of change does this PR introduce?
Feature. The last remaining piece of this PR's original scope, rebased
onto current `master`.
## What is the current behavior?
A BigQuery pipeline replicates every published table into a flat
destination table. There is no way to say how those tables should be
laid out in BigQuery, so partitioning and clustering have to be applied
by hand after the fact, and are lost whenever a destination table is
reset.
The rest of this PR's original scope has since merged separately:
#49841, #49842, #49843, #49844 and #49845. The branch now carries only
the table layout work, and adds nothing to `packages/api-types`. It
consumes the `table_options` contract that #49844 already brought in.
## What is the new behavior?
The BigQuery destination form gains a "Table layout" section under
Advanced settings. Every table in the selected publication appears as a
row, and expanding one reveals optional partitioning and clustering:
- Time-column partitioning, by hour, day, month or year
- Integer-range partitioning, with start, end and interval
- Ingestion-time partitioning
- Clustering, up to four columns
Rows are always present, so nothing implies a table can be excluded
here. A collapsed row summarises what is applied: `Not configured`,
`Daily by created_at`, `Integer range by id`, plus clustering counts.
`Not configured` is dimmed a step further so configured rows are what
the eye lands on in a long publication.
Notes on behavior:
- Partition columns are resolved from the published table's real
columns, filtered to the types BigQuery accepts for each partition mode.
- A row you expand but leave empty is dropped from the payload rather
than saved half-configured. Choosing a partition mode without a column
is a validation error, not a silent drop.
- Clear returns a row to `Not configured` and keeps it in the list.
Remove is reserved for stale configuration whose table has left the
publication.
- Updates send `null` to clear previously stored table options, since
omitting the property leaves the stored value unchanged.
- Layout applies when a destination table is first created or reset,
matching the backend.
Roughly 40% of the diff is tests. The bulk of the rest is four new files
under `DestinationForm/BigQuery`, which are all one feature. The three
commits are readable in order: types and payload builders, then the
columns query, then the UI.
## To test
Open a project's Database > Replication, then create or edit a BigQuery
pipeline.
1. Expand Advanced settings. "Table layout" lists every table in the
selected publication.
2. Expand a row, set Partition by to Time column and pick a column.
Collapse. The row reads `Daily by <column>`. Hit Clear. The row returns
to `Not configured` and stays in the list.
3. Set Partition by to Time column, leave Partition column empty,
collapse, and Save. The row explains "Select a partition column" in red
rather than saving and silently dropping it.
4. Expand a row and add clustering columns. The fifth is refused.
5. Switch publications and watch the loading state. The row list should
not jump size when it resolves.
6. Narrow the sheet. The integer range Start, End and Interval fields
should reflow rather than stay in three columns.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* BigQuery replication destinations now support per-table layout
settings.
* Configure partitioning by time column, integer range, or ingestion
time.
* Configure up to four clustering columns per table.
* View available columns and validation feedback while editing table
layouts.
* Table settings are preserved when editing destinations and cleaned up
when publications change.
* **Bug Fixes**
* Improved handling of invalid, unavailable, or removed table and column
configurations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Danny White <3104761+dnywh@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
## What kind of change does this PR introduce?
Studio data-layer migration.
## What is the current behavior?
Studio loads complete publication details through the original bulk
endpoint and creates publications by executing SQL against the source
database. Publication and source-table data use names where stable table
IDs are available.
## What is the new behavior?
Uses the v2 publication-name, publication-detail, publication mutation,
and source-table endpoints. The existing creation sheet continues to
behave the same, including publishing partition changes through the
parent table by default. Initial-sync selection and Analytics Bucket
associations now consume the selected publication detail. Generated
platform API types and their required nullability updates are included.
The generated Platform contract accounts for roughly 10,000 changed
lines in this PR.
## Dependency
Depends on the v2 source table, table column, and publication endpoints
from
[supabase/platform#37505](https://github.com/supabase/platform/pull/37505),
which are deployed to production.
## To test
1. Open the pipeline creation sheet and select an existing publication.
2. Create a publication with mixed-case schema and table names, then
confirm the table names are shown while stable IDs are submitted.
3. Exercise all four initial-sync policies, including selecting
individual tables.
4. Reopen the publication and table selectors and confirm they refresh
without replacing populated options.
5. Edit and delete a publication.
6. Open an Analytics Bucket associated with a pipeline and confirm its
publication tables resolve correctly.
7. Confirm unlimited WAL retention renders as Unlimited on pipeline
status.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- Improved replication publication setup with on-demand table loading,
refresh controls, clearer table labels, and streamlined publication
selection.
- Publication creation and updates now use the latest replication API
and table-based configurations.
- Added clearer handling for tables removed from publications, including
stale-selection warnings.
- **Bug Fixes**
- Prevented table selections from carrying over when switching
publications.
- Improved replication status displays when lag or WAL metrics are
unavailable.
- Updated replication deletion and table management for the latest API
behavior.
- **Tests**
- Expanded coverage for publication creation, table selection, stale
tables, loading states, and replication metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
## Context
Related to unifying read replicas into database replication page which
is currently in internal testing
## Changes involved
- Updates the "New replica" CTA in `DatabaseSelector` (e.g within the
SQL Editor) to link to the replication page instead of settings
infrastructure
- Refactor replication UI to use URL query states for new destination +
edit destination
## To test
- [ ] Verify that the "New replica" CTA links correctly to replication
page if flag is on
- [ ] Verify that new + edit destination UI works as expected
- Note that there's some server side validation for ETL replication now
so might be tricky to test creating an actual ETL replication
- Minimally can test creating a read replica and ensure that the UI
behaviour is all as expected
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added unified replication experience with URL-based state management
for destination creation and editing
* **Refactor**
* Simplified replication panel component interfaces and consolidated
destination data fetching logic
* Enhanced edit flows to leverage URL parameters for seamless navigation
* **Chores**
* Marked legacy read replica creation panel as deprecated
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* add dummy sinks and pipelines pages
* update api types
* show empty sources state
* show empty replication state at /replication
* create source when enable replication button is clicked
* improve replication page when replication is enabled
* replace sources page with publications page
* publications table
* show publications in table
* create publication wip
* show toast error instead of throwing an exception
* user can now delete a publication
* show empty sinks page
* create and list sinks
* add ui to delete a sink
* show pipelines on the pipelines page
* add ui to create and delete pipelines
* get pipeline status wip
* show pipeline status wip
* show correct label on action buttons
* start and stop pipelines
* remove a couple of console.logs
* fix error when deleting a pipeline
* only consider replication enabled when a source with name = ref is present
* add source and sink names
* correct colspan for 'no pipelines' row
* hide 'supabase_realtime' publication on ui
* move filtering to fetch query
* show sink name in ui
* show source/sink names on pipelines page
* fix start/stop status shown on ui
* fix prettier formatting
* update api types
* extract pipeline action button as a separate component
* fix a crashing page
* fixed publications page crash
* update to match with changes in api
* add new replication page under database
* hide replication page behind feature flag
* update types
* update api types
* show destinations empty state
* add destinations table
* factor out components from Destinations table
* show status dot
* move pipeline fetch query to parent component
* add ability to enable/disable a pipeline
* show loader when starting or stopping a pipeline
* fix a bug in which loading & empty states were shown together
* fix a bug in which error & empty states were shown together
* wrap in default layout
* add new destination panel
* add type field
* fix a forwardRef error
* fix layout
* create destination
* delete destination
* add ability to create or delete destinations with pipelines
* create source if missing
* show only a single error
* add an enable switch
* new layout
* add subsections
* comment out unused code
* show enabled switch only in the header
* close panel when destination is created
* disable buttons when api requests in flight
* reduce panel size
* remove commented out code
* treat max size and max fill secs as numbers
* use drop down to show publications
* simpler vertical layout
* add separators
* add form validation
* remove publications drop down padding
* add new publication button
* hide advanced settings behind an accordion
* add some margin between icon and text
* show publications panel on clicking new publication button
* add header to new publication panel
* fix validation not running for publication drop down
* create publication in the new publication panel
* add table selector in new publication panel
* update api types
* remove old code
* update platform.d.ts
* update navigation bar utils
* remove a redirect from replication page to publications page
* ask user for confirmation before deleting destination
* edit destination panel
* edit destination panel values fixed
* bug fixes
* fix prettier formatting
* enable/disable pipeline after editing
* rename snake_case params to camelCase
* loading button when editing
* remove merge markers
* update api types
* add max_staleness parameter in sinks for bigquery
* add read replicas flow diagram to replication page
* remove an unused import
* Revert "add read replicas flow diagram to replication page"
This reverts commit 8852d7847b457885603dba786141a8aaf8e99350.
* add panel to warn users about additional cost before creating a destination
* hide replication page contents behind a feature flag
* fix merge conflicts
* styling changes
* revert static flag
* styling updates
* fixes
* fix switch
* copy
* fix layout
---------
Co-authored-by: Saxon Fletcher <saxonafletcher@gmail.com>
