mirror of
https://github.com/supabase/supabase.git
synced 2026-10-06 01:45:10 +03:00
## What kind of change does this PR introduce? Studio data-layer migration. ## What is the current behavior? Studio loads complete publication details through the original bulk endpoint and creates publications by executing SQL against the source database. Publication and source-table data use names where stable table IDs are available. ## What is the new behavior? Uses the v2 publication-name, publication-detail, publication mutation, and source-table endpoints. The existing creation sheet continues to behave the same, including publishing partition changes through the parent table by default. Initial-sync selection and Analytics Bucket associations now consume the selected publication detail. Generated platform API types and their required nullability updates are included. The generated Platform contract accounts for roughly 10,000 changed lines in this PR. ## Dependency Depends on the v2 source table, table column, and publication endpoints from [supabase/platform#37505](https://github.com/supabase/platform/pull/37505), which are deployed to production. ## To test 1. Open the pipeline creation sheet and select an existing publication. 2. Create a publication with mixed-case schema and table names, then confirm the table names are shown while stable IDs are submitted. 3. Exercise all four initial-sync policies, including selecting individual tables. 4. Reopen the publication and table selectors and confirm they refresh without replacing populated options. 5. Edit and delete a publication. 6. Open an Analytics Bucket associated with a pipeline and confirm its publication tables resolve correctly. 7. Confirm unlimited WAL retention renders as Unlimited on pipeline status. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Improved replication publication setup with on-demand table loading, refresh controls, clearer table labels, and streamlined publication selection. - Publication creation and updates now use the latest replication API and table-based configurations. - Added clearer handling for tables removed from publications, including stale-selection warnings. - **Bug Fixes** - Prevented table selections from carrying over when switching publications. - Improved replication status displays when lag or WAL metrics are unavailable. - Updated replication deletion and table management for the latest API behavior. - **Tests** - Expanded coverage for publication creation, table selection, stale tables, loading states, and replication metrics. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
355 lines
12 KiB
TypeScript
355 lines
12 KiB
TypeScript
import { useQueryClient } from '@tanstack/react-query'
|
|
import { useParams } from 'common'
|
|
import { MessageSquare, MoreVertical, Plus, Search, Workflow, X } from 'lucide-react'
|
|
import Link from 'next/link'
|
|
import { parseAsStringEnum, useQueryState } from 'nuqs'
|
|
import { useEffect, useMemo, useRef, useState } from 'react'
|
|
import {
|
|
Button,
|
|
Card,
|
|
CardContent,
|
|
DropdownMenu,
|
|
DropdownMenuContent,
|
|
DropdownMenuItem,
|
|
DropdownMenuSeparator,
|
|
DropdownMenuTrigger,
|
|
Table,
|
|
TableBody,
|
|
TableCell,
|
|
TableHead,
|
|
TableHeader,
|
|
TableRow,
|
|
} from 'ui'
|
|
import { Input } from 'ui-patterns/DataInputs/Input'
|
|
import { EmptyStatePresentational } from 'ui-patterns/EmptyStatePresentational'
|
|
import { GenericSkeletonLoader } from 'ui-patterns/ShimmeringLoader'
|
|
|
|
import { DestinationPanel } from './DestinationPanel/DestinationPanel'
|
|
import { DestinationType } from './DestinationPanel/DestinationPanel.types'
|
|
import { DestinationRow } from './DestinationRow'
|
|
import { DisablePipelinesDialog } from './DisablePipelinesDialog'
|
|
import { EnablePipelinesModal } from './EnablePipelinesCallout'
|
|
import { PIPELINES_FEEDBACK_URL } from './Replication.constants'
|
|
import {
|
|
useIsETLBigQueryPrivateAlpha,
|
|
useIsETLClickHousePrivateAlpha,
|
|
useIsETLDucklakePrivateAlpha,
|
|
useIsETLIcebergPrivateAlpha,
|
|
useIsETLSnowflakePrivateAlpha,
|
|
} from './useIsETLPrivateAlpha'
|
|
import { useRedirectLegacyReadReplicaDestination } from './useRedirectLegacyReadReplicaDestination'
|
|
import { AlertError } from '@/components/ui/AlertError'
|
|
import { DocsButton } from '@/components/ui/DocsButton'
|
|
import { DropdownMenuItemTooltip } from '@/components/ui/DropdownMenuItemTooltip'
|
|
import { Shortcut } from '@/components/ui/Shortcut'
|
|
import { useReplicationDestinationsQuery } from '@/data/replication/destinations-query'
|
|
import { replicationKeys } from '@/data/replication/keys'
|
|
import { fetchReplicationPipelineVersion } from '@/data/replication/pipeline-version-query'
|
|
import { useReplicationPipelinesQuery } from '@/data/replication/pipelines-query'
|
|
import { useReplicationSourcesQuery } from '@/data/replication/sources-query'
|
|
import { checkLocalETLNotSetUp } from '@/data/replication/utils'
|
|
import { useSelectedOrganizationQuery } from '@/hooks/misc/useSelectedOrganization'
|
|
import { DOCS_URL } from '@/lib/constants'
|
|
import { onSearchInputEscape } from '@/lib/keyboard'
|
|
import { SHORTCUT_IDS } from '@/state/shortcuts/registry'
|
|
import { useShortcut } from '@/state/shortcuts/useShortcut'
|
|
|
|
export const Destinations = () => {
|
|
const queryClient = useQueryClient()
|
|
const { ref: projectRef } = useParams()
|
|
const { data: organization } = useSelectedOrganizationQuery()
|
|
|
|
useRedirectLegacyReadReplicaDestination()
|
|
|
|
const etlEnableBigQuery = useIsETLBigQueryPrivateAlpha()
|
|
const etlEnableIceberg = useIsETLIcebergPrivateAlpha()
|
|
const etlEnableDucklake = useIsETLDucklakePrivateAlpha()
|
|
const etlEnableSnowflake = useIsETLSnowflakePrivateAlpha()
|
|
const etlEnableClickHouse = useIsETLClickHousePrivateAlpha()
|
|
|
|
const newDestinationDefaultType: DestinationType | null = etlEnableBigQuery
|
|
? 'BigQuery'
|
|
: etlEnableIceberg
|
|
? 'Analytics Bucket'
|
|
: etlEnableDucklake
|
|
? 'DuckLake'
|
|
: etlEnableSnowflake
|
|
? 'Snowflake'
|
|
: etlEnableClickHouse
|
|
? 'ClickHouse'
|
|
: null
|
|
|
|
const prefetchedRef = useRef(false)
|
|
const searchInputRef = useRef<HTMLInputElement>(null)
|
|
const [filterString, setFilterString] = useState<string>('')
|
|
const [showEnablePipelinesDialog, setShowEnablePipelinesDialog] = useState(false)
|
|
const [showDisablePipelinesDialog, setShowDisablePipelinesDialog] = useState(false)
|
|
|
|
const [, setDestinationType] = useQueryState(
|
|
'destinationType',
|
|
parseAsStringEnum<DestinationType>([
|
|
'BigQuery',
|
|
'Analytics Bucket',
|
|
'DuckLake',
|
|
'Snowflake',
|
|
'ClickHouse',
|
|
]).withOptions({
|
|
history: 'push',
|
|
clearOnDefault: true,
|
|
})
|
|
)
|
|
|
|
const {
|
|
data: destinationsData,
|
|
error: destinationsError,
|
|
isPending: isDestinationsLoading,
|
|
isError: isDestinationsError,
|
|
isSuccess: isDestinationsSuccess,
|
|
} = useReplicationDestinationsQuery({
|
|
projectRef,
|
|
})
|
|
const destinations = destinationsData?.destinations ?? []
|
|
const hasDestinations = isDestinationsSuccess && destinationsData?.destinations.length > 0
|
|
const filteredDestinations =
|
|
filterString.length === 0
|
|
? (destinations ?? [])
|
|
: (destinations ?? []).filter((destination) =>
|
|
destination.name.toLowerCase().includes(filterString.toLowerCase())
|
|
)
|
|
|
|
const { data: pipelinesData, isSuccess: isPipelinesSuccess } = useReplicationPipelinesQuery({
|
|
projectRef,
|
|
})
|
|
const pipelines = pipelinesData?.pipelines ?? []
|
|
|
|
const { data: sourcesData, isSuccess: isSourcesSuccess } = useReplicationSourcesQuery({
|
|
projectRef,
|
|
})
|
|
const externalReplicationSource = useMemo(
|
|
() => sourcesData?.sources.find((source) => source.name === projectRef),
|
|
[projectRef, sourcesData?.sources]
|
|
)
|
|
const replicationNotEnabled = isSourcesSuccess && !externalReplicationSource
|
|
|
|
const canDisablePipelines =
|
|
isSourcesSuccess &&
|
|
isDestinationsSuccess &&
|
|
isPipelinesSuccess &&
|
|
!!externalReplicationSource &&
|
|
destinations.length === 0 &&
|
|
pipelines.length === 0
|
|
|
|
const isLocalETLNotSetUp = checkLocalETLNotSetUp(destinationsError)
|
|
const hasErrorsFetchingData = !isLocalETLNotSetUp && isDestinationsError
|
|
|
|
const openDestinationPanel = () => {
|
|
if (!newDestinationDefaultType) return
|
|
setDestinationType(newDestinationDefaultType)
|
|
}
|
|
|
|
useShortcut(
|
|
SHORTCUT_IDS.LIST_PAGE_FOCUS_SEARCH,
|
|
() => {
|
|
searchInputRef.current?.focus()
|
|
searchInputRef.current?.select()
|
|
},
|
|
{ label: 'Search destinations' }
|
|
)
|
|
|
|
useShortcut(SHORTCUT_IDS.LIST_PAGE_RESET_FILTERS, () => setFilterString(''))
|
|
|
|
useEffect(() => {
|
|
if (
|
|
projectRef &&
|
|
!prefetchedRef.current &&
|
|
pipelinesData?.pipelines &&
|
|
pipelinesData.pipelines.length > 0 &&
|
|
isPipelinesSuccess
|
|
) {
|
|
prefetchedRef.current = true
|
|
pipelinesData.pipelines.forEach((p) => {
|
|
if (!p?.id) return
|
|
queryClient.prefetchQuery({
|
|
queryKey: replicationKeys.pipelinesVersion(projectRef, p.id),
|
|
queryFn: ({ signal }) =>
|
|
fetchReplicationPipelineVersion({ projectRef, pipelineId: p.id }, signal),
|
|
staleTime: Infinity,
|
|
})
|
|
})
|
|
}
|
|
}, [projectRef, pipelinesData?.pipelines, isPipelinesSuccess, queryClient])
|
|
|
|
return (
|
|
<div className="w-full space-y-4">
|
|
<div className="flex items-center justify-between">
|
|
<div className="flex items-center">
|
|
<Input
|
|
ref={searchInputRef}
|
|
placeholder="Filter destinations"
|
|
size="tiny"
|
|
icon={<Search />}
|
|
value={filterString}
|
|
className="w-full lg:w-52"
|
|
onChange={(e) => setFilterString(e.target.value)}
|
|
onKeyDown={onSearchInputEscape(filterString, setFilterString)}
|
|
actions={
|
|
filterString.length > 0 && (
|
|
<Button
|
|
aria-label="Clear filter"
|
|
variant="text"
|
|
icon={<X />}
|
|
className="p-0 h-5 w-5"
|
|
onClick={() => setFilterString('')}
|
|
/>
|
|
)
|
|
}
|
|
/>
|
|
</div>
|
|
<div className="flex items-center gap-x-2">
|
|
<DropdownMenu>
|
|
<DropdownMenuTrigger asChild>
|
|
<Button
|
|
aria-label="More actions"
|
|
variant="default"
|
|
icon={<MoreVertical />}
|
|
className="px-1"
|
|
/>
|
|
</DropdownMenuTrigger>
|
|
<DropdownMenuContent align="end" className="w-52">
|
|
<DropdownMenuItem asChild>
|
|
<Link href={`/org/${organization?.slug}/usage#pipeline-initial-sync-data`}>
|
|
View Pipelines usage
|
|
</Link>
|
|
</DropdownMenuItem>
|
|
<DropdownMenuSeparator />
|
|
{replicationNotEnabled ? (
|
|
<DropdownMenuItem onClick={() => setShowEnablePipelinesDialog(true)}>
|
|
Enable Pipelines
|
|
</DropdownMenuItem>
|
|
) : (
|
|
<DropdownMenuItemTooltip
|
|
disabled={!canDisablePipelines}
|
|
tooltip={{
|
|
content: {
|
|
side: 'left',
|
|
text: 'Remove all existing destinations before disabling Pipelines',
|
|
},
|
|
}}
|
|
onClick={() => setShowDisablePipelinesDialog(true)}
|
|
>
|
|
Disable Pipelines
|
|
</DropdownMenuItemTooltip>
|
|
)}
|
|
</DropdownMenuContent>
|
|
</DropdownMenu>
|
|
|
|
<Button asChild variant="default" icon={<MessageSquare />}>
|
|
<a href={PIPELINES_FEEDBACK_URL} target="_blank" rel="noreferrer noopener">
|
|
Leave feedback
|
|
</a>
|
|
</Button>
|
|
<DocsButton href={`${DOCS_URL}/guides/database/replication`} />
|
|
|
|
<Shortcut
|
|
id={SHORTCUT_IDS.LIST_PAGE_NEW_ITEM}
|
|
label="Add destination"
|
|
onTrigger={openDestinationPanel}
|
|
options={{ enabled: !!newDestinationDefaultType }}
|
|
side="bottom"
|
|
>
|
|
<Button
|
|
variant="primary"
|
|
icon={<Plus />}
|
|
disabled={!newDestinationDefaultType}
|
|
onClick={openDestinationPanel}
|
|
>
|
|
Add destination
|
|
</Button>
|
|
</Shortcut>
|
|
</div>
|
|
</div>
|
|
|
|
<div className="w-full overflow-hidden overflow-x-auto flex flex-col gap-y-4">
|
|
{hasErrorsFetchingData && (
|
|
<AlertError error={destinationsError} subject="Failed to retrieve destinations" />
|
|
)}
|
|
|
|
{isDestinationsLoading ? (
|
|
<GenericSkeletonLoader />
|
|
) : hasDestinations ? (
|
|
<Card>
|
|
<CardContent className="p-0">
|
|
<Table>
|
|
<TableHeader>
|
|
<TableRow>
|
|
<TableHead key="type" className="w-[20px]" />
|
|
<TableHead key="name" className="w-[250px]">
|
|
Name
|
|
</TableHead>
|
|
<TableHead key="status" className="w-[150px]">
|
|
Status
|
|
</TableHead>
|
|
<TableHead key="lag" className="w-[150px]">
|
|
Lag
|
|
</TableHead>
|
|
<TableHead key="publication">Publication</TableHead>
|
|
<TableHead key="actions" />
|
|
</TableRow>
|
|
</TableHeader>
|
|
<TableBody>
|
|
{filteredDestinations.map((destination) => (
|
|
<DestinationRow key={destination.id} destinationId={destination.id} />
|
|
))}
|
|
|
|
{!isDestinationsLoading &&
|
|
filteredDestinations.length === 0 &&
|
|
hasDestinations && (
|
|
<TableRow>
|
|
<TableCell colSpan={6}>
|
|
<p>No results found</p>
|
|
<p className="text-foreground-light">
|
|
Your search for "{filterString}" did not return any results.
|
|
</p>
|
|
</TableCell>
|
|
</TableRow>
|
|
)}
|
|
</TableBody>
|
|
</Table>
|
|
</CardContent>
|
|
</Card>
|
|
) : (
|
|
!isDestinationsLoading &&
|
|
!hasErrorsFetchingData && (
|
|
<EmptyStatePresentational
|
|
icon={Workflow}
|
|
title="Add a destination"
|
|
description="Connect an external destination for analytics workloads."
|
|
>
|
|
<Button
|
|
variant="default"
|
|
icon={<Plus />}
|
|
disabled={!newDestinationDefaultType}
|
|
onClick={openDestinationPanel}
|
|
>
|
|
Add destination
|
|
</Button>
|
|
</EmptyStatePresentational>
|
|
)
|
|
)}
|
|
</div>
|
|
|
|
<DestinationPanel />
|
|
|
|
<EnablePipelinesModal
|
|
open={showEnablePipelinesDialog}
|
|
onOpenChange={setShowEnablePipelinesDialog}
|
|
/>
|
|
|
|
<DisablePipelinesDialog
|
|
open={showDisablePipelinesDialog}
|
|
setOpen={setShowDisablePipelinesDialog}
|
|
/>
|
|
</div>
|
|
)
|
|
}
|