feat(studio): ClickHouse dialect for logs snippet AI + rewrite to ClickHouse (#48501)

## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Feature, plus a refactor of the shared logs-rewrite flow.

PR 8 of the SQL editor query-source series. Stacked on #48457 — review
that one first, and merge this after it.

## What is the current behavior?

A `log_sql` snippet runs against the ClickHouse-backed analytics
endpoint, but the SQL editor's AI still writes Postgres: inline edits
get Postgres system prompts, and the result is run through
`sql-formatter`, which mangles ClickHouse backticks and `log_attributes`
map lookups.

Legacy Logs Explorer saved queries open in the editor as `log_sql`
snippets. Those are BigQuery dialect and error against the ClickHouse
endpoint the editor runs them on, with no in-editor way out — only the
Logs Explorer offered a rewrite.

The completion route was also asymmetric. It assembled a
schema/code/instruction message for Postgres but forwarded `prompt`
verbatim for ClickHouse, so a client wanting ClickHouse had to
hand-build the equivalent string.

## What is the new behavior?

**Inline AI speaks ClickHouse for logs snippets.** `sqlSourceToDialect`
maps a snippet's source to `postgres`/`clickhouse` and
`buildCompletionRequestBody` threads it through. For ClickHouse,
`useSqlEditorAi` strips code fences from the response and skips
`formatSql`. Execution and dialect both follow the snippet type, so a
snippet's valid dialect never flips.

**Rewrite to ClickHouse in the editor.** A banner offers the rewrite for
a logs snippet whose text trips `looksLikeLegacyLogsQuery`, and proposes
the result through the editor's existing AI diff view rather than
replacing the snippet, so it's accepted or discarded like any other AI
edit. Gated on `otelLegacyLogs`: on a non-migrated org the BigQuery text
is still correct, so rewriting it would break a working query.

The offer is a state machine (`offered` / `rewriting` / `failed` /
`noRewriteNeeded` / `dismissed`) with a declarative table of valid
transitions, so the states are mutually exclusive by construction and
dismissal is terminal. A failure keeps its message and offers a retry; a
response identical to the input is reported rather than opening an empty
diff.

**One place assembles completion prompts.** The route now uses a single
template for both dialects, branching only the schema section and — for
`intent: 'rewrite'` — the instruction. `lib/ai/clickhouse-logs.ts` is
the single home for ClickHouse-logs prompt content, replacing two
independently maintained descriptions of the same table. Clients carry
no prompt text.

**The rewrite flow is shared with the Logs Explorer.** Both surfaces
previously hand-rolled the same sequence and had drifted: only one
detected a no-op rewrite, they sourced `log_attributes` keys
differently, and the Explorer formatted errors with an `as Error` cast.
Both now use `useLegacyLogsRewrite` and the same state-driven banner, so
the Explorer picks up no-op detection and typed error extraction.

**Attribute keys are fetched on submit, not while typing.** The detected
source would otherwise feed a reactive query key, making every edit that
changed it cost another network call. `useLogsAttributeKeys` is
imperative and goes through `queryClient.fetchQuery`, so a source
already cached — including by the Explorer header and query panel, which
subscribe reactively — is reused. This also closes a gap where inline
edits never received keys at all, unlike full rewrites.

`getErrorMessage` gains an optional typed fallback and no longer
stringifies a bare object into `'[object Object]'`; every existing
caller already hand-rolled a fallback, except `QueueSettings`, which
interpolated the raw result and now passes one.

Nothing here is user-visible until the `sqlEditorLogsSource` flag is
enabled.

Tests: dialect selection and request-body shape, the ClickHouse prompt
content (including that the schema section does not restate the dialect
rules), the reducer's valid and invalid transitions,
`shouldOfferLegacyLogsRewrite`, on-submit key discovery with cache
reuse, and `getErrorMessage`.

## Additional context

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added an Assistant banner to help rewrite legacy BigQuery-style logs
queries into ClickHouse SQL.
* SQL assistance now adapts to the selected query type, including
relevant log attribute context.
* Rewrite suggestions can be reviewed as editor diffs before being
applied.

* **Bug Fixes**
* Improved rewrite failure handling, retry options, dismissal behavior,
and “no rewrite needed” messaging.
* Error notifications now provide a clearer fallback message when
details are unavailable.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
Charis authored and GitHub committed 2026-08-04 09:02:40 -04:00
1 parent b3c5c9fc04
commit 8b38e0d1ed
24 files changed
+1286 -314

No files matched your search

@@ -214,7 +214,7 @@ export const QueueSettings = ({}: QueueSettingsProps) => {
toast.success('Successfully updated permissions')
setOpen(false)
} catch (error: unknown) {
toast.error(`Failed to update permissions: ${getErrorMessage(error)}`)
toast.error(`Failed to update permissions: ${getErrorMessage(error, 'unknown error')}`)
} finally {
setIsSaving(false)
}
@@ -0,0 +1,77 @@
import { useDebounce } from '@uidotdev/usehooks'
import { useFlag } from 'common'
import { useMemo } from 'react'
import { DiffType } from './SQLEditor.types'
import { useSqlEditorAssistant, useSqlEditorRun, useSqlEditorSnippet } from './SQLEditorControllers'
import { LegacyLogsRewriteAdmonition } from '@/components/interfaces/Settings/Logs/LegacyLogsRewriteAdmonition'
import {
LEGACY_LOGS_DIALECT_CHECK_DEBOUNCE_MS,
shouldOfferLegacyLogsRewrite,
} from '@/data/logs/logs-sql-rewrite'
import { useLegacyLogsRewrite } from '@/hooks/analytics/useLegacyLogsRewrite'
import {
getSqlEditorV2StateSnapshot,
useSqlEditorV2StateSnapshot,
} from '@/state/sql-editor/sql-editor-state'
/**
* Offers to rewrite a logs snippet still written in the old BigQuery dialect
* (per-service `FROM` tables, `unnest(metadata)` joins) to ClickHouse SQL. Legacy
* Logs Explorer saved queries open in the SQL editor as `log_sql` snippets, and
* those queries error against the ClickHouse-backed endpoint the editor runs them
* on — this is the in-editor path out.
*
* The request itself is `useLegacyLogsRewrite` (shared with the Logs Explorer);
* this decides when to offer it and routes the result into the editor's existing
* AI diff view, so the user accepts or discards it the same way as any other AI
* edit rather than having the snippet rewritten under them.
*
* Mount with `key={id}` so the offer resets when the user switches snippets. This
* component must NOT be conditionally mounted by its parent — it hides itself, so
* that opening a diff doesn't unmount it and throw away a dismissal.
*/
export const LegacyLogsRewriteBanner = () => {
const { id } = useSqlEditorSnippet()
const { runSource } = useSqlEditorRun()
const {
diff: { isDiffOpen, setSourceSqlDiff, setSelectedDiffType },
} = useSqlEditorAssistant()
const isOtelLogsEnabled = useFlag('otelLegacyLogs')
const snapV2 = useSqlEditorV2StateSnapshot()
// The store is written on every keystroke, so debounce before running the
// dialect heuristics — the banner's visibility doesn't need per-character
// precision, and a settled value avoids flapping mid-edit.
const liveSql = snapV2.snippets[id]?.snippet.content?.unchecked_sql ?? ''
const settledSql = useDebounce(liveSql, LEGACY_LOGS_DIALECT_CHECK_DEBOUNCE_MS)
const isLogsSnippetNeedingRewrite = useMemo(
() =>
runSource.type === 'logs' &&
shouldOfferLegacyLogsRewrite({ sql: settledSql, isClickhouseLogsEnabled: isOtelLogsEnabled }),
[runSource.type, settledSql, isOtelLogsEnabled]
)
const { state, requestRewrite, dismiss } = useLegacyLogsRewrite({
// Rewrite exactly what's in the editor now, not the debounced value the
// visibility check used — they differ if the user clicked mid-edit.
readSql: () => getSqlEditorV2StateSnapshot().snippets[id]?.snippet.content?.unchecked_sql ?? '',
onProposal: ({ original, modified }) => {
setSourceSqlDiff({ original, modified })
setSelectedDiffType(DiffType.Modification)
},
})
// An outcome the user hasn't acknowledged stays up even once the query no longer
// looks legacy — otherwise a successful proposal would yank its own result away.
const hasUnacknowledgedOutcome = state.status === 'failed' || state.status === 'noRewriteNeeded'
const canShowBanner = !isDiffOpen && (isLogsSnippetNeedingRewrite || hasUnacknowledgedOutcome)
if (!canShowBanner) return null
return (
<LegacyLogsRewriteAdmonition state={state} onRewrite={requestRewrite} onDismiss={dismiss} />
)
}
@@ -29,6 +29,7 @@ import {
resolveConnectionString,
resolveDiffKeyAction,
shouldAutoGenerateTitle,
sqlSourceToDialect,
trimTrailingSemicolons,
} from './SQLEditor.utils'
import type { DatabaseEventTrigger } from '@/data/database-event-triggers/database-event-triggers-query'
@@ -432,6 +433,31 @@ describe('SQLEditor.utils.ts:buildCompletionRequestBody', () => {
completionMetadata: { prompt: 'add a where clause' },
})
})
test('omits dialect when not provided, so the route keeps its Postgres default', () => {
const body = buildCompletionRequestBody({
projectRef: 'default',
connectionString: null,
orgSlug: 'acme',
})
expect(body).not.toHaveProperty('dialect')
})
test('includes the dialect when provided', () => {
expect(
buildCompletionRequestBody({
projectRef: 'default',
connectionString: null,
orgSlug: 'acme',
dialect: 'clickhouse',
}).dialect
).toBe('clickhouse')
})
})
describe('SQLEditor.utils.ts:sqlSourceToDialect', () => {
test('logs snippets get ClickHouse, database snippets get Postgres', () => {
expect(sqlSourceToDialect('logs')).toBe('clickhouse')
expect(sqlSourceToDialect('database')).toBe('postgres')
})
})
describe('SQLEditor.utils.ts:createSqlSnippetSkeletonV2', () => {
@@ -497,25 +497,46 @@ export function assembleCompletionDiff(
}
/**
* Builds the request body sent to the AI completion endpoint. `options` is
* the caller-provided extra fields (e.g. `completionMetadata`), merged in
* last so it can override the defaults if it ever needs to.
* The SQL dialect the AI writes. Mirrors the `dialect` enum the completion API
* route accepts — a snippet's dialect follows its source and never flips, so a
* logs snippet always gets ClickHouse SQL and a database snippet Postgres.
*/
export type SqlDialect = 'postgres' | 'clickhouse'
/**
* Maps a snippet's query source to the dialect the AI should write in. Logs
* snippets run against the ClickHouse-backed analytics endpoint; everything
* else runs against the user's Postgres database.
*/
export function sqlSourceToDialect(source: SqlSnippetSource): SqlDialect {
return source === 'logs' ? 'clickhouse' : 'postgres'
}
/**
* Builds the request body sent to the AI completion endpoint. `dialect` is
* omitted when undefined so callers that don't care keep the route's Postgres
* default. `options` is the caller-provided extra fields (e.g.
* `completionMetadata`), merged in last so it can override the defaults if it
* ever needs to.
*/
export function buildCompletionRequestBody({
projectRef,
connectionString,
orgSlug,
dialect,
options,
}: {
projectRef: string | undefined
connectionString: string | undefined | null
orgSlug: string | undefined
dialect?: SqlDialect
options?: { completionMetadata?: unknown }
}): {
projectRef: string | undefined
connectionString: string | undefined | null
language: 'sql'
orgSlug: string | undefined
dialect?: SqlDialect
completionMetadata?: unknown
} {
return {
@@ -523,6 +544,7 @@ export function buildCompletionRequestBody({
connectionString,
language: 'sql',
orgSlug,
...(dialect !== undefined && { dialect }),
...(options ?? {}),
}
}
@@ -191,7 +191,7 @@ export const SQLEditorControllersProvider = ({ children }: PropsWithChildren) =>
const isExecuting = isExecutingDb || isExecutingLogs
const ai = useSqlEditorAi({ id, editorMountCount, diff, prompt })
const ai = useSqlEditorAi({ id, editorMountCount, diff, prompt, sqlSource: runSource.type })
const { acceptAiHandler, discardAiHandler } = ai
useSqlEditorShortcuts({
@@ -4,6 +4,7 @@ import dynamic from 'next/dynamic'
import { useCallback } from 'react'
import { cn } from 'ui'
import { LegacyLogsRewriteBanner } from './LegacyLogsRewriteBanner'
import { useSQLEditorContext } from './SQLEditorContext'
import {
useSqlEditorAssistant,
@@ -171,23 +172,32 @@ const SQLEditorMainView = () => {
)
}
/** The top (editor) resizable panel: loading state, diff view, and main editor. */
export const SQLEditorEditorPanel = () => {
const { isLoading } = useSqlEditorSnippet()
const { diff } = useSqlEditorAssistant()
function LoadingSpinner() {
return (
<div className="grow overflow-y-auto border-b h-full">
{isLoading ? (
<div className="flex h-full w-full items-center justify-center">
<Loader2 className="animate-spin text-brand" />
</div>
) : (
<>
{diff.isDiffOpen && <SQLEditorDiffView />}
<SQLEditorMainView />
</>
)}
<div className="overflow-y-auto h-full">
<div className="flex h-full w-full items-center justify-center">
<Loader2 className="animate-spin text-brand" />
</div>
</div>
)
}
/** The top (editor) resizable panel: loading state, diff view, and main editor. */
export const SQLEditorEditorPanel = () => {
const { id, isLoading } = useSqlEditorSnippet()
const { diff } = useSqlEditorAssistant()
if (isLoading) {
return <LoadingSpinner />
}
return (
<div className="h-full flex flex-col">
<LegacyLogsRewriteBanner key={id} />
<div className="grow overflow-y-auto min-h-0">
{diff.isDiffOpen && <SQLEditorDiffView />}
<SQLEditorMainView />
</div>
</div>
)
}
@@ -1,13 +1,17 @@
import { act, waitFor } from '@testing-library/react'
import { http, HttpResponse } from 'msw'
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { useSqlEditorDiff, useSqlEditorPrompt } from './hooks'
import type { SqlSnippetSource } from './querySource'
import { DiffType } from './SQLEditor.types'
import { useSqlEditorAi } from './useSqlEditorAi'
import { SIDEBAR_KEYS } from '@/components/layouts/ProjectLayout/LayoutSidebar/LayoutSidebarProvider'
import { API_URL } from '@/lib/constants'
import { sidebarManagerState } from '@/state/sidebar-manager-state'
import { sqlEditorDiffRequestState } from '@/state/sql-editor/sql-editor-diff-request'
import { sqlEditorSessionState } from '@/state/sql-editor/sql-editor-session-state'
import { mswServer } from '@/tests/lib/msw'
import {
createInMemoryEditor,
renderSqlEditorHook,
@@ -18,15 +22,17 @@ import {
const SNIPPET_ID = 'ai-snippet'
type HarnessProps = { editorMountCount?: number; sqlSource?: SqlSnippetSource }
/**
* Composes the diff + prompt state hooks the AI hook depends on (production
* wires these together in `SQLEditorControllers`), so tests drive the real
* accept/discard/drain flows end to end.
*/
function useAiHarness({ editorMountCount = 1 }: { editorMountCount?: number } = {}) {
function useAiHarness({ editorMountCount = 1, sqlSource = 'database' }: HarnessProps = {}) {
const diff = useSqlEditorDiff()
const prompt = useSqlEditorPrompt()
const ai = useSqlEditorAi({ id: SNIPPET_ID, editorMountCount, diff, prompt })
const ai = useSqlEditorAi({ id: SNIPPET_ID, editorMountCount, diff, prompt, sqlSource })
return { ai, diff, prompt }
}
@@ -119,6 +125,110 @@ describe('useSqlEditorAi — accept / discard diff', () => {
})
})
describe('useSqlEditorAi — completion dialect', () => {
type CompletionRequestBody = {
dialect?: string
intent?: string
completionMetadata: {
prompt: string
selection: string
textBeforeCursor: string
textAfterCursor: string
availableKeys?: string[]
}
}
/** Replaces the default completion mock so we can read what was posted. */
function captureCompletionRequests(response: string) {
const bodies: CompletionRequestBody[] = []
mswServer.use(
http.post(`${API_URL}/ai/code/complete`, async ({ request }) => {
bodies.push((await request.json()) as CompletionRequestBody)
return HttpResponse.json(response)
})
)
return bodies
}
const context = {
beforeSelection: "select timestamp from logs where source = 'edge_logs'\n",
selection: 'limit 5',
afterSelection: '',
}
it('posts the clickhouse dialect with the raw instruction and cursor context', async () => {
const bodies = captureCompletionRequests('limit 10')
const { result } = renderSqlEditorHook(useAiHarness, {
initialProps: { sqlSource: 'logs' },
})
await act(async () => {
await result.current.ai.handlePrompt('only keep 5xx responses', context)
})
expect(bodies).toHaveLength(1)
expect(bodies[0].dialect).toBe('clickhouse')
// The route assembles the schema section and the selection-wrapped code around
// the instruction, so the client posts the instruction verbatim — the same
// shape as the Postgres path — and never hand-builds prompt text.
expect(bodies[0].completionMetadata.prompt).toBe('only keep 5xx responses')
expect(bodies[0].completionMetadata.selection).toBe('limit 5')
expect(bodies[0].completionMetadata.textBeforeCursor).toBe(context.beforeSelection)
// An inline edit is not a rewrite.
expect(bodies[0].intent).toBeUndefined()
})
it('posts the postgres dialect with the raw instruction for a database snippet', async () => {
const bodies = captureCompletionRequests('limit 10')
const { result } = renderSqlEditorHook(useAiHarness, {
initialProps: { sqlSource: 'database' },
})
await act(async () => {
await result.current.ai.handlePrompt('bump the limit', context)
})
expect(bodies).toHaveLength(1)
expect(bodies[0].dialect).toBe('postgres')
expect(bodies[0].completionMetadata.prompt).toBe('bump the limit')
})
it('strips code fences and leaves clickhouse output unformatted', async () => {
captureCompletionRequests('```sql\nlimit 10\n```')
const { result } = renderSqlEditorHook(useAiHarness, {
initialProps: { sqlSource: 'logs' },
})
await act(async () => {
await result.current.ai.handlePrompt('bump the limit', context)
})
await waitFor(() => expect(result.current.diff.isDiffOpen).toBe(true))
// sql-formatter is Postgres-only, so the ClickHouse diff is the reassembled
// query verbatim — fences stripped, nothing else touched.
expect(result.current.diff.sourceSqlDiff).toEqual({
original: `${context.beforeSelection}limit 5`,
modified: `${context.beforeSelection}limit 10`,
})
})
it('still formats database output through sql-formatter', async () => {
captureCompletionRequests('limit 10')
const { result } = renderSqlEditorHook(useAiHarness, {
initialProps: { sqlSource: 'database' },
})
await act(async () => {
await result.current.ai.handlePrompt('bump the limit', context)
})
await waitFor(() => expect(result.current.diff.isDiffOpen).toBe(true))
expect(result.current.diff.sourceSqlDiff?.modified).not.toBe(
`${context.beforeSelection}limit 10`
)
})
})
describe('useSqlEditorAi — debug', () => {
it('onDebug opens the assistant sidebar and starts a debug chat from the failing snippet', async () => {
seedSnippet({ id: SNIPPET_ID, name: 'Broken query', sql: 'selct 1;' })
@@ -4,6 +4,7 @@ import { useCallback, useEffect, useEffectEvent, useMemo, useState } from 'react
import { toast } from 'sonner'
import type { useSqlEditorDiff, useSqlEditorPrompt } from './hooks'
import type { SqlSnippetSource } from './querySource'
import { DiffType, type IStandaloneDiffEditor } from './SQLEditor.types'
import {
assembleCompletionDiff,
@@ -13,12 +14,15 @@ import {
createSqlSnippetSkeletonV2,
extractDebugContext,
planDiffRequestApplication,
sqlSourceToDialect,
} from './SQLEditor.utils'
import { useSQLEditorContext } from './SQLEditorContext'
import { useSnippetTitleGenerator } from './useSnippetTitleGenerator'
import { SIDEBAR_KEYS } from '@/components/layouts/ProjectLayout/LayoutSidebar/LayoutSidebarProvider'
import { constructHeaders } from '@/data/fetchers'
import { stripSqlCodeFences } from '@/data/logs/logs-sql-rewrite'
import { isError } from '@/data/utils/error-check'
import { useLogsAttributeKeys } from '@/hooks/analytics/useLogsAttributeKeys'
import { useSelectedOrganizationQuery } from '@/hooks/misc/useSelectedOrganization'
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
import { BASE_PATH } from '@/lib/constants'
@@ -37,6 +41,11 @@ type UseSqlEditorAiArgs = {
editorMountCount: number
diff: ReturnType<typeof useSqlEditorDiff>
prompt: ReturnType<typeof useSqlEditorPrompt>
/**
* Where the snippet runs. Selects the dialect the AI writes in — logs snippets
* get ClickHouse SQL for the `logs` table, database snippets get Postgres.
*/
sqlSource: SqlSnippetSource
}
/**
@@ -45,7 +54,13 @@ type UseSqlEditorAiArgs = {
* lifecycle effects (one-shot diff-request drain, diff-editor value sync, and the
* ask-AI widget visibility).
*/
export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEditorAiArgs) {
export function useSqlEditorAi({
id,
editorMountCount,
diff,
prompt,
sqlSource,
}: UseSqlEditorAiArgs) {
const {
sourceSqlDiff,
setSourceSqlDiff,
@@ -77,6 +92,14 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
const [isDiffEditorMounted, setIsDiffEditorMounted] = useState(false)
const [showWidget, setShowWidget] = useState(false)
const dialect = sqlSourceToDialect(sqlSource)
const isClickhouse = dialect === 'clickhouse'
// Grounds ClickHouse edits in the source's real log_attributes keys, the same way
// the whole-query rewrite does — otherwise inline edits invent dotted paths.
// Looked up when the user submits, not while they type.
const { fetchAttributeKeys } = useLogsAttributeKeys()
const handleNewQuery = useCallback(
async (sql: string, name: string) => {
if (!ref) return console.error('Project ref is required')
@@ -196,6 +219,7 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
projectRef: project?.ref,
connectionString: project?.connectionString,
orgSlug: org?.slug,
dialect,
options: options?.body,
})
),
@@ -210,9 +234,16 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
const text: string = await response.json()
const meta = options?.body?.completionMetadata ?? {}
const { original, modified } = assembleCompletionDiff(meta, text)
// The clickhouse system prompt forbids fences, but strip them defensively
// so a chatty model can't leak backticks into the snippet.
const { original, modified } = assembleCompletionDiff(
meta,
isClickhouse ? stripSqlCodeFences(text) : text
)
const formattedModified = formatSql(modified)
// sql-formatter is Postgres-only — it mangles ClickHouse backticks and
// map lookups — so ClickHouse output goes into the diff unformatted.
const formattedModified = isClickhouse ? modified : formatSql(modified)
setSourceSqlDiff({ original, modified: formattedModified })
setSelectedDiffType(DiffType.Modification)
setPromptState((prev) => ({ ...prev, isLoading: false }))
@@ -224,6 +255,8 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
}
},
[
dialect,
isClickhouse,
org?.slug,
project?.connectionString,
project?.ref,
@@ -249,10 +282,23 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
beforeSelection: context.beforeSelection,
afterSelection: context.afterSelection,
}))
const headerData = await constructHeaders()
// ClickHouse only: there's no server-side schema to fetch for the logs
// table, so the real log_attributes keys travel with the request. Detected
// from the whole document, which is what the three context fields spell.
const [headerData, availableKeys] = await Promise.all([
constructHeaders(),
isClickhouse
? fetchAttributeKeys(
context.beforeSelection + context.selection + context.afterSelection
)
: undefined,
])
const authorizationHeader = headerData.get('Authorization')
// The instruction goes over as-is for both dialects — the route assembles
// the schema section and the cursor context around it, so there's exactly
// one place that knows how a completion prompt is built.
await complete(prompt, {
...(authorizationHeader
? { headers: { Authorization: authorizationHeader } }
@@ -264,6 +310,7 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
language: 'pgsql',
prompt,
selection: context.selection,
...(availableKeys ? { availableKeys } : {}),
},
},
})
@@ -271,7 +318,7 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
setPromptState((prev) => ({ ...prev, isLoading: false }))
}
},
[complete, setPromptState]
[complete, fetchAttributeKeys, isClickhouse, setPromptState]
)
const handleDiffEditorMount = useCallback(
@@ -0,0 +1,107 @@
import { Button } from 'ui'
import { Admonition } from 'ui-patterns/Admonition'
import { ButtonTooltip } from '@/components/ui/ButtonTooltip'
import type { LegacyLogsRewriteState } from '@/hooks/analytics/useLegacyLogsRewrite'
interface LegacyLogsRewriteAdmonitionProps {
state: LegacyLogsRewriteState
onRewrite: () => void
onDismiss: () => void
}
const BANNER_CLASSES = 'mb-0 rounded-none border-x-0 border-t-0'
/**
* Presentation for the BigQuery → ClickHouse rewrite offer, covering every state
* of `useLegacyLogsRewrite`. Shared by the Logs Explorer and the SQL editor so the
* copy and the outcome handling live in one place — a surface only decides *when*
* to show this, not what each state says.
*
* Renders nothing once dismissed; callers can also hide it earlier if they have
* their own reasons to (the SQL editor hides it while a diff is open).
*/
export const LegacyLogsRewriteAdmonition = ({
state,
onRewrite,
onDismiss,
}: LegacyLogsRewriteAdmonitionProps) => {
if (state.status === 'dismissed') return null
if (state.status === 'failed') {
return (
<Admonition
type="warning"
layout="horizontal"
className={BANNER_CLASSES}
title="Unable to rewrite the query"
description={state.message}
actions={
<div className="flex items-center gap-2">
<Button variant="default" size="tiny" onClick={onRewrite}>
Try again
</Button>
<Button variant="text" size="tiny" onClick={onDismiss}>
Dismiss
</Button>
</div>
}
/>
)
}
// The dialect check is a heuristic, so an unchanged rewrite is the Assistant
// disagreeing with it. Say so and let the user close it, rather than proposing
// an empty diff or silently giving up.
if (state.status === 'noRewriteNeeded') {
return (
<Admonition
type="default"
layout="horizontal"
className={BANNER_CLASSES}
title="No rewrite needed"
description="The Assistant found nothing to change — this query already runs on ClickHouse."
actions={
<Button variant="default" size="tiny" onClick={onDismiss}>
Dismiss
</Button>
}
/>
)
}
const isRewriting = state.status === 'rewriting'
return (
<Admonition
type="default"
layout="horizontal"
className={BANNER_CLASSES}
title="Logs now run on a ClickHouse-backed engine"
description="This query needs to be adjusted to ClickHouse SQL, which the Assistant can do for you."
actions={
<div className="flex items-center gap-2">
<Button variant="default" size="tiny" loading={isRewriting} onClick={onRewrite}>
Rewrite with Assistant
</Button>
{/* Disabled mid-rewrite so the offer can't be dismissed out from under
an in-flight request — the two states stay mutually exclusive. */}
<ButtonTooltip
variant="text"
size="tiny"
disabled={isRewriting}
onClick={onDismiss}
tooltip={{
content: {
side: 'bottom',
text: isRewriting ? 'Available once the rewrite finishes' : undefined,
},
}}
>
Dismiss
</ButtonTooltip>
</div>
}
/>
)
}
@@ -1,34 +0,0 @@
import { Button } from 'ui'
import { Admonition } from 'ui-patterns/Admonition'
interface LogsExplorerOtelBannerProps {
isRewriting: boolean
onRewrite: () => void
onDismiss: () => void
}
export const LogsExplorerOtelBanner = ({
isRewriting,
onRewrite,
onDismiss,
}: LogsExplorerOtelBannerProps) => {
return (
<Admonition
type="default"
layout="horizontal"
className="mb-0 rounded-none border-x-0 border-t-0"
title="Logs now run on a ClickHouse-backed engine"
description="This query needs to be adjusted to ClickHouse SQL, which the Assistant can do for you."
actions={
<div className="flex items-center gap-2">
<Button variant="default" size="tiny" loading={isRewriting} onClick={onRewrite}>
Rewrite with Assistant
</Button>
<Button variant="text" size="tiny" onClick={onDismiss}>
Dismiss
</Button>
</div>
}
/>
)
}
+2
View File
@@ -1,6 +1,8 @@
import { QuerySearchParamsType } from '@/components/interfaces/UnifiedLogs/UnifiedLogs.types'
export const logsKeys = {
otelLogKeys: (projectRef: string | undefined, source: string | undefined) =>
['projects', projectRef, 'otel-log-keys', source] as const,
unifiedLogsInfinite: (
projectRef: string | undefined,
searchParams: QuerySearchParamsType | undefined
@@ -1,45 +1,36 @@
import { afterEach, describe, expect, it, vi } from 'vitest'
import {
buildClickhouseRewritePrompt,
detectLogSource,
looksLikeLegacyLogsQuery,
rewriteLogsSqlWithAI,
shouldOfferLegacyLogsRewrite,
stripSqlCodeFences,
} from './logs-sql-rewrite'
describe('buildClickhouseRewritePrompt', () => {
it('includes the query, the schema, and a reply-with-only-SQL instruction', () => {
const prompt = buildClickhouseRewritePrompt('select count(*) from edge_logs')
expect(prompt).toContain('select count(*) from edge_logs')
expect(prompt).toContain('log_attributes')
expect(prompt).toContain("source = 'edge_logs'")
expect(prompt.toLowerCase()).toContain('reply with only')
describe('shouldOfferLegacyLogsRewrite', () => {
const legacySql = 'select 1 from edge_logs cross join unnest(metadata) as m'
it('offers the rewrite for BigQuery-dialect SQL once logs run on ClickHouse', () => {
expect(shouldOfferLegacyLogsRewrite({ sql: legacySql, isClickhouseLogsEnabled: true })).toBe(
true
)
})
it('spells out the FROM-to-logs conversion and shows a worked example', () => {
const prompt = buildClickhouseRewritePrompt('select 1 from postgres_logs')
expect(prompt).toContain("from logs where source = 'postgres_logs'")
expect(prompt.toLowerCase()).toContain('remove every')
expect(prompt).toContain('cross join unnest')
expect(prompt).toContain('BigQuery:')
expect(prompt).toContain('ClickHouse:')
expect(prompt).toContain("log_attributes['parsed.error_severity']")
it('never offers it on a non-migrated org, where the BigQuery SQL is still correct', () => {
expect(shouldOfferLegacyLogsRewrite({ sql: legacySql, isClickhouseLogsEnabled: false })).toBe(
false
)
})
it('lists the real log_attributes keys when provided and demands exact paths', () => {
const prompt = buildClickhouseRewritePrompt('select 1 from edge_logs', [
'request.headers.x_real_ip',
'request.cf.country',
])
expect(prompt).toContain("log_attributes['request.headers.x_real_ip']")
expect(prompt).toContain("log_attributes['request.cf.country']")
expect(prompt.toLowerCase()).toContain('exact')
})
it('omits the keys section when none are provided', () => {
const prompt = buildClickhouseRewritePrompt('select 1 from edge_logs')
expect(prompt).not.toContain('actual log_attributes keys present')
it('does not offer it for SQL that is already ClickHouse, or for empty SQL', () => {
expect(
shouldOfferLegacyLogsRewrite({
sql: "select timestamp from logs where source = 'edge_logs' limit 5",
isClickhouseLogsEnabled: true,
})
).toBe(false)
expect(shouldOfferLegacyLogsRewrite({ sql: '', isClickhouseLogsEnabled: true })).toBe(false)
})
})
@@ -66,6 +57,21 @@ describe('detectLogSource', () => {
it('returns undefined when nothing matches', () => {
expect(detectLogSource('select 1')).toBeUndefined()
})
it('ignores a column that merely ends in "source"', () => {
expect(detectLogSource("select 1 from logs where resource = 'nope'")).toBeUndefined()
expect(detectLogSource("select 1 from logs where datasource = 'nope'")).toBeUndefined()
})
it('still reads a qualified source column', () => {
expect(detectLogSource("select 1 from logs t where t.source = 'auth_logs'")).toBe('auth_logs')
})
it('prefers the real source column over a lookalike earlier in the query', () => {
expect(detectLogSource("select resource = 'nope' from logs where source = 'edge_logs'")).toBe(
'edge_logs'
)
})
})
describe('looksLikeLegacyLogsQuery', () => {
@@ -112,7 +118,7 @@ describe('rewriteLogsSqlWithAI', () => {
vi.unstubAllGlobals()
})
it('posts to the completion endpoint and returns the cleaned query', async () => {
it('declares the rewrite intent and sends the query as the selection', async () => {
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
json: async () => '```sql\nselect 1 from logs\n```',
@@ -122,6 +128,7 @@ describe('rewriteLogsSqlWithAI', () => {
const result = await rewriteLogsSqlWithAI({
sql: 'select 1 from edge_logs',
projectRef: 'abc',
availableKeys: ['request.method'],
})
expect(result).toBe('select 1 from logs')
@@ -129,8 +136,14 @@ describe('rewriteLogsSqlWithAI', () => {
expect(url).toContain('/api/ai/code/complete')
const body = JSON.parse(init.body)
expect(body.dialect).toBe('clickhouse')
expect(body.intent).toBe('rewrite')
// The whole query is the selection, so the rewrite replaces all of it.
expect(body.completionMetadata.selection).toBe('select 1 from edge_logs')
expect(body.completionMetadata.prompt.toLowerCase()).toContain('reply with only')
expect(body.completionMetadata.textBeforeCursor).toBe('')
expect(body.completionMetadata.textAfterCursor).toBe('')
expect(body.completionMetadata.availableKeys).toEqual(['request.method'])
// No prompt text is carried client-side — the route owns the instruction.
expect(body.completionMetadata.prompt).toBe('')
})
it('throws when the request fails', async () => {
+113
View File
@@ -0,0 +1,113 @@
import { BASE_PATH } from '@/lib/constants'
export function stripSqlCodeFences(text: string): string {
const trimmed = text.trim()
const fenced = trimmed.match(/```(?:sql)?\s*\n?([\s\S]*?)\n?```/i)
return (fenced ? fenced[1] : trimmed).trim()
}
const SOURCE_ALIASES: Record<string, string> = {
pg_cron_logs: 'postgres_logs',
}
export function detectLogSource(sql: string): string | undefined {
// `\b` so only a standalone `source` column counts — an unanchored match reads
// the value out of `resource = '...'` or `datasource = '...'` too.
const bySource = sql.match(/\bsource\s*=\s*'([^']+)'/i)
if (bySource) {
const source = bySource[1].toLowerCase()
return SOURCE_ALIASES[source] ?? source
}
const byFrom = sql.match(/\bfrom\s+([a-z_][a-z0-9_]*)/i)
if (byFrom) {
const table = byFrom[1].toLowerCase()
if (table === 'logs') return undefined
return SOURCE_ALIASES[table] ?? table
}
return undefined
}
export function looksLikeLegacyLogsQuery(sql: string): boolean {
const lower = sql.toLowerCase()
if (/\bunnest\s*\(/.test(lower)) return true
if (/cast\s*\(\s*timestamp\s+as\s+datetime\s*\)/.test(lower)) return true
const byFrom = lower.match(/\bfrom\s+([a-z_][a-z0-9_]*)/)
return byFrom ? byFrom[1] !== 'logs' : false
}
/**
* How long to let the query text settle before re-running the dialect check.
* Shared so every surface offering the rewrite reacts on the same cadence.
*/
export const LEGACY_LOGS_DIALECT_CHECK_DEBOUNCE_MS = 500
/**
* Whether to offer the ClickHouse rewrite for a query. Both the flag and the
* dialect check matter: on an org whose logs haven't moved to ClickHouse the
* BigQuery text is still *correct*, so offering to rewrite it would break a
* working query. Callers layer their own dismissal state on top.
*/
export function shouldOfferLegacyLogsRewrite({
sql,
isClickhouseLogsEnabled,
}: {
sql: string
isClickhouseLogsEnabled: boolean
}): boolean {
return isClickhouseLogsEnabled && looksLikeLegacyLogsQuery(sql)
}
export interface RewriteLogsSqlArgs {
sql: string
projectRef: string
connectionString?: string | null
orgSlug?: string
authorizationHeader?: string | null
availableKeys?: string[]
}
/**
* Asks the completion route to rewrite a whole BigQuery logs query as ClickHouse
* SQL. The prompt itself lives server-side (`lib/ai/clickhouse-logs.ts`) — this
* only declares the intent and hands the query over as the selection, the same
* shape an inline edit uses, so exactly one place knows how a completion prompt
* is assembled.
*/
export async function rewriteLogsSqlWithAI(args: RewriteLogsSqlArgs) {
const { sql, projectRef, connectionString, orgSlug, authorizationHeader, availableKeys } = args
const response = await fetch(`${BASE_PATH}/api/ai/code/complete`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
...(authorizationHeader ? { Authorization: authorizationHeader } : {}),
},
body: JSON.stringify({
projectRef,
connectionString,
language: 'sql',
dialect: 'clickhouse',
intent: 'rewrite',
orgSlug,
completionMetadata: {
// The whole query is the selection, so the rewrite replaces all of it and
// the route supplies the instruction for the `rewrite` intent.
textBeforeCursor: '',
textAfterCursor: '',
prompt: '',
selection: sql,
availableKeys,
},
}),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(errorText || 'Failed to rewrite the query')
}
const raw = await response.json()
const rewritten = stripSqlCodeFences(typeof raw === 'string' ? raw : String(raw))
if (!rewritten) throw new Error('The assistant returned an empty query')
return rewritten
}
+24 -5
View File
@@ -1,11 +1,14 @@
import { useQuery } from '@tanstack/react-query'
import { queryOptions, useQuery } from '@tanstack/react-query'
import { executeAnalyticsSql } from './execute-analytics-sql'
import { logsKeys } from './keys'
import { logsAllEndpointUrl } from './logs-endpoint'
import { analyticsLiteral, safeSql } from './safe-analytics-sql'
const LOOKBACK_HOURS = 24 * 7
const KEYS_STALE_TIME = 5 * 60 * 1000
export async function fetchOtelLogKeys({
projectRef,
source,
@@ -31,15 +34,31 @@ export async function fetchOtelLogKeys({
return rows.map((r) => r.key).filter(Boolean)
}
/**
* Shared by the reactive hook and imperative `queryClient.fetchQuery` callers, so
* a lookup triggered on submit reuses whatever a subscribed component already
* cached for the same source (and vice versa).
*/
export function otelLogKeysQueryOptions({
projectRef,
source,
}: {
projectRef: string
source: string
}) {
return queryOptions({
queryKey: logsKeys.otelLogKeys(projectRef, source),
queryFn: ({ signal }) => fetchOtelLogKeys({ projectRef, source, signal }),
staleTime: KEYS_STALE_TIME,
})
}
export function useOtelLogKeysQuery(
{ projectRef, source }: { projectRef?: string; source?: string },
{ enabled = true }: { enabled?: boolean } = {}
) {
return useQuery({
queryKey: ['projects', projectRef, 'otel-log-keys', source],
queryFn: ({ signal }) =>
fetchOtelLogKeys({ projectRef: projectRef ?? '', source: source ?? '', signal }),
...otelLogKeysQueryOptions({ projectRef: projectRef ?? '', source: source ?? '' }),
enabled: enabled && Boolean(projectRef) && Boolean(source),
staleTime: 5 * 60 * 1000,
})
}
@@ -0,0 +1,163 @@
import { act, waitFor } from '@testing-library/react'
import { delay, http, HttpResponse } from 'msw'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import {
INITIAL_LEGACY_LOGS_REWRITE_STATE,
legacyLogsRewriteReducer as reduce,
useLegacyLogsRewrite,
type LegacyLogsRewriteEvent,
type LegacyLogsRewriteState,
} from './useLegacyLogsRewrite'
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
import { API_URL } from '@/lib/constants'
import { mswServer } from '@/tests/lib/msw'
import { renderSqlEditorHook, setupSqlEditorMocks } from '@/tests/lib/sql-editor-test-utils'
const run = (
events: LegacyLogsRewriteEvent[],
from: LegacyLogsRewriteState = INITIAL_LEGACY_LOGS_REWRITE_STATE
) => events.reduce(reduce, from)
const FAILED: LegacyLogsRewriteEvent = { type: 'rewriteFailed', message: 'boom' }
describe('legacyLogsRewriteReducer', () => {
it('starts out offering the rewrite', () => {
expect(INITIAL_LEGACY_LOGS_REWRITE_STATE).toEqual({ status: 'offered' })
})
it('requesting a rewrite moves to rewriting, and a proposal returns to offered', () => {
expect(run([{ type: 'rewriteRequested' }])).toEqual({ status: 'rewriting' })
expect(run([{ type: 'rewriteRequested' }, { type: 'rewriteProposed' }])).toEqual({
status: 'offered',
})
})
it('a failure lands in failed and keeps its message for the UI', () => {
expect(run([{ type: 'rewriteRequested' }, FAILED])).toEqual({
status: 'failed',
message: 'boom',
})
})
it('a failure is recoverable — the same request retries it', () => {
expect(run([{ type: 'rewriteRequested' }, FAILED, { type: 'rewriteRequested' }])).toEqual({
status: 'rewriting',
})
})
it('an unchanged response waits for acknowledgement instead of retiring silently', () => {
const noop = run([{ type: 'rewriteRequested' }, { type: 'rewriteNoop' }])
expect(noop).toEqual({ status: 'noRewriteNeeded' })
expect(run([{ type: 'dismissed' }], noop)).toEqual({ status: 'dismissed' })
})
it('both outcomes can be dismissed, and neither can be retried into a new outcome', () => {
const failed = run([{ type: 'rewriteRequested' }, FAILED])
expect(run([{ type: 'dismissed' }], failed)).toEqual({ status: 'dismissed' })
// noRewriteNeeded only accepts dismissal — no silent retry.
const noop = run([{ type: 'rewriteRequested' }, { type: 'rewriteNoop' }])
expect(run([{ type: 'rewriteRequested' }], noop)).toEqual({ status: 'noRewriteNeeded' })
})
it('dismissal is terminal — nothing resurrects the offer', () => {
const dismissed = run([{ type: 'dismissed' }])
expect(dismissed).toEqual({ status: 'dismissed' })
expect(
run(
[
{ type: 'rewriteRequested' },
{ type: 'rewriteProposed' },
FAILED,
{ type: 'rewriteNoop' },
],
dismissed
)
).toEqual({ status: 'dismissed' })
})
it('cannot be dismissed mid-rewrite, so a settling request never resurrects it', () => {
expect(run([{ type: 'rewriteRequested' }, { type: 'dismissed' }])).toEqual({
status: 'rewriting',
})
})
it('ignores events that are invalid for the current state', () => {
// No rewrite in flight to settle.
expect(run([{ type: 'rewriteProposed' }])).toEqual({ status: 'offered' })
expect(run([{ type: 'rewriteNoop' }])).toEqual({ status: 'offered' })
expect(run([FAILED])).toEqual({ status: 'offered' })
// Already rewriting; a second request is a no-op rather than a restart.
expect(run([{ type: 'rewriteRequested' }, { type: 'rewriteRequested' }])).toEqual({
status: 'rewriting',
})
})
})
describe('useLegacyLogsRewrite — dismiss', () => {
// No detectable source, so key discovery is skipped and the only outbound
// request is the completion call we control below.
const SQL_WITHOUT_SOURCE = 'select 1 from logs limit 5'
/** Keeps a requested rewrite in flight for the duration of the test. */
function stallTheRewrite() {
mswServer.use(
http.post(`${API_URL}/ai/code/complete`, async () => {
await delay(10_000)
return HttpResponse.json('select 1 from logs')
})
)
}
/**
* Exposes the resolved project alongside the hook: `requestRewrite` no-ops
* without a project ref, so tests must wait for that query before asking.
*/
async function renderDismissHarness() {
const onDismissed = vi.fn()
const utils = renderSqlEditorHook(() => {
const { data: project } = useSelectedProjectQuery()
const rewrite = useLegacyLogsRewrite({
readSql: () => SQL_WITHOUT_SOURCE,
onProposal: vi.fn(),
onDismissed,
})
return { ...rewrite, projectRef: project?.ref }
})
await waitFor(() => expect(utils.result.current.projectRef).toBe('default'))
return { ...utils, onDismissed }
}
beforeEach(() => {
setupSqlEditorMocks()
})
it('dismisses from the offer and reports it', async () => {
const { result, onDismissed } = await renderDismissHarness()
await act(async () => {
result.current.dismiss()
})
expect(result.current.state.status).toBe('dismissed')
expect(onDismissed).toHaveBeenCalledTimes(1)
})
it('does not report a dismissal the machine rejects mid-rewrite', async () => {
stallTheRewrite()
const { result, onDismissed } = await renderDismissHarness()
act(() => {
void result.current.requestRewrite()
})
await waitFor(() => expect(result.current.state.status).toBe('rewriting'))
await act(async () => {
result.current.dismiss()
})
// Persisting this would suppress an offer that's still live.
expect(onDismissed).not.toHaveBeenCalled()
expect(result.current.state.status).toBe('rewriting')
})
})
@@ -0,0 +1,162 @@
import { useReducer } from 'react'
import { constructHeaders } from '@/data/fetchers'
import { rewriteLogsSqlWithAI } from '@/data/logs/logs-sql-rewrite'
import { useLogsAttributeKeys } from '@/hooks/analytics/useLogsAttributeKeys'
import { useSelectedOrganizationQuery } from '@/hooks/misc/useSelectedOrganization'
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
import { getErrorMessage } from '@/lib/get-error-message'
export type LegacyLogsRewriteState =
| { status: 'offered' }
| { status: 'rewriting' }
| { status: 'failed'; message: string }
| { status: 'noRewriteNeeded' }
| { status: 'dismissed' }
export type LegacyLogsRewriteEvent =
| { type: 'rewriteRequested' }
| { type: 'rewriteProposed' }
| { type: 'rewriteFailed'; message: string }
| { type: 'rewriteNoop' }
| { type: 'dismissed' }
export const INITIAL_LEGACY_LOGS_REWRITE_STATE: LegacyLogsRewriteState = { status: 'offered' }
/**
* The events each state accepts. Anything absent is an invalid transition and
* leaves the state untouched — notably `dismissed` is terminal, and the offer
* can't be dismissed mid-rewrite.
*/
const VALID_EVENTS: {
[S in LegacyLogsRewriteState['status']]: readonly LegacyLogsRewriteEvent['type'][]
} = {
offered: ['rewriteRequested', 'dismissed'],
rewriting: ['rewriteProposed', 'rewriteFailed', 'rewriteNoop'],
// A failure is recoverable: the same Rewrite control retries it.
failed: ['rewriteRequested', 'dismissed'],
noRewriteNeeded: ['dismissed'],
dismissed: [],
}
function targetState(event: LegacyLogsRewriteEvent): LegacyLogsRewriteState {
switch (event.type) {
case 'rewriteRequested':
return { status: 'rewriting' }
// The proposal is handed to the caller; the offer returns to idle behind it so
// it's ready again if the user discards the proposal.
case 'rewriteProposed':
return { status: 'offered' }
case 'rewriteFailed':
return { status: 'failed', message: event.message }
case 'rewriteNoop':
return { status: 'noRewriteNeeded' }
case 'dismissed':
return { status: 'dismissed' }
}
}
export function legacyLogsRewriteReducer(
state: LegacyLogsRewriteState,
event: LegacyLogsRewriteEvent
): LegacyLogsRewriteState {
if (!VALID_EVENTS[state.status].includes(event.type)) return state
return targetState(event)
}
const CHANGED_WHILE_REWRITING_MESSAGE =
'The query changed while the Assistant was working, so the rewrite no longer matches it.'
const NO_RESPONSE_MESSAGE = 'The Assistant did not respond. Try again.'
export type LegacyLogsRewriteProposal = { original: string; modified: string }
type UseLegacyLogsRewriteArgs = {
/**
* Reads the query to rewrite at the moment the user asks. A callback rather than
* a value so the rewrite operates on exactly what the user sees, not on whatever
* a surface last rendered.
*/
readSql: () => string
/** Receives a rewrite worth reviewing. Each surface routes this to its own diff. */
onProposal: (proposal: LegacyLogsRewriteProposal) => void
/**
* Called when the offer is dismissed, for surfaces that persist that. The
* machine covers the current session only — a surface that remembers dismissals
* across sessions layers that on top of its own visibility check, since a value
* read from storage isn't available in time to seed the machine.
*/
onDismissed?: () => void
}
/**
* Owns the BigQuery → ClickHouse rewrite request end to end: key discovery, the
* completion call, the stale-edit guard, no-op detection, and the resulting state.
*/
export function useLegacyLogsRewrite({
readSql,
onProposal,
onDismissed,
}: UseLegacyLogsRewriteArgs) {
const { data: project } = useSelectedProjectQuery()
const { data: organization } = useSelectedOrganizationQuery()
const projectRef = project?.ref
const [state, dispatch] = useReducer(legacyLogsRewriteReducer, INITIAL_LEGACY_LOGS_REWRITE_STATE)
const { fetchAttributeKeys } = useLogsAttributeKeys()
const requestRewrite = async () => {
if (!projectRef) return console.error('[useLegacyLogsRewrite] Project ref is required')
const currentSql = readSql()
if (currentSql.trim().length === 0) return
dispatch({ type: 'rewriteRequested' })
try {
const [headerData, availableKeys] = await Promise.all([
constructHeaders(),
fetchAttributeKeys(currentSql),
])
const rewritten = await rewriteLogsSqlWithAI({
sql: currentSql,
projectRef,
connectionString: project?.connectionString,
orgSlug: organization?.slug,
authorizationHeader: headerData.get('Authorization'),
availableKeys,
})
// The user may have kept typing while the model worked; a proposal built from
// stale text would clobber those edits when accepted.
if (readSql() !== currentSql) {
dispatch({ type: 'rewriteFailed', message: CHANGED_WHILE_REWRITING_MESSAGE })
return
}
// An unchanged response means the query already runs on ClickHouse and the
// dialect heuristic was over-eager. Proposing it would show an empty diff.
if (rewritten.trim() === currentSql.trim()) {
dispatch({ type: 'rewriteNoop' })
return
}
onProposal({ original: currentSql, modified: rewritten })
dispatch({ type: 'rewriteProposed' })
} catch (error) {
dispatch({ type: 'rewriteFailed', message: getErrorMessage(error, NO_RESPONSE_MESSAGE) })
}
}
const dismiss = () => {
const dismissed: LegacyLogsRewriteEvent = { type: 'dismissed' }
// The transition table is the contract, not the UI that happens to disable the
// control: never report a dismissal the machine rejected (mid-rewrite, say),
// or a surface that persists it would suppress an offer that's still live.
if (legacyLogsRewriteReducer(state, dismissed) === state) return
dispatch(dismissed)
onDismissed?.()
}
return { state, requestRewrite, dismiss }
}
@@ -0,0 +1,124 @@
import { act, waitFor } from '@testing-library/react'
import { HttpResponse } from 'msw'
import { beforeEach, describe, expect, it } from 'vitest'
import { useLogsAttributeKeys } from './useLogsAttributeKeys'
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
import { addAPIMock } from '@/tests/lib/msw'
import { renderSqlEditorHook, setupSqlEditorMocks } from '@/tests/lib/sql-editor-test-utils'
const OTEL_ENDPOINT = '/platform/projects/:ref/analytics/endpoints/logs.all.otel'
const queryFor = (source: string) => `select 1 from logs where source = '${source}'`
/** Records the SQL of every key-discovery request so we can count them. */
function mockKeyDiscovery({ fails = false }: { fails?: boolean } = {}) {
const requests: string[] = []
addAPIMock({
method: 'post',
path: OTEL_ENDPOINT,
response: async ({ request }) => {
const body = (await request.clone().json()) as { sql: string }
requests.push(body.sql)
if (fails) return HttpResponse.json({ message: 'boom' }, { status: 500 })
return HttpResponse.json({ result: [{ key: 'request.method' }] })
},
})
return requests
}
/**
* Exposes the resolved project alongside the hook. Discovery needs a project ref,
* so tests must wait for that query before asking — otherwise a lookup no-ops and
* the request counts below would pass for the wrong reason.
*/
function useKeysHarness() {
const { data: project } = useSelectedProjectQuery()
const { fetchAttributeKeys } = useLogsAttributeKeys()
return { projectRef: project?.ref, fetchAttributeKeys }
}
async function renderReadyHarness() {
const utils = renderSqlEditorHook(useKeysHarness)
await waitFor(() => expect(utils.result.current.projectRef).toBe('default'))
return utils
}
beforeEach(() => {
setupSqlEditorMocks()
})
describe('useLogsAttributeKeys', () => {
it('makes no request until asked', async () => {
const requests = mockKeyDiscovery()
await renderReadyHarness()
expect(requests).toHaveLength(0)
})
it('returns the discovered keys for the query source when asked', async () => {
const requests = mockKeyDiscovery()
const { result } = await renderReadyHarness()
let keys: string[] | undefined
await act(async () => {
keys = await result.current.fetchAttributeKeys(queryFor('edge_logs'))
})
expect(keys).toEqual(['request.method'])
expect(requests).toHaveLength(1)
expect(requests[0]).toContain("source = 'edge_logs'")
})
it('reuses the cached result for a source already looked up', async () => {
const requests = mockKeyDiscovery()
const { result } = await renderReadyHarness()
await act(async () => {
await result.current.fetchAttributeKeys(queryFor('edge_logs'))
await result.current.fetchAttributeKeys(queryFor('edge_logs'))
})
expect(requests).toHaveLength(1)
})
it('looks up a different source separately', async () => {
const requests = mockKeyDiscovery()
const { result } = await renderReadyHarness()
await act(async () => {
await result.current.fetchAttributeKeys(queryFor('edge_logs'))
await result.current.fetchAttributeKeys(queryFor('postgres_logs'))
})
expect(requests).toHaveLength(2)
expect(requests[1]).toContain("source = 'postgres_logs'")
})
it('resolves undefined without a request when no source is detectable', async () => {
const requests = mockKeyDiscovery()
const { result } = await renderReadyHarness()
let keys: string[] | undefined
await act(async () => {
keys = await result.current.fetchAttributeKeys('select 1 from logs limit 5')
})
expect(keys).toBeUndefined()
expect(requests).toHaveLength(0)
})
it('resolves undefined rather than throwing when discovery fails', async () => {
mockKeyDiscovery({ fails: true })
const { result } = await renderReadyHarness()
let keys: string[] | undefined
await act(async () => {
keys = await result.current.fetchAttributeKeys(queryFor('edge_logs'))
})
// Keys are an enhancement — a failed lookup must not block the caller.
expect(keys).toBeUndefined()
})
})
@@ -0,0 +1,43 @@
import { useQueryClient } from '@tanstack/react-query'
import { useCallback } from 'react'
import { detectLogSource } from '@/data/logs/logs-sql-rewrite'
import { otelLogKeysQueryOptions } from '@/data/logs/otel-log-keys-query'
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
/**
* Looks up the real `log_attributes` keys for whichever source a logs query
* targets. The AI flows pass these along so the model uses exact dotted paths
* instead of inventing them.
*
* Deliberately imperative: discovery aggregates a week of logs, and the source is
* derived from query text the user is editing, so anything reactive fires requests
* for half-typed source names. Fetching at submit time means one request per
* action the user actually took. It still goes through the query client, so a
* result already cached for that source (by an earlier submit, or by a component
* subscribed via `useOtelLogKeysQuery`) is reused rather than refetched.
*
* Keys are an enhancement, never a requirement — a failed or impossible lookup
* resolves to `undefined` and the caller proceeds without them.
*/
export function useLogsAttributeKeys() {
const queryClient = useQueryClient()
const { data: project } = useSelectedProjectQuery()
const projectRef = project?.ref
const fetchAttributeKeys = useCallback(
async (sql: string): Promise<string[] | undefined> => {
const source = detectLogSource(sql)
if (!projectRef || source === undefined) return undefined
try {
return await queryClient.fetchQuery(otelLogKeysQueryOptions({ projectRef, source }))
} catch {
return undefined
}
},
[projectRef, queryClient]
)
return { fetchAttributeKeys }
}
@@ -1,12 +1,34 @@
import { BASE_PATH } from '@/lib/constants'
/**
* Everything the model needs to know about the ClickHouse-backed Supabase `logs`
* table, in one place. Both ClickHouse completion flows — the inline "edit this
* query" path and the whole-query BigQuery rewrite — are assembled from these,
* server-side in `pages/api/ai/code/complete.ts`, so there is exactly one
* description of the schema to keep current.
*/
export const LOGS_SCHEMA_REFERENCE = `The logs table (ClickHouse) has these columns:
/** System-prompt half: the dialect rules that hold for every ClickHouse request. */
export const CLICKHOUSE_LOGS_COMPLETION_INSTRUCTIONS = `
# Supabase logs SQL (ClickHouse)
You are writing SQL for Supabase logs, which run on a ClickHouse-backed engine. This is NOT Postgres and NOT BigQuery. Output valid ClickHouse SQL only.
- All logs are in a single table named \`logs\`, keyed by a \`source\` column. There are no per-service tables (no \`edge_logs\`, \`postgres_logs\`, and so on) and no \`unnest\` joins.
- Per-source fields live in the \`log_attributes\` Map(String, String), read as \`log_attributes['key']\`. Map values are strings, so wrap numeric ones in \`toInt32OrZero(...)\`.
- Use ClickHouse functions, not Postgres or BigQuery ones. Use \`match(col, 'regex')\` or \`col ILIKE '%text%'\` instead of \`regexp_contains\`, \`count()\` instead of \`count(*)\`, and select the \`timestamp\` column directly instead of \`cast(timestamp as datetime)\`.
- Do not quote identifiers with double quotes and do not append a trailing semicolon.
- Do not use \`select *\`, this is disallowed by the backend.
`
/**
* User-message half: the concrete shape of the table. Complements the dialect
* rules above — this is the part that changes when a log source or its fields
* change, and it is the ClickHouse counterpart to the Postgres DDL section.
*/
const CLICKHOUSE_LOGS_COLUMN_REFERENCE = `The logs table has these columns:
- id (String)
- timestamp (DateTime64, UTC) formatted like 2026-06-22T09:34:06.215000 (ISO 8601, microsecond precision, no trailing Z)
- event_message (String): the raw log line
- severity_text (String): log level when present
- source (String): the service the log belongs to. Always filter by it, e.g. where source = 'edge_logs'.
- log_attributes (Map(String, String)): structured per-source fields, read as log_attributes['key']. Values are strings, so wrap numeric ones in toInt32OrZero(...) for comparisons.
- log_attributes (Map(String, String)): structured per-source fields, read as log_attributes['key']
Sources and their common log_attributes keys:
- edge_logs: request.method, request.path, request.search, response.status_code, identifier
@@ -17,7 +39,7 @@ Sources and their common log_attributes keys:
- function_logs: event_type, function_id, execution_id, level
- storage_logs, realtime_logs, postgrest_logs, supavisor_logs, pgbouncer_logs: mostly id, timestamp, event_message, with extra fields in log_attributes
Rules: always filter by source; the editor applies the selected time range so a timestamp filter is usually unnecessary; the old BigQuery unnest joins become log_attributes['key'] lookups (drop the metadata root).`
The editor applies the user's selected time range as a request parameter, so an explicit timestamp filter is usually unnecessary.`
function renderAvailableKeys(availableKeys?: string[]): string {
if (!availableKeys || availableKeys.length === 0) return ''
@@ -26,10 +48,25 @@ function renderAvailableKeys(availableKeys?: string[]): string {
${list}\n`
}
export function buildClickhouseRewritePrompt(sql: string, availableKeys?: string[]): string {
return `${LOGS_SCHEMA_REFERENCE}
${renderAvailableKeys(availableKeys)}
Convert the BigQuery logs query below to ClickHouse SQL for the logs table. There are no per-service tables and no unnest joins in ClickHouse. Follow these rules exactly:
/**
* The ClickHouse schema section of the user message — the counterpart to
* `buildDatabaseSchemaSection` for Postgres. `availableKeys` are the real
* `log_attributes` keys observed for the query's source, when the caller
* discovered them.
*/
export function buildClickhouseLogsSchemaSection(availableKeys?: string[]): string {
return `${CLICKHOUSE_LOGS_COLUMN_REFERENCE}\n${renderAvailableKeys(availableKeys)}`
}
/**
* The instruction for the whole-query BigQuery → ClickHouse rewrite. Used in
* place of a user instruction when the request's intent is `rewrite`.
*
* It states that a rewrite is REQUIRED: the system prompt covers writing and
* editing ClickHouse SQL generally, and without an explicit demand here the model
* echoes the input back, which surfaces to the user as an empty diff.
*/
export const CLICKHOUSE_LOGS_REWRITE_INSTRUCTION = `Your task is to REWRITE the selected query. It is BigQuery SQL and will not run on ClickHouse, so returning it unchanged is likely wrong — every rule below that applies must be applied.
1. Replace the FROM table with the single logs table and filter by source. The old table name is the source value: "from postgres_logs as t" becomes "from logs where source = 'postgres_logs'". This is required, never select from a table like postgres_logs or edge_logs.
2. Remove every join that unnests metadata or its structs. This includes "cross join unnest(...)" and "left join unnest(...) on true".
@@ -56,87 +93,4 @@ where source = 'postgres_logs'
and log_attributes['parsed.error_severity'] in ('ERROR', 'FATAL', 'PANIC')
group by log_attributes['parsed.error_severity']
order by count desc
limit 100
Reply with ONLY the rewritten SQL query: no explanation, no comments, and no markdown code fences.
${sql}`
}
export function stripSqlCodeFences(text: string): string {
const trimmed = text.trim()
const fenced = trimmed.match(/```(?:sql)?\s*\n?([\s\S]*?)\n?```/i)
return (fenced ? fenced[1] : trimmed).trim()
}
const SOURCE_ALIASES: Record<string, string> = {
pg_cron_logs: 'postgres_logs',
}
export function detectLogSource(sql: string): string | undefined {
const bySource = sql.match(/source\s*=\s*'([^']+)'/i)
if (bySource) {
const source = bySource[1].toLowerCase()
return SOURCE_ALIASES[source] ?? source
}
const byFrom = sql.match(/\bfrom\s+([a-z_][a-z0-9_]*)/i)
if (byFrom) {
const table = byFrom[1].toLowerCase()
if (table === 'logs') return undefined
return SOURCE_ALIASES[table] ?? table
}
return undefined
}
export function looksLikeLegacyLogsQuery(sql: string): boolean {
const lower = sql.toLowerCase()
if (/\bunnest\s*\(/.test(lower)) return true
if (/cast\s*\(\s*timestamp\s+as\s+datetime\s*\)/.test(lower)) return true
const byFrom = lower.match(/\bfrom\s+([a-z_][a-z0-9_]*)/)
return byFrom ? byFrom[1] !== 'logs' : false
}
export interface RewriteLogsSqlArgs {
sql: string
projectRef: string
connectionString?: string | null
orgSlug?: string
authorizationHeader?: string | null
availableKeys?: string[]
}
export async function rewriteLogsSqlWithAI(args: RewriteLogsSqlArgs) {
const { sql, projectRef, connectionString, orgSlug, authorizationHeader, availableKeys } = args
const response = await fetch(`${BASE_PATH}/api/ai/code/complete`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
...(authorizationHeader ? { Authorization: authorizationHeader } : {}),
},
body: JSON.stringify({
projectRef,
connectionString,
language: 'sql',
dialect: 'clickhouse',
orgSlug,
completionMetadata: {
textBeforeCursor: '',
textAfterCursor: '',
language: 'pgsql',
prompt: buildClickhouseRewritePrompt(sql, availableKeys),
selection: sql,
},
}),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(errorText || 'Failed to rewrite the query')
}
const raw = await response.json()
const rewritten = stripSqlCodeFences(typeof raw === 'string' ? raw : String(raw))
if (!rewritten) throw new Error('The assistant returned an empty query')
return rewritten
}
limit 100`
-9
View File
@@ -786,15 +786,6 @@ export const SQL_COMPLETION_INSTRUCTIONS = `
Do not quote identifiers unless they actually require it (uppercase letters, reserved words, or special characters). Plain lowercase identifiers should not be quoted.
`
export const CLICKHOUSE_LOGS_COMPLETION_INSTRUCTIONS = `
# Supabase logs SQL (ClickHouse)
You are writing SQL for Supabase logs, which run on a ClickHouse-backed engine. This is NOT Postgres and NOT BigQuery. Output valid ClickHouse SQL only.
- All logs are in a single table named \`logs\`, keyed by a \`source\` column. There are no per-service tables (no \`edge_logs\`, \`postgres_logs\`, and so on) and no \`unnest\` joins.
- Per-source fields live in the \`log_attributes\` Map(String, String), read as \`log_attributes['key']\`. Map values are strings, so wrap numeric ones in \`toInt32OrZero(...)\`.
- Use ClickHouse functions, not Postgres or BigQuery ones. Use \`match(col, 'regex')\` or \`col ILIKE '%text%'\` instead of \`regexp_contains\`, \`count()\` instead of \`count(*)\`, and select the \`timestamp\` column directly instead of \`cast(timestamp as datetime)\`.
- Do not quote identifiers with double quotes and do not append a trailing semicolon.
`
export const LIMITATIONS_PROMPT = `
# Limitations
- You are to only answer Supabase, database, or edge function related questions. All other questions should be declined with a polite message.
+31 -27
View File
@@ -3,44 +3,48 @@ import { describe, expect, it } from 'vitest'
import { getErrorMessage } from './get-error-message'
describe('getErrorMessage', () => {
it('returns null for null', () => {
expect(getErrorMessage(null)).toBe(null)
})
it('returns null for undefined', () => {
expect(getErrorMessage(undefined)).toBe(null)
})
it('returns the string for string errors', () => {
expect(getErrorMessage('Something went wrong')).toBe('Something went wrong')
expect(getErrorMessage('')).toBe('')
})
it('returns the message for Error instances', () => {
expect(getErrorMessage(new Error('Failed to load'))).toBe('Failed to load')
expect(getErrorMessage(new TypeError('Invalid type'))).toBe('Invalid type')
})
it('returns the message property for objects with message', () => {
it('returns a string throw, trimmed', () => {
expect(getErrorMessage('Something went wrong')).toBe('Something went wrong')
expect(getErrorMessage(' boom ')).toBe('boom')
})
it('reads a string message off a plain object', () => {
expect(getErrorMessage({ message: 'Custom error' })).toBe('Custom error')
expect(getErrorMessage({ message: 123 })).toBe('123')
expect(getErrorMessage({ message: null })).toBe('null')
})
it('converts other types to string', () => {
expect(getErrorMessage(123)).toBe('123')
expect(getErrorMessage(true)).toBe('true')
expect(getErrorMessage(false)).toBe('false')
expect(getErrorMessage({})).toBe('[object Object]')
expect(getErrorMessage([])).toBe('')
it('returns null when there is no usable message', () => {
expect(getErrorMessage(null)).toBe(null)
expect(getErrorMessage(undefined)).toBe(null)
expect(getErrorMessage('')).toBe(null)
expect(getErrorMessage(' ')).toBe(null)
expect(getErrorMessage(123)).toBe(null)
expect(getErrorMessage(true)).toBe(null)
expect(getErrorMessage([])).toBe(null)
})
it('handles objects without message property', () => {
expect(getErrorMessage({ code: 500 })).toBe('[object Object]')
expect(getErrorMessage({ error: 'test' })).toBe('[object Object]')
it('never surfaces a stringified object as the message', () => {
expect(getErrorMessage({})).toBe(null)
expect(getErrorMessage({ code: 500 })).toBe(null)
expect(getErrorMessage({ error: 'test' })).toBe(null)
expect(getErrorMessage({ message: 123 })).toBe(null)
expect(getErrorMessage({ message: null })).toBe(null)
expect(getErrorMessage({ message: { nested: 'error' } })).toBe(null)
})
it('handles nested error objects', () => {
expect(getErrorMessage({ message: { nested: 'error' } })).toBe('[object Object]')
it('returns the fallback instead of null when one is given', () => {
expect(getErrorMessage(null, 'fallback')).toBe('fallback')
expect(getErrorMessage({}, 'fallback')).toBe('fallback')
expect(getErrorMessage({ message: ' ' }, 'fallback')).toBe('fallback')
expect(getErrorMessage(123, 'fallback')).toBe('fallback')
})
it('prefers a real message over the fallback', () => {
expect(getErrorMessage(new Error('Rewrite failed'), 'fallback')).toBe('Rewrite failed')
expect(getErrorMessage({ message: 'Bad request' }, 'fallback')).toBe('Bad request')
})
})
+18 -8
View File
@@ -1,12 +1,22 @@
/**
* Extracts a human-readable error message from various error types.
* Extracts a human-readable message from an unknown thrown value, optionally
* falling back when there isn't one.
*/
export function getErrorMessage(error: unknown): string | null {
if (error === null || error === undefined) return null
if (typeof error === 'string') return error
if (error instanceof Error) return error.message
if (typeof error === 'object' && error !== null && 'message' in error) {
return String(error.message)
export function getErrorMessage(error: unknown): string | null
export function getErrorMessage(error: unknown, fallback: string): string
export function getErrorMessage(error: unknown, fallback?: string): string | null {
if (typeof error === 'string') {
const trimmed = error.trim()
if (trimmed.length > 0) return trimmed
}
return String(error)
if (typeof error === 'object' && error !== null && 'message' in error) {
const { message } = error
if (typeof message === 'string') {
const trimmed = message.trim()
if (trimmed.length > 0) return trimmed
}
}
return fallback ?? null
}
+44 -17
View File
@@ -8,10 +8,14 @@ import z from 'zod'
import { executeSql } from '@/data/sql/execute-sql-mutation'
import { AiOptInLevel } from '@/hooks/misc/useOrgOptedIntoAi'
import { getOrgAIDetails } from '@/lib/ai/ai-details'
import {
buildClickhouseLogsSchemaSection,
CLICKHOUSE_LOGS_COMPLETION_INSTRUCTIONS,
CLICKHOUSE_LOGS_REWRITE_INSTRUCTION,
} from '@/lib/ai/clickhouse-logs'
import { getModel } from '@/lib/ai/model'
import { DEFAULT_COMPLETION_MODEL, LOGS_REWRITE_MODEL } from '@/lib/ai/model.utils'
import {
CLICKHOUSE_LOGS_COMPLETION_INSTRUCTIONS,
COMPLETION_PROMPT,
EDGE_FUNCTION_PROMPT,
PG_BEST_PRACTICES,
@@ -124,12 +128,24 @@ const requestBodySchema = z.object({
textAfterCursor: z.string(),
prompt: z.string(),
selection: z.string(),
/**
* The real `log_attributes` keys observed for the query's source, when the
* client discovered them. ClickHouse-only — there is no schema to fetch
* server-side for the logs table the way there is for Postgres DDL.
*/
availableKeys: z.array(z.string()).optional(),
}),
projectRef: z.string(),
connectionString: z.string().nullish(),
orgSlug: z.string().optional(),
language: z.string().optional(),
dialect: z.enum(['postgres', 'clickhouse']).optional(),
/**
* What the caller wants done. `rewrite` swaps the user instruction for the
* canonical BigQuery → ClickHouse rewrite instruction, so the client never has
* to carry prompt text. ClickHouse-only; defaults to `edit`.
*/
intent: z.enum(['edit', 'rewrite']).optional(),
})
async function handler(req: NextApiRequest, res: NextApiResponse) {
@@ -150,8 +166,10 @@ async function handler(req: NextApiRequest, res: NextApiResponse) {
return res.status(400).json({ error: 'Invalid request body', issues: parseError.issues })
}
const { completionMetadata, projectRef, connectionString, orgSlug, language, dialect } = data
const { textBeforeCursor, textAfterCursor, prompt, selection } = completionMetadata
const { completionMetadata, projectRef, connectionString, orgSlug, language, dialect, intent } =
data
const { textBeforeCursor, textAfterCursor, prompt, selection, availableKeys } =
completionMetadata
const isClickhouse = dialect === 'clickhouse'
const authorization = req.headers.authorization
@@ -232,8 +250,9 @@ async function handler(req: NextApiRequest, res: NextApiResponse) {
const system = isClickhouse
? source`
You rewrite SQL queries to ClickHouse SQL for the Supabase logs table.
Output only the rewritten SQL query: no explanation, no markdown, and no code fences.
You write and edit ClickHouse SQL for the Supabase logs table.
Reply with ONLY the SQL that replaces the <selection> block below, keeping the
surrounding query valid: no explanation, no comments, and no markdown code fences.
${CLICKHOUSE_LOGS_COMPLETION_INSTRUCTIONS}
${SECURITY_PROMPT}
`
@@ -243,23 +262,31 @@ async function handler(req: NextApiRequest, res: NextApiResponse) {
${SECURITY_PROMPT}
`
const userMessage = isClickhouse
? prompt
: source`
## Database Schema
const schemaSection = isClickhouse
? { heading: 'Logs Schema', body: buildClickhouseLogsSchemaSection(availableKeys) }
: {
heading: 'Database Schema',
body: buildDatabaseSchemaSection({ includeSchema, schemaListResult, schemaDDLResult }),
}
${buildDatabaseSchemaSection({ includeSchema, schemaListResult, schemaDDLResult })}
const instruction =
isClickhouse && intent === 'rewrite' ? CLICKHOUSE_LOGS_REWRITE_INSTRUCTION : prompt
## Code
const userMessage = source`
## ${schemaSection.heading}
\`\`\`${language ?? ''}
${textBeforeCursor}<selection>${selection}</selection>${textAfterCursor}
\`\`\`
${schemaSection.body}
## Instruction
## Code
${prompt}
`
\`\`\`${language ?? ''}
${textBeforeCursor}<selection>${selection}</selection>${textAfterCursor}
\`\`\`
## Instruction
${instruction}
`
// Note: these must be of type `CoreMessage` to prevent AI SDK from stripping `providerOptions`
// https://github.com/vercel/ai/blob/81ef2511311e8af34d75e37fc8204a82e775e8c3/packages/ai/core/prompt/standardize-prompt.ts#L83-L88
@@ -1,5 +1,5 @@
import { useMonaco } from '@monaco-editor/react'
import { useLocalStorage } from '@uidotdev/usehooks'
import { useDebounce, useLocalStorage } from '@uidotdev/usehooks'
import { IS_PLATFORM, LOCAL_STORAGE_KEYS, useFlag, useParams } from 'common'
import dayjs from 'dayjs'
import type { editor } from 'monaco-editor'
@@ -8,11 +8,7 @@ import { useEffect, useMemo, useRef, useState } from 'react'
import { toast } from 'sonner'
import { Button, ResizableHandle, ResizablePanel, ResizablePanelGroup } from 'ui'
import {
detectLogSource,
looksLikeLegacyLogsQuery,
rewriteLogsSqlWithAI,
} from '@/components/interfaces/Settings/Logs/logs-sql-rewrite'
import { LegacyLogsRewriteAdmonition } from '@/components/interfaces/Settings/Logs/LegacyLogsRewriteAdmonition'
import {
EXPLORER_DATEPICKER_HELPERS,
getDefaultHelper,
@@ -36,7 +32,6 @@ import {
buildLogQueryParams,
resolveLogDateRange,
} from '@/components/interfaces/Settings/Logs/logsDateRange'
import { LogsExplorerOtelBanner } from '@/components/interfaces/Settings/Logs/LogsExplorerOtelBanner'
import { LogsQueryPanel } from '@/components/interfaces/Settings/Logs/LogsQueryPanel'
import { LogTable } from '@/components/interfaces/Settings/Logs/LogTable'
import UpgradePrompt from '@/components/interfaces/Settings/Logs/UpgradePrompt'
@@ -52,16 +47,20 @@ import {
UpsertContentPayload,
useContentUpsertMutation,
} from '@/data/content/content-upsert-mutation'
import { constructHeaders } from '@/data/fetchers'
import { fetchOtelLogKeys } from '@/data/logs/otel-log-keys-query'
import {
LEGACY_LOGS_DIALECT_CHECK_DEBOUNCE_MS,
shouldOfferLegacyLogsRewrite,
} from '@/data/logs/logs-sql-rewrite'
import { untrustedLogSql } from '@/data/logs/safe-analytics-sql'
import {
useLegacyLogsRewrite,
type LegacyLogsRewriteProposal,
} from '@/hooks/analytics/useLegacyLogsRewrite'
import { useLogsQuery } from '@/hooks/analytics/useLogsQuery'
import { useLogsUrlState } from '@/hooks/analytics/useLogsUrlState'
import { useCustomContent } from '@/hooks/custom-content/useCustomContent'
import { useCheckEntitlements } from '@/hooks/misc/useCheckEntitlements'
import { useIsFeatureEnabled } from '@/hooks/misc/useIsFeatureEnabled'
import { useSelectedOrganizationQuery } from '@/hooks/misc/useSelectedOrganization'
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
import { useUpgradePrompt } from '@/hooks/misc/useUpgradePrompt'
import { uuidv4 } from '@/lib/helpers'
import { useProfile } from '@/lib/profile'
@@ -101,9 +100,6 @@ export const LogsExplorerPage: NextPageWithLayout = () => {
const useOtelEndpoint = useFlag('otelLegacyLogs')
const { logsShowMetadataIpTemplate } = useIsFeatureEnabled(['logs:show_metadata_ip_template'])
const { data: project } = useSelectedProjectQuery()
const { data: organization } = useSelectedOrganizationQuery()
const allTemplates = useMemo(() => {
const templates = getLogsTemplates(useOtelEndpoint)
if (logsShowMetadataIpTemplate) return templates
@@ -142,11 +138,7 @@ export const LogsExplorerPage: NextPageWithLayout = () => {
const [warnings, setWarnings] = useState<LogsWarning[]>([])
const [showMissingLimitError, setShowMissingLimitError] = useState<boolean>(false)
const [selectedLog, setSelectedLog] = useState<LogData | null>(null)
const [rewriteProposal, setRewriteProposal] = useState<{
original: string
modified: string
} | null>(null)
const [isRewriting, setIsRewriting] = useState<boolean>(false)
const [rewriteProposal, setRewriteProposal] = useState<LegacyLogsRewriteProposal | null>(null)
const [rewriteBannerDismissed, setRewriteBannerDismissed] = useLocalStorage<boolean>(
`project-${projectRef}-logs-rewrite-banner-dismissed`,
false
@@ -189,7 +181,32 @@ export const LogsExplorerPage: NextPageWithLayout = () => {
const results = logData
const isLoading = logsLoading
const showRewriteCTA = useOtelEndpoint && looksLikeLegacyLogsQuery(editorValue)
// Debounced so the dialect heuristics don't run on every keystroke, matching the
// SQL editor's rewrite banner.
const settledEditorValue = useDebounce(editorValue, LEGACY_LOGS_DIALECT_CHECK_DEBOUNCE_MS)
const shouldShowRewriteCTA = useMemo(
() =>
shouldOfferLegacyLogsRewrite({
sql: settledEditorValue,
isClickhouseLogsEnabled: useOtelEndpoint,
}),
[settledEditorValue, useOtelEndpoint]
)
const {
state: rewriteState,
requestRewrite,
dismiss: dismissRewriteBanner,
} = useLegacyLogsRewrite({
// Read straight from the editor instance — `editorValue` state can lag the
// most recent keystroke.
readSql: () => editorRef.current?.getValue() ?? editorValue,
onProposal: setRewriteProposal,
onDismissed: () => setRewriteBannerDismissed(true),
})
const isRewriting = rewriteState.status === 'rewriting'
const hasUnacknowledgedRewriteOutcome =
rewriteState.status === 'failed' || rewriteState.status === 'noRewriteNeeded'
const { mutateAsync: upsertContent, isPending: isUpsertingContent } = useContentUpsertMutation({
onError: (e) => {
@@ -243,42 +260,6 @@ export const LogsExplorerPage: NextPageWithLayout = () => {
addRecentLogSqlSnippet({ unchecked_sql: untrustedLogSql(template.searchString) })
}
const handleRewrite = async () => {
const currentSql = editorRef.current?.getValue() ?? editorValue
if (!currentSql.trim()) {
toast.info('Write a query to rewrite first')
return
}
setIsRewriting(true)
try {
const headerData = await constructHeaders()
const source = detectLogSource(currentSql)
const availableKeys = source
? await fetchOtelLogKeys({ projectRef: projectRef!, source }).catch(() => undefined)
: undefined
const rewritten = await rewriteLogsSqlWithAI({
sql: currentSql,
projectRef: projectRef!,
connectionString: project?.connectionString,
orgSlug: organization?.slug,
authorizationHeader: headerData.get('Authorization'),
availableKeys,
})
// The editor may have changed while awaiting key discovery and the AI call;
// don't offer a proposal that would clobber intervening edits.
const latestSql = editorRef.current?.getValue() ?? editorValue
if (latestSql !== currentSql) {
toast.info('The query changed while rewriting. Please try again.')
return
}
setRewriteProposal({ original: currentSql, modified: rewritten })
} catch (error) {
toast.error(`Couldn't rewrite the query: ${(error as Error).message}`)
} finally {
setIsRewriting(false)
}
}
const acceptRewrite = () => {
if (!rewriteProposal) return
editorRef.current?.setValue(rewriteProposal.modified)
@@ -504,15 +485,16 @@ export const LogsExplorerPage: NextPageWithLayout = () => {
templates={allTemplates.filter((template) => template.mode === 'custom')}
onSelectTemplate={onSelectTemplate}
warnings={warnings}
showRewriteAction={showRewriteCTA && rewriteBannerDismissed}
showRewriteAction={shouldShowRewriteCTA && rewriteBannerDismissed}
isRewriting={isRewriting}
onRewrite={handleRewrite}
onRewrite={requestRewrite}
/>
{showRewriteCTA && !rewriteBannerDismissed && (
<LogsExplorerOtelBanner
isRewriting={isRewriting}
onRewrite={handleRewrite}
onDismiss={() => setRewriteBannerDismissed(true)}
{(hasUnacknowledgedRewriteOutcome ||
(shouldShowRewriteCTA && !rewriteBannerDismissed)) && (
<LegacyLogsRewriteAdmonition
state={rewriteState}
onRewrite={requestRewrite}
onDismiss={dismissRewriteBanner}
/>
)}
<ShimmerLine active={isLoading} />