mirror of
https://github.com/supabase/supabase.git
synced 2026-10-05 09:25:06 +03:00
feat(studio): ClickHouse dialect for logs snippet AI + rewrite to ClickHouse (#48501)
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Feature, plus a refactor of the shared logs-rewrite flow. PR 8 of the SQL editor query-source series. Stacked on #48457 — review that one first, and merge this after it. ## What is the current behavior? A `log_sql` snippet runs against the ClickHouse-backed analytics endpoint, but the SQL editor's AI still writes Postgres: inline edits get Postgres system prompts, and the result is run through `sql-formatter`, which mangles ClickHouse backticks and `log_attributes` map lookups. Legacy Logs Explorer saved queries open in the editor as `log_sql` snippets. Those are BigQuery dialect and error against the ClickHouse endpoint the editor runs them on, with no in-editor way out — only the Logs Explorer offered a rewrite. The completion route was also asymmetric. It assembled a schema/code/instruction message for Postgres but forwarded `prompt` verbatim for ClickHouse, so a client wanting ClickHouse had to hand-build the equivalent string. ## What is the new behavior? **Inline AI speaks ClickHouse for logs snippets.** `sqlSourceToDialect` maps a snippet's source to `postgres`/`clickhouse` and `buildCompletionRequestBody` threads it through. For ClickHouse, `useSqlEditorAi` strips code fences from the response and skips `formatSql`. Execution and dialect both follow the snippet type, so a snippet's valid dialect never flips. **Rewrite to ClickHouse in the editor.** A banner offers the rewrite for a logs snippet whose text trips `looksLikeLegacyLogsQuery`, and proposes the result through the editor's existing AI diff view rather than replacing the snippet, so it's accepted or discarded like any other AI edit. Gated on `otelLegacyLogs`: on a non-migrated org the BigQuery text is still correct, so rewriting it would break a working query. The offer is a state machine (`offered` / `rewriting` / `failed` / `noRewriteNeeded` / `dismissed`) with a declarative table of valid transitions, so the states are mutually exclusive by construction and dismissal is terminal. A failure keeps its message and offers a retry; a response identical to the input is reported rather than opening an empty diff. **One place assembles completion prompts.** The route now uses a single template for both dialects, branching only the schema section and — for `intent: 'rewrite'` — the instruction. `lib/ai/clickhouse-logs.ts` is the single home for ClickHouse-logs prompt content, replacing two independently maintained descriptions of the same table. Clients carry no prompt text. **The rewrite flow is shared with the Logs Explorer.** Both surfaces previously hand-rolled the same sequence and had drifted: only one detected a no-op rewrite, they sourced `log_attributes` keys differently, and the Explorer formatted errors with an `as Error` cast. Both now use `useLegacyLogsRewrite` and the same state-driven banner, so the Explorer picks up no-op detection and typed error extraction. **Attribute keys are fetched on submit, not while typing.** The detected source would otherwise feed a reactive query key, making every edit that changed it cost another network call. `useLogsAttributeKeys` is imperative and goes through `queryClient.fetchQuery`, so a source already cached — including by the Explorer header and query panel, which subscribe reactively — is reused. This also closes a gap where inline edits never received keys at all, unlike full rewrites. `getErrorMessage` gains an optional typed fallback and no longer stringifies a bare object into `'[object Object]'`; every existing caller already hand-rolled a fallback, except `QueueSettings`, which interpolated the raw result and now passes one. Nothing here is user-visible until the `sqlEditorLogsSource` flag is enabled. Tests: dialect selection and request-body shape, the ClickHouse prompt content (including that the schema section does not restate the dialect rules), the reducer's valid and invalid transitions, `shouldOfferLegacyLogsRewrite`, on-submit key discovery with cache reuse, and `getErrorMessage`. ## Additional context <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an Assistant banner to help rewrite legacy BigQuery-style logs queries into ClickHouse SQL. * SQL assistance now adapts to the selected query type, including relevant log attribute context. * Rewrite suggestions can be reviewed as editor diffs before being applied. * **Bug Fixes** * Improved rewrite failure handling, retry options, dismissal behavior, and “no rewrite needed” messaging. * Error notifications now provide a clearer fallback message when details are unavailable. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
1 parent
b3c5c9fc04
commit
8b38e0d1ed
24 files changed
+1286
-314
No files matched your search
@@ -214,7 +214,7 @@ export const QueueSettings = ({}: QueueSettingsProps) => {
|
||||
toast.success('Successfully updated permissions')
|
||||
setOpen(false)
|
||||
} catch (error: unknown) {
|
||||
toast.error(`Failed to update permissions: ${getErrorMessage(error)}`)
|
||||
toast.error(`Failed to update permissions: ${getErrorMessage(error, 'unknown error')}`)
|
||||
} finally {
|
||||
setIsSaving(false)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
import { useDebounce } from '@uidotdev/usehooks'
|
||||
import { useFlag } from 'common'
|
||||
import { useMemo } from 'react'
|
||||
|
||||
import { DiffType } from './SQLEditor.types'
|
||||
import { useSqlEditorAssistant, useSqlEditorRun, useSqlEditorSnippet } from './SQLEditorControllers'
|
||||
import { LegacyLogsRewriteAdmonition } from '@/components/interfaces/Settings/Logs/LegacyLogsRewriteAdmonition'
|
||||
import {
|
||||
LEGACY_LOGS_DIALECT_CHECK_DEBOUNCE_MS,
|
||||
shouldOfferLegacyLogsRewrite,
|
||||
} from '@/data/logs/logs-sql-rewrite'
|
||||
import { useLegacyLogsRewrite } from '@/hooks/analytics/useLegacyLogsRewrite'
|
||||
import {
|
||||
getSqlEditorV2StateSnapshot,
|
||||
useSqlEditorV2StateSnapshot,
|
||||
} from '@/state/sql-editor/sql-editor-state'
|
||||
|
||||
/**
|
||||
* Offers to rewrite a logs snippet still written in the old BigQuery dialect
|
||||
* (per-service `FROM` tables, `unnest(metadata)` joins) to ClickHouse SQL. Legacy
|
||||
* Logs Explorer saved queries open in the SQL editor as `log_sql` snippets, and
|
||||
* those queries error against the ClickHouse-backed endpoint the editor runs them
|
||||
* on — this is the in-editor path out.
|
||||
*
|
||||
* The request itself is `useLegacyLogsRewrite` (shared with the Logs Explorer);
|
||||
* this decides when to offer it and routes the result into the editor's existing
|
||||
* AI diff view, so the user accepts or discards it the same way as any other AI
|
||||
* edit rather than having the snippet rewritten under them.
|
||||
*
|
||||
* Mount with `key={id}` so the offer resets when the user switches snippets. This
|
||||
* component must NOT be conditionally mounted by its parent — it hides itself, so
|
||||
* that opening a diff doesn't unmount it and throw away a dismissal.
|
||||
*/
|
||||
export const LegacyLogsRewriteBanner = () => {
|
||||
const { id } = useSqlEditorSnippet()
|
||||
const { runSource } = useSqlEditorRun()
|
||||
const {
|
||||
diff: { isDiffOpen, setSourceSqlDiff, setSelectedDiffType },
|
||||
} = useSqlEditorAssistant()
|
||||
|
||||
const isOtelLogsEnabled = useFlag('otelLegacyLogs')
|
||||
const snapV2 = useSqlEditorV2StateSnapshot()
|
||||
|
||||
// The store is written on every keystroke, so debounce before running the
|
||||
// dialect heuristics — the banner's visibility doesn't need per-character
|
||||
// precision, and a settled value avoids flapping mid-edit.
|
||||
const liveSql = snapV2.snippets[id]?.snippet.content?.unchecked_sql ?? ''
|
||||
const settledSql = useDebounce(liveSql, LEGACY_LOGS_DIALECT_CHECK_DEBOUNCE_MS)
|
||||
|
||||
const isLogsSnippetNeedingRewrite = useMemo(
|
||||
() =>
|
||||
runSource.type === 'logs' &&
|
||||
shouldOfferLegacyLogsRewrite({ sql: settledSql, isClickhouseLogsEnabled: isOtelLogsEnabled }),
|
||||
[runSource.type, settledSql, isOtelLogsEnabled]
|
||||
)
|
||||
|
||||
const { state, requestRewrite, dismiss } = useLegacyLogsRewrite({
|
||||
// Rewrite exactly what's in the editor now, not the debounced value the
|
||||
// visibility check used — they differ if the user clicked mid-edit.
|
||||
readSql: () => getSqlEditorV2StateSnapshot().snippets[id]?.snippet.content?.unchecked_sql ?? '',
|
||||
onProposal: ({ original, modified }) => {
|
||||
setSourceSqlDiff({ original, modified })
|
||||
setSelectedDiffType(DiffType.Modification)
|
||||
},
|
||||
})
|
||||
|
||||
// An outcome the user hasn't acknowledged stays up even once the query no longer
|
||||
// looks legacy — otherwise a successful proposal would yank its own result away.
|
||||
const hasUnacknowledgedOutcome = state.status === 'failed' || state.status === 'noRewriteNeeded'
|
||||
const canShowBanner = !isDiffOpen && (isLogsSnippetNeedingRewrite || hasUnacknowledgedOutcome)
|
||||
|
||||
if (!canShowBanner) return null
|
||||
|
||||
return (
|
||||
<LegacyLogsRewriteAdmonition state={state} onRewrite={requestRewrite} onDismiss={dismiss} />
|
||||
)
|
||||
}
|
||||
@@ -29,6 +29,7 @@ import {
|
||||
resolveConnectionString,
|
||||
resolveDiffKeyAction,
|
||||
shouldAutoGenerateTitle,
|
||||
sqlSourceToDialect,
|
||||
trimTrailingSemicolons,
|
||||
} from './SQLEditor.utils'
|
||||
import type { DatabaseEventTrigger } from '@/data/database-event-triggers/database-event-triggers-query'
|
||||
@@ -432,6 +433,31 @@ describe('SQLEditor.utils.ts:buildCompletionRequestBody', () => {
|
||||
completionMetadata: { prompt: 'add a where clause' },
|
||||
})
|
||||
})
|
||||
test('omits dialect when not provided, so the route keeps its Postgres default', () => {
|
||||
const body = buildCompletionRequestBody({
|
||||
projectRef: 'default',
|
||||
connectionString: null,
|
||||
orgSlug: 'acme',
|
||||
})
|
||||
expect(body).not.toHaveProperty('dialect')
|
||||
})
|
||||
test('includes the dialect when provided', () => {
|
||||
expect(
|
||||
buildCompletionRequestBody({
|
||||
projectRef: 'default',
|
||||
connectionString: null,
|
||||
orgSlug: 'acme',
|
||||
dialect: 'clickhouse',
|
||||
}).dialect
|
||||
).toBe('clickhouse')
|
||||
})
|
||||
})
|
||||
|
||||
describe('SQLEditor.utils.ts:sqlSourceToDialect', () => {
|
||||
test('logs snippets get ClickHouse, database snippets get Postgres', () => {
|
||||
expect(sqlSourceToDialect('logs')).toBe('clickhouse')
|
||||
expect(sqlSourceToDialect('database')).toBe('postgres')
|
||||
})
|
||||
})
|
||||
|
||||
describe('SQLEditor.utils.ts:createSqlSnippetSkeletonV2', () => {
|
||||
|
||||
@@ -497,25 +497,46 @@ export function assembleCompletionDiff(
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds the request body sent to the AI completion endpoint. `options` is
|
||||
* the caller-provided extra fields (e.g. `completionMetadata`), merged in
|
||||
* last so it can override the defaults if it ever needs to.
|
||||
* The SQL dialect the AI writes. Mirrors the `dialect` enum the completion API
|
||||
* route accepts — a snippet's dialect follows its source and never flips, so a
|
||||
* logs snippet always gets ClickHouse SQL and a database snippet Postgres.
|
||||
*/
|
||||
export type SqlDialect = 'postgres' | 'clickhouse'
|
||||
|
||||
/**
|
||||
* Maps a snippet's query source to the dialect the AI should write in. Logs
|
||||
* snippets run against the ClickHouse-backed analytics endpoint; everything
|
||||
* else runs against the user's Postgres database.
|
||||
*/
|
||||
export function sqlSourceToDialect(source: SqlSnippetSource): SqlDialect {
|
||||
return source === 'logs' ? 'clickhouse' : 'postgres'
|
||||
}
|
||||
|
||||
/**
|
||||
* Builds the request body sent to the AI completion endpoint. `dialect` is
|
||||
* omitted when undefined so callers that don't care keep the route's Postgres
|
||||
* default. `options` is the caller-provided extra fields (e.g.
|
||||
* `completionMetadata`), merged in last so it can override the defaults if it
|
||||
* ever needs to.
|
||||
*/
|
||||
export function buildCompletionRequestBody({
|
||||
projectRef,
|
||||
connectionString,
|
||||
orgSlug,
|
||||
dialect,
|
||||
options,
|
||||
}: {
|
||||
projectRef: string | undefined
|
||||
connectionString: string | undefined | null
|
||||
orgSlug: string | undefined
|
||||
dialect?: SqlDialect
|
||||
options?: { completionMetadata?: unknown }
|
||||
}): {
|
||||
projectRef: string | undefined
|
||||
connectionString: string | undefined | null
|
||||
language: 'sql'
|
||||
orgSlug: string | undefined
|
||||
dialect?: SqlDialect
|
||||
completionMetadata?: unknown
|
||||
} {
|
||||
return {
|
||||
@@ -523,6 +544,7 @@ export function buildCompletionRequestBody({
|
||||
connectionString,
|
||||
language: 'sql',
|
||||
orgSlug,
|
||||
...(dialect !== undefined && { dialect }),
|
||||
...(options ?? {}),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -191,7 +191,7 @@ export const SQLEditorControllersProvider = ({ children }: PropsWithChildren) =>
|
||||
|
||||
const isExecuting = isExecutingDb || isExecutingLogs
|
||||
|
||||
const ai = useSqlEditorAi({ id, editorMountCount, diff, prompt })
|
||||
const ai = useSqlEditorAi({ id, editorMountCount, diff, prompt, sqlSource: runSource.type })
|
||||
const { acceptAiHandler, discardAiHandler } = ai
|
||||
|
||||
useSqlEditorShortcuts({
|
||||
|
||||
@@ -4,6 +4,7 @@ import dynamic from 'next/dynamic'
|
||||
import { useCallback } from 'react'
|
||||
import { cn } from 'ui'
|
||||
|
||||
import { LegacyLogsRewriteBanner } from './LegacyLogsRewriteBanner'
|
||||
import { useSQLEditorContext } from './SQLEditorContext'
|
||||
import {
|
||||
useSqlEditorAssistant,
|
||||
@@ -171,23 +172,32 @@ const SQLEditorMainView = () => {
|
||||
)
|
||||
}
|
||||
|
||||
/** The top (editor) resizable panel: loading state, diff view, and main editor. */
|
||||
export const SQLEditorEditorPanel = () => {
|
||||
const { isLoading } = useSqlEditorSnippet()
|
||||
const { diff } = useSqlEditorAssistant()
|
||||
|
||||
function LoadingSpinner() {
|
||||
return (
|
||||
<div className="grow overflow-y-auto border-b h-full">
|
||||
{isLoading ? (
|
||||
<div className="flex h-full w-full items-center justify-center">
|
||||
<Loader2 className="animate-spin text-brand" />
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
{diff.isDiffOpen && <SQLEditorDiffView />}
|
||||
<SQLEditorMainView />
|
||||
</>
|
||||
)}
|
||||
<div className="overflow-y-auto h-full">
|
||||
<div className="flex h-full w-full items-center justify-center">
|
||||
<Loader2 className="animate-spin text-brand" />
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
/** The top (editor) resizable panel: loading state, diff view, and main editor. */
|
||||
export const SQLEditorEditorPanel = () => {
|
||||
const { id, isLoading } = useSqlEditorSnippet()
|
||||
const { diff } = useSqlEditorAssistant()
|
||||
|
||||
if (isLoading) {
|
||||
return <LoadingSpinner />
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="h-full flex flex-col">
|
||||
<LegacyLogsRewriteBanner key={id} />
|
||||
<div className="grow overflow-y-auto min-h-0">
|
||||
{diff.isDiffOpen && <SQLEditorDiffView />}
|
||||
<SQLEditorMainView />
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -1,13 +1,17 @@
|
||||
import { act, waitFor } from '@testing-library/react'
|
||||
import { http, HttpResponse } from 'msw'
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
|
||||
import { useSqlEditorDiff, useSqlEditorPrompt } from './hooks'
|
||||
import type { SqlSnippetSource } from './querySource'
|
||||
import { DiffType } from './SQLEditor.types'
|
||||
import { useSqlEditorAi } from './useSqlEditorAi'
|
||||
import { SIDEBAR_KEYS } from '@/components/layouts/ProjectLayout/LayoutSidebar/LayoutSidebarProvider'
|
||||
import { API_URL } from '@/lib/constants'
|
||||
import { sidebarManagerState } from '@/state/sidebar-manager-state'
|
||||
import { sqlEditorDiffRequestState } from '@/state/sql-editor/sql-editor-diff-request'
|
||||
import { sqlEditorSessionState } from '@/state/sql-editor/sql-editor-session-state'
|
||||
import { mswServer } from '@/tests/lib/msw'
|
||||
import {
|
||||
createInMemoryEditor,
|
||||
renderSqlEditorHook,
|
||||
@@ -18,15 +22,17 @@ import {
|
||||
|
||||
const SNIPPET_ID = 'ai-snippet'
|
||||
|
||||
type HarnessProps = { editorMountCount?: number; sqlSource?: SqlSnippetSource }
|
||||
|
||||
/**
|
||||
* Composes the diff + prompt state hooks the AI hook depends on (production
|
||||
* wires these together in `SQLEditorControllers`), so tests drive the real
|
||||
* accept/discard/drain flows end to end.
|
||||
*/
|
||||
function useAiHarness({ editorMountCount = 1 }: { editorMountCount?: number } = {}) {
|
||||
function useAiHarness({ editorMountCount = 1, sqlSource = 'database' }: HarnessProps = {}) {
|
||||
const diff = useSqlEditorDiff()
|
||||
const prompt = useSqlEditorPrompt()
|
||||
const ai = useSqlEditorAi({ id: SNIPPET_ID, editorMountCount, diff, prompt })
|
||||
const ai = useSqlEditorAi({ id: SNIPPET_ID, editorMountCount, diff, prompt, sqlSource })
|
||||
return { ai, diff, prompt }
|
||||
}
|
||||
|
||||
@@ -119,6 +125,110 @@ describe('useSqlEditorAi — accept / discard diff', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('useSqlEditorAi — completion dialect', () => {
|
||||
type CompletionRequestBody = {
|
||||
dialect?: string
|
||||
intent?: string
|
||||
completionMetadata: {
|
||||
prompt: string
|
||||
selection: string
|
||||
textBeforeCursor: string
|
||||
textAfterCursor: string
|
||||
availableKeys?: string[]
|
||||
}
|
||||
}
|
||||
|
||||
/** Replaces the default completion mock so we can read what was posted. */
|
||||
function captureCompletionRequests(response: string) {
|
||||
const bodies: CompletionRequestBody[] = []
|
||||
mswServer.use(
|
||||
http.post(`${API_URL}/ai/code/complete`, async ({ request }) => {
|
||||
bodies.push((await request.json()) as CompletionRequestBody)
|
||||
return HttpResponse.json(response)
|
||||
})
|
||||
)
|
||||
return bodies
|
||||
}
|
||||
|
||||
const context = {
|
||||
beforeSelection: "select timestamp from logs where source = 'edge_logs'\n",
|
||||
selection: 'limit 5',
|
||||
afterSelection: '',
|
||||
}
|
||||
|
||||
it('posts the clickhouse dialect with the raw instruction and cursor context', async () => {
|
||||
const bodies = captureCompletionRequests('limit 10')
|
||||
const { result } = renderSqlEditorHook(useAiHarness, {
|
||||
initialProps: { sqlSource: 'logs' },
|
||||
})
|
||||
|
||||
await act(async () => {
|
||||
await result.current.ai.handlePrompt('only keep 5xx responses', context)
|
||||
})
|
||||
|
||||
expect(bodies).toHaveLength(1)
|
||||
expect(bodies[0].dialect).toBe('clickhouse')
|
||||
// The route assembles the schema section and the selection-wrapped code around
|
||||
// the instruction, so the client posts the instruction verbatim — the same
|
||||
// shape as the Postgres path — and never hand-builds prompt text.
|
||||
expect(bodies[0].completionMetadata.prompt).toBe('only keep 5xx responses')
|
||||
expect(bodies[0].completionMetadata.selection).toBe('limit 5')
|
||||
expect(bodies[0].completionMetadata.textBeforeCursor).toBe(context.beforeSelection)
|
||||
// An inline edit is not a rewrite.
|
||||
expect(bodies[0].intent).toBeUndefined()
|
||||
})
|
||||
|
||||
it('posts the postgres dialect with the raw instruction for a database snippet', async () => {
|
||||
const bodies = captureCompletionRequests('limit 10')
|
||||
const { result } = renderSqlEditorHook(useAiHarness, {
|
||||
initialProps: { sqlSource: 'database' },
|
||||
})
|
||||
|
||||
await act(async () => {
|
||||
await result.current.ai.handlePrompt('bump the limit', context)
|
||||
})
|
||||
|
||||
expect(bodies).toHaveLength(1)
|
||||
expect(bodies[0].dialect).toBe('postgres')
|
||||
expect(bodies[0].completionMetadata.prompt).toBe('bump the limit')
|
||||
})
|
||||
|
||||
it('strips code fences and leaves clickhouse output unformatted', async () => {
|
||||
captureCompletionRequests('```sql\nlimit 10\n```')
|
||||
const { result } = renderSqlEditorHook(useAiHarness, {
|
||||
initialProps: { sqlSource: 'logs' },
|
||||
})
|
||||
|
||||
await act(async () => {
|
||||
await result.current.ai.handlePrompt('bump the limit', context)
|
||||
})
|
||||
|
||||
await waitFor(() => expect(result.current.diff.isDiffOpen).toBe(true))
|
||||
// sql-formatter is Postgres-only, so the ClickHouse diff is the reassembled
|
||||
// query verbatim — fences stripped, nothing else touched.
|
||||
expect(result.current.diff.sourceSqlDiff).toEqual({
|
||||
original: `${context.beforeSelection}limit 5`,
|
||||
modified: `${context.beforeSelection}limit 10`,
|
||||
})
|
||||
})
|
||||
|
||||
it('still formats database output through sql-formatter', async () => {
|
||||
captureCompletionRequests('limit 10')
|
||||
const { result } = renderSqlEditorHook(useAiHarness, {
|
||||
initialProps: { sqlSource: 'database' },
|
||||
})
|
||||
|
||||
await act(async () => {
|
||||
await result.current.ai.handlePrompt('bump the limit', context)
|
||||
})
|
||||
|
||||
await waitFor(() => expect(result.current.diff.isDiffOpen).toBe(true))
|
||||
expect(result.current.diff.sourceSqlDiff?.modified).not.toBe(
|
||||
`${context.beforeSelection}limit 10`
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
describe('useSqlEditorAi — debug', () => {
|
||||
it('onDebug opens the assistant sidebar and starts a debug chat from the failing snippet', async () => {
|
||||
seedSnippet({ id: SNIPPET_ID, name: 'Broken query', sql: 'selct 1;' })
|
||||
|
||||
@@ -4,6 +4,7 @@ import { useCallback, useEffect, useEffectEvent, useMemo, useState } from 'react
|
||||
import { toast } from 'sonner'
|
||||
|
||||
import type { useSqlEditorDiff, useSqlEditorPrompt } from './hooks'
|
||||
import type { SqlSnippetSource } from './querySource'
|
||||
import { DiffType, type IStandaloneDiffEditor } from './SQLEditor.types'
|
||||
import {
|
||||
assembleCompletionDiff,
|
||||
@@ -13,12 +14,15 @@ import {
|
||||
createSqlSnippetSkeletonV2,
|
||||
extractDebugContext,
|
||||
planDiffRequestApplication,
|
||||
sqlSourceToDialect,
|
||||
} from './SQLEditor.utils'
|
||||
import { useSQLEditorContext } from './SQLEditorContext'
|
||||
import { useSnippetTitleGenerator } from './useSnippetTitleGenerator'
|
||||
import { SIDEBAR_KEYS } from '@/components/layouts/ProjectLayout/LayoutSidebar/LayoutSidebarProvider'
|
||||
import { constructHeaders } from '@/data/fetchers'
|
||||
import { stripSqlCodeFences } from '@/data/logs/logs-sql-rewrite'
|
||||
import { isError } from '@/data/utils/error-check'
|
||||
import { useLogsAttributeKeys } from '@/hooks/analytics/useLogsAttributeKeys'
|
||||
import { useSelectedOrganizationQuery } from '@/hooks/misc/useSelectedOrganization'
|
||||
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
|
||||
import { BASE_PATH } from '@/lib/constants'
|
||||
@@ -37,6 +41,11 @@ type UseSqlEditorAiArgs = {
|
||||
editorMountCount: number
|
||||
diff: ReturnType<typeof useSqlEditorDiff>
|
||||
prompt: ReturnType<typeof useSqlEditorPrompt>
|
||||
/**
|
||||
* Where the snippet runs. Selects the dialect the AI writes in — logs snippets
|
||||
* get ClickHouse SQL for the `logs` table, database snippets get Postgres.
|
||||
*/
|
||||
sqlSource: SqlSnippetSource
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -45,7 +54,13 @@ type UseSqlEditorAiArgs = {
|
||||
* lifecycle effects (one-shot diff-request drain, diff-editor value sync, and the
|
||||
* ask-AI widget visibility).
|
||||
*/
|
||||
export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEditorAiArgs) {
|
||||
export function useSqlEditorAi({
|
||||
id,
|
||||
editorMountCount,
|
||||
diff,
|
||||
prompt,
|
||||
sqlSource,
|
||||
}: UseSqlEditorAiArgs) {
|
||||
const {
|
||||
sourceSqlDiff,
|
||||
setSourceSqlDiff,
|
||||
@@ -77,6 +92,14 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
|
||||
const [isDiffEditorMounted, setIsDiffEditorMounted] = useState(false)
|
||||
const [showWidget, setShowWidget] = useState(false)
|
||||
|
||||
const dialect = sqlSourceToDialect(sqlSource)
|
||||
const isClickhouse = dialect === 'clickhouse'
|
||||
|
||||
// Grounds ClickHouse edits in the source's real log_attributes keys, the same way
|
||||
// the whole-query rewrite does — otherwise inline edits invent dotted paths.
|
||||
// Looked up when the user submits, not while they type.
|
||||
const { fetchAttributeKeys } = useLogsAttributeKeys()
|
||||
|
||||
const handleNewQuery = useCallback(
|
||||
async (sql: string, name: string) => {
|
||||
if (!ref) return console.error('Project ref is required')
|
||||
@@ -196,6 +219,7 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
|
||||
projectRef: project?.ref,
|
||||
connectionString: project?.connectionString,
|
||||
orgSlug: org?.slug,
|
||||
dialect,
|
||||
options: options?.body,
|
||||
})
|
||||
),
|
||||
@@ -210,9 +234,16 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
|
||||
const text: string = await response.json()
|
||||
|
||||
const meta = options?.body?.completionMetadata ?? {}
|
||||
const { original, modified } = assembleCompletionDiff(meta, text)
|
||||
// The clickhouse system prompt forbids fences, but strip them defensively
|
||||
// so a chatty model can't leak backticks into the snippet.
|
||||
const { original, modified } = assembleCompletionDiff(
|
||||
meta,
|
||||
isClickhouse ? stripSqlCodeFences(text) : text
|
||||
)
|
||||
|
||||
const formattedModified = formatSql(modified)
|
||||
// sql-formatter is Postgres-only — it mangles ClickHouse backticks and
|
||||
// map lookups — so ClickHouse output goes into the diff unformatted.
|
||||
const formattedModified = isClickhouse ? modified : formatSql(modified)
|
||||
setSourceSqlDiff({ original, modified: formattedModified })
|
||||
setSelectedDiffType(DiffType.Modification)
|
||||
setPromptState((prev) => ({ ...prev, isLoading: false }))
|
||||
@@ -224,6 +255,8 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
|
||||
}
|
||||
},
|
||||
[
|
||||
dialect,
|
||||
isClickhouse,
|
||||
org?.slug,
|
||||
project?.connectionString,
|
||||
project?.ref,
|
||||
@@ -249,10 +282,23 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
|
||||
beforeSelection: context.beforeSelection,
|
||||
afterSelection: context.afterSelection,
|
||||
}))
|
||||
const headerData = await constructHeaders()
|
||||
// ClickHouse only: there's no server-side schema to fetch for the logs
|
||||
// table, so the real log_attributes keys travel with the request. Detected
|
||||
// from the whole document, which is what the three context fields spell.
|
||||
const [headerData, availableKeys] = await Promise.all([
|
||||
constructHeaders(),
|
||||
isClickhouse
|
||||
? fetchAttributeKeys(
|
||||
context.beforeSelection + context.selection + context.afterSelection
|
||||
)
|
||||
: undefined,
|
||||
])
|
||||
|
||||
const authorizationHeader = headerData.get('Authorization')
|
||||
|
||||
// The instruction goes over as-is for both dialects — the route assembles
|
||||
// the schema section and the cursor context around it, so there's exactly
|
||||
// one place that knows how a completion prompt is built.
|
||||
await complete(prompt, {
|
||||
...(authorizationHeader
|
||||
? { headers: { Authorization: authorizationHeader } }
|
||||
@@ -264,6 +310,7 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
|
||||
language: 'pgsql',
|
||||
prompt,
|
||||
selection: context.selection,
|
||||
...(availableKeys ? { availableKeys } : {}),
|
||||
},
|
||||
},
|
||||
})
|
||||
@@ -271,7 +318,7 @@ export function useSqlEditorAi({ id, editorMountCount, diff, prompt }: UseSqlEdi
|
||||
setPromptState((prev) => ({ ...prev, isLoading: false }))
|
||||
}
|
||||
},
|
||||
[complete, setPromptState]
|
||||
[complete, fetchAttributeKeys, isClickhouse, setPromptState]
|
||||
)
|
||||
|
||||
const handleDiffEditorMount = useCallback(
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
import { Button } from 'ui'
|
||||
import { Admonition } from 'ui-patterns/Admonition'
|
||||
|
||||
import { ButtonTooltip } from '@/components/ui/ButtonTooltip'
|
||||
import type { LegacyLogsRewriteState } from '@/hooks/analytics/useLegacyLogsRewrite'
|
||||
|
||||
interface LegacyLogsRewriteAdmonitionProps {
|
||||
state: LegacyLogsRewriteState
|
||||
onRewrite: () => void
|
||||
onDismiss: () => void
|
||||
}
|
||||
|
||||
const BANNER_CLASSES = 'mb-0 rounded-none border-x-0 border-t-0'
|
||||
|
||||
/**
|
||||
* Presentation for the BigQuery → ClickHouse rewrite offer, covering every state
|
||||
* of `useLegacyLogsRewrite`. Shared by the Logs Explorer and the SQL editor so the
|
||||
* copy and the outcome handling live in one place — a surface only decides *when*
|
||||
* to show this, not what each state says.
|
||||
*
|
||||
* Renders nothing once dismissed; callers can also hide it earlier if they have
|
||||
* their own reasons to (the SQL editor hides it while a diff is open).
|
||||
*/
|
||||
export const LegacyLogsRewriteAdmonition = ({
|
||||
state,
|
||||
onRewrite,
|
||||
onDismiss,
|
||||
}: LegacyLogsRewriteAdmonitionProps) => {
|
||||
if (state.status === 'dismissed') return null
|
||||
|
||||
if (state.status === 'failed') {
|
||||
return (
|
||||
<Admonition
|
||||
type="warning"
|
||||
layout="horizontal"
|
||||
className={BANNER_CLASSES}
|
||||
title="Unable to rewrite the query"
|
||||
description={state.message}
|
||||
actions={
|
||||
<div className="flex items-center gap-2">
|
||||
<Button variant="default" size="tiny" onClick={onRewrite}>
|
||||
Try again
|
||||
</Button>
|
||||
<Button variant="text" size="tiny" onClick={onDismiss}>
|
||||
Dismiss
|
||||
</Button>
|
||||
</div>
|
||||
}
|
||||
/>
|
||||
)
|
||||
}
|
||||
|
||||
// The dialect check is a heuristic, so an unchanged rewrite is the Assistant
|
||||
// disagreeing with it. Say so and let the user close it, rather than proposing
|
||||
// an empty diff or silently giving up.
|
||||
if (state.status === 'noRewriteNeeded') {
|
||||
return (
|
||||
<Admonition
|
||||
type="default"
|
||||
layout="horizontal"
|
||||
className={BANNER_CLASSES}
|
||||
title="No rewrite needed"
|
||||
description="The Assistant found nothing to change — this query already runs on ClickHouse."
|
||||
actions={
|
||||
<Button variant="default" size="tiny" onClick={onDismiss}>
|
||||
Dismiss
|
||||
</Button>
|
||||
}
|
||||
/>
|
||||
)
|
||||
}
|
||||
|
||||
const isRewriting = state.status === 'rewriting'
|
||||
|
||||
return (
|
||||
<Admonition
|
||||
type="default"
|
||||
layout="horizontal"
|
||||
className={BANNER_CLASSES}
|
||||
title="Logs now run on a ClickHouse-backed engine"
|
||||
description="This query needs to be adjusted to ClickHouse SQL, which the Assistant can do for you."
|
||||
actions={
|
||||
<div className="flex items-center gap-2">
|
||||
<Button variant="default" size="tiny" loading={isRewriting} onClick={onRewrite}>
|
||||
Rewrite with Assistant
|
||||
</Button>
|
||||
{/* Disabled mid-rewrite so the offer can't be dismissed out from under
|
||||
an in-flight request — the two states stay mutually exclusive. */}
|
||||
<ButtonTooltip
|
||||
variant="text"
|
||||
size="tiny"
|
||||
disabled={isRewriting}
|
||||
onClick={onDismiss}
|
||||
tooltip={{
|
||||
content: {
|
||||
side: 'bottom',
|
||||
text: isRewriting ? 'Available once the rewrite finishes' : undefined,
|
||||
},
|
||||
}}
|
||||
>
|
||||
Dismiss
|
||||
</ButtonTooltip>
|
||||
</div>
|
||||
}
|
||||
/>
|
||||
)
|
||||
}
|
||||
@@ -1,34 +0,0 @@
|
||||
import { Button } from 'ui'
|
||||
import { Admonition } from 'ui-patterns/Admonition'
|
||||
|
||||
interface LogsExplorerOtelBannerProps {
|
||||
isRewriting: boolean
|
||||
onRewrite: () => void
|
||||
onDismiss: () => void
|
||||
}
|
||||
|
||||
export const LogsExplorerOtelBanner = ({
|
||||
isRewriting,
|
||||
onRewrite,
|
||||
onDismiss,
|
||||
}: LogsExplorerOtelBannerProps) => {
|
||||
return (
|
||||
<Admonition
|
||||
type="default"
|
||||
layout="horizontal"
|
||||
className="mb-0 rounded-none border-x-0 border-t-0"
|
||||
title="Logs now run on a ClickHouse-backed engine"
|
||||
description="This query needs to be adjusted to ClickHouse SQL, which the Assistant can do for you."
|
||||
actions={
|
||||
<div className="flex items-center gap-2">
|
||||
<Button variant="default" size="tiny" loading={isRewriting} onClick={onRewrite}>
|
||||
Rewrite with Assistant
|
||||
</Button>
|
||||
<Button variant="text" size="tiny" onClick={onDismiss}>
|
||||
Dismiss
|
||||
</Button>
|
||||
</div>
|
||||
}
|
||||
/>
|
||||
)
|
||||
}
|
||||
@@ -1,6 +1,8 @@
|
||||
import { QuerySearchParamsType } from '@/components/interfaces/UnifiedLogs/UnifiedLogs.types'
|
||||
|
||||
export const logsKeys = {
|
||||
otelLogKeys: (projectRef: string | undefined, source: string | undefined) =>
|
||||
['projects', projectRef, 'otel-log-keys', source] as const,
|
||||
unifiedLogsInfinite: (
|
||||
projectRef: string | undefined,
|
||||
searchParams: QuerySearchParamsType | undefined
|
||||
|
||||
+44
-31
@@ -1,45 +1,36 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
import {
|
||||
buildClickhouseRewritePrompt,
|
||||
detectLogSource,
|
||||
looksLikeLegacyLogsQuery,
|
||||
rewriteLogsSqlWithAI,
|
||||
shouldOfferLegacyLogsRewrite,
|
||||
stripSqlCodeFences,
|
||||
} from './logs-sql-rewrite'
|
||||
|
||||
describe('buildClickhouseRewritePrompt', () => {
|
||||
it('includes the query, the schema, and a reply-with-only-SQL instruction', () => {
|
||||
const prompt = buildClickhouseRewritePrompt('select count(*) from edge_logs')
|
||||
expect(prompt).toContain('select count(*) from edge_logs')
|
||||
expect(prompt).toContain('log_attributes')
|
||||
expect(prompt).toContain("source = 'edge_logs'")
|
||||
expect(prompt.toLowerCase()).toContain('reply with only')
|
||||
describe('shouldOfferLegacyLogsRewrite', () => {
|
||||
const legacySql = 'select 1 from edge_logs cross join unnest(metadata) as m'
|
||||
|
||||
it('offers the rewrite for BigQuery-dialect SQL once logs run on ClickHouse', () => {
|
||||
expect(shouldOfferLegacyLogsRewrite({ sql: legacySql, isClickhouseLogsEnabled: true })).toBe(
|
||||
true
|
||||
)
|
||||
})
|
||||
|
||||
it('spells out the FROM-to-logs conversion and shows a worked example', () => {
|
||||
const prompt = buildClickhouseRewritePrompt('select 1 from postgres_logs')
|
||||
expect(prompt).toContain("from logs where source = 'postgres_logs'")
|
||||
expect(prompt.toLowerCase()).toContain('remove every')
|
||||
expect(prompt).toContain('cross join unnest')
|
||||
expect(prompt).toContain('BigQuery:')
|
||||
expect(prompt).toContain('ClickHouse:')
|
||||
expect(prompt).toContain("log_attributes['parsed.error_severity']")
|
||||
it('never offers it on a non-migrated org, where the BigQuery SQL is still correct', () => {
|
||||
expect(shouldOfferLegacyLogsRewrite({ sql: legacySql, isClickhouseLogsEnabled: false })).toBe(
|
||||
false
|
||||
)
|
||||
})
|
||||
|
||||
it('lists the real log_attributes keys when provided and demands exact paths', () => {
|
||||
const prompt = buildClickhouseRewritePrompt('select 1 from edge_logs', [
|
||||
'request.headers.x_real_ip',
|
||||
'request.cf.country',
|
||||
])
|
||||
expect(prompt).toContain("log_attributes['request.headers.x_real_ip']")
|
||||
expect(prompt).toContain("log_attributes['request.cf.country']")
|
||||
expect(prompt.toLowerCase()).toContain('exact')
|
||||
})
|
||||
|
||||
it('omits the keys section when none are provided', () => {
|
||||
const prompt = buildClickhouseRewritePrompt('select 1 from edge_logs')
|
||||
expect(prompt).not.toContain('actual log_attributes keys present')
|
||||
it('does not offer it for SQL that is already ClickHouse, or for empty SQL', () => {
|
||||
expect(
|
||||
shouldOfferLegacyLogsRewrite({
|
||||
sql: "select timestamp from logs where source = 'edge_logs' limit 5",
|
||||
isClickhouseLogsEnabled: true,
|
||||
})
|
||||
).toBe(false)
|
||||
expect(shouldOfferLegacyLogsRewrite({ sql: '', isClickhouseLogsEnabled: true })).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -66,6 +57,21 @@ describe('detectLogSource', () => {
|
||||
it('returns undefined when nothing matches', () => {
|
||||
expect(detectLogSource('select 1')).toBeUndefined()
|
||||
})
|
||||
|
||||
it('ignores a column that merely ends in "source"', () => {
|
||||
expect(detectLogSource("select 1 from logs where resource = 'nope'")).toBeUndefined()
|
||||
expect(detectLogSource("select 1 from logs where datasource = 'nope'")).toBeUndefined()
|
||||
})
|
||||
|
||||
it('still reads a qualified source column', () => {
|
||||
expect(detectLogSource("select 1 from logs t where t.source = 'auth_logs'")).toBe('auth_logs')
|
||||
})
|
||||
|
||||
it('prefers the real source column over a lookalike earlier in the query', () => {
|
||||
expect(detectLogSource("select resource = 'nope' from logs where source = 'edge_logs'")).toBe(
|
||||
'edge_logs'
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
describe('looksLikeLegacyLogsQuery', () => {
|
||||
@@ -112,7 +118,7 @@ describe('rewriteLogsSqlWithAI', () => {
|
||||
vi.unstubAllGlobals()
|
||||
})
|
||||
|
||||
it('posts to the completion endpoint and returns the cleaned query', async () => {
|
||||
it('declares the rewrite intent and sends the query as the selection', async () => {
|
||||
const fetchMock = vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => '```sql\nselect 1 from logs\n```',
|
||||
@@ -122,6 +128,7 @@ describe('rewriteLogsSqlWithAI', () => {
|
||||
const result = await rewriteLogsSqlWithAI({
|
||||
sql: 'select 1 from edge_logs',
|
||||
projectRef: 'abc',
|
||||
availableKeys: ['request.method'],
|
||||
})
|
||||
|
||||
expect(result).toBe('select 1 from logs')
|
||||
@@ -129,8 +136,14 @@ describe('rewriteLogsSqlWithAI', () => {
|
||||
expect(url).toContain('/api/ai/code/complete')
|
||||
const body = JSON.parse(init.body)
|
||||
expect(body.dialect).toBe('clickhouse')
|
||||
expect(body.intent).toBe('rewrite')
|
||||
// The whole query is the selection, so the rewrite replaces all of it.
|
||||
expect(body.completionMetadata.selection).toBe('select 1 from edge_logs')
|
||||
expect(body.completionMetadata.prompt.toLowerCase()).toContain('reply with only')
|
||||
expect(body.completionMetadata.textBeforeCursor).toBe('')
|
||||
expect(body.completionMetadata.textAfterCursor).toBe('')
|
||||
expect(body.completionMetadata.availableKeys).toEqual(['request.method'])
|
||||
// No prompt text is carried client-side — the route owns the instruction.
|
||||
expect(body.completionMetadata.prompt).toBe('')
|
||||
})
|
||||
|
||||
it('throws when the request fails', async () => {
|
||||
@@ -0,0 +1,113 @@
|
||||
import { BASE_PATH } from '@/lib/constants'
|
||||
|
||||
export function stripSqlCodeFences(text: string): string {
|
||||
const trimmed = text.trim()
|
||||
const fenced = trimmed.match(/```(?:sql)?\s*\n?([\s\S]*?)\n?```/i)
|
||||
return (fenced ? fenced[1] : trimmed).trim()
|
||||
}
|
||||
|
||||
const SOURCE_ALIASES: Record<string, string> = {
|
||||
pg_cron_logs: 'postgres_logs',
|
||||
}
|
||||
|
||||
export function detectLogSource(sql: string): string | undefined {
|
||||
// `\b` so only a standalone `source` column counts — an unanchored match reads
|
||||
// the value out of `resource = '...'` or `datasource = '...'` too.
|
||||
const bySource = sql.match(/\bsource\s*=\s*'([^']+)'/i)
|
||||
if (bySource) {
|
||||
const source = bySource[1].toLowerCase()
|
||||
return SOURCE_ALIASES[source] ?? source
|
||||
}
|
||||
const byFrom = sql.match(/\bfrom\s+([a-z_][a-z0-9_]*)/i)
|
||||
if (byFrom) {
|
||||
const table = byFrom[1].toLowerCase()
|
||||
if (table === 'logs') return undefined
|
||||
return SOURCE_ALIASES[table] ?? table
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
export function looksLikeLegacyLogsQuery(sql: string): boolean {
|
||||
const lower = sql.toLowerCase()
|
||||
if (/\bunnest\s*\(/.test(lower)) return true
|
||||
if (/cast\s*\(\s*timestamp\s+as\s+datetime\s*\)/.test(lower)) return true
|
||||
const byFrom = lower.match(/\bfrom\s+([a-z_][a-z0-9_]*)/)
|
||||
return byFrom ? byFrom[1] !== 'logs' : false
|
||||
}
|
||||
|
||||
/**
|
||||
* How long to let the query text settle before re-running the dialect check.
|
||||
* Shared so every surface offering the rewrite reacts on the same cadence.
|
||||
*/
|
||||
export const LEGACY_LOGS_DIALECT_CHECK_DEBOUNCE_MS = 500
|
||||
|
||||
/**
|
||||
* Whether to offer the ClickHouse rewrite for a query. Both the flag and the
|
||||
* dialect check matter: on an org whose logs haven't moved to ClickHouse the
|
||||
* BigQuery text is still *correct*, so offering to rewrite it would break a
|
||||
* working query. Callers layer their own dismissal state on top.
|
||||
*/
|
||||
export function shouldOfferLegacyLogsRewrite({
|
||||
sql,
|
||||
isClickhouseLogsEnabled,
|
||||
}: {
|
||||
sql: string
|
||||
isClickhouseLogsEnabled: boolean
|
||||
}): boolean {
|
||||
return isClickhouseLogsEnabled && looksLikeLegacyLogsQuery(sql)
|
||||
}
|
||||
|
||||
export interface RewriteLogsSqlArgs {
|
||||
sql: string
|
||||
projectRef: string
|
||||
connectionString?: string | null
|
||||
orgSlug?: string
|
||||
authorizationHeader?: string | null
|
||||
availableKeys?: string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Asks the completion route to rewrite a whole BigQuery logs query as ClickHouse
|
||||
* SQL. The prompt itself lives server-side (`lib/ai/clickhouse-logs.ts`) — this
|
||||
* only declares the intent and hands the query over as the selection, the same
|
||||
* shape an inline edit uses, so exactly one place knows how a completion prompt
|
||||
* is assembled.
|
||||
*/
|
||||
export async function rewriteLogsSqlWithAI(args: RewriteLogsSqlArgs) {
|
||||
const { sql, projectRef, connectionString, orgSlug, authorizationHeader, availableKeys } = args
|
||||
|
||||
const response = await fetch(`${BASE_PATH}/api/ai/code/complete`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...(authorizationHeader ? { Authorization: authorizationHeader } : {}),
|
||||
},
|
||||
body: JSON.stringify({
|
||||
projectRef,
|
||||
connectionString,
|
||||
language: 'sql',
|
||||
dialect: 'clickhouse',
|
||||
intent: 'rewrite',
|
||||
orgSlug,
|
||||
completionMetadata: {
|
||||
// The whole query is the selection, so the rewrite replaces all of it and
|
||||
// the route supplies the instruction for the `rewrite` intent.
|
||||
textBeforeCursor: '',
|
||||
textAfterCursor: '',
|
||||
prompt: '',
|
||||
selection: sql,
|
||||
availableKeys,
|
||||
},
|
||||
}),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
throw new Error(errorText || 'Failed to rewrite the query')
|
||||
}
|
||||
|
||||
const raw = await response.json()
|
||||
const rewritten = stripSqlCodeFences(typeof raw === 'string' ? raw : String(raw))
|
||||
if (!rewritten) throw new Error('The assistant returned an empty query')
|
||||
return rewritten
|
||||
}
|
||||
@@ -1,11 +1,14 @@
|
||||
import { useQuery } from '@tanstack/react-query'
|
||||
import { queryOptions, useQuery } from '@tanstack/react-query'
|
||||
|
||||
import { executeAnalyticsSql } from './execute-analytics-sql'
|
||||
import { logsKeys } from './keys'
|
||||
import { logsAllEndpointUrl } from './logs-endpoint'
|
||||
import { analyticsLiteral, safeSql } from './safe-analytics-sql'
|
||||
|
||||
const LOOKBACK_HOURS = 24 * 7
|
||||
|
||||
const KEYS_STALE_TIME = 5 * 60 * 1000
|
||||
|
||||
export async function fetchOtelLogKeys({
|
||||
projectRef,
|
||||
source,
|
||||
@@ -31,15 +34,31 @@ export async function fetchOtelLogKeys({
|
||||
return rows.map((r) => r.key).filter(Boolean)
|
||||
}
|
||||
|
||||
/**
|
||||
* Shared by the reactive hook and imperative `queryClient.fetchQuery` callers, so
|
||||
* a lookup triggered on submit reuses whatever a subscribed component already
|
||||
* cached for the same source (and vice versa).
|
||||
*/
|
||||
export function otelLogKeysQueryOptions({
|
||||
projectRef,
|
||||
source,
|
||||
}: {
|
||||
projectRef: string
|
||||
source: string
|
||||
}) {
|
||||
return queryOptions({
|
||||
queryKey: logsKeys.otelLogKeys(projectRef, source),
|
||||
queryFn: ({ signal }) => fetchOtelLogKeys({ projectRef, source, signal }),
|
||||
staleTime: KEYS_STALE_TIME,
|
||||
})
|
||||
}
|
||||
|
||||
export function useOtelLogKeysQuery(
|
||||
{ projectRef, source }: { projectRef?: string; source?: string },
|
||||
{ enabled = true }: { enabled?: boolean } = {}
|
||||
) {
|
||||
return useQuery({
|
||||
queryKey: ['projects', projectRef, 'otel-log-keys', source],
|
||||
queryFn: ({ signal }) =>
|
||||
fetchOtelLogKeys({ projectRef: projectRef ?? '', source: source ?? '', signal }),
|
||||
...otelLogKeysQueryOptions({ projectRef: projectRef ?? '', source: source ?? '' }),
|
||||
enabled: enabled && Boolean(projectRef) && Boolean(source),
|
||||
staleTime: 5 * 60 * 1000,
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
import { act, waitFor } from '@testing-library/react'
|
||||
import { delay, http, HttpResponse } from 'msw'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
import {
|
||||
INITIAL_LEGACY_LOGS_REWRITE_STATE,
|
||||
legacyLogsRewriteReducer as reduce,
|
||||
useLegacyLogsRewrite,
|
||||
type LegacyLogsRewriteEvent,
|
||||
type LegacyLogsRewriteState,
|
||||
} from './useLegacyLogsRewrite'
|
||||
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
|
||||
import { API_URL } from '@/lib/constants'
|
||||
import { mswServer } from '@/tests/lib/msw'
|
||||
import { renderSqlEditorHook, setupSqlEditorMocks } from '@/tests/lib/sql-editor-test-utils'
|
||||
|
||||
const run = (
|
||||
events: LegacyLogsRewriteEvent[],
|
||||
from: LegacyLogsRewriteState = INITIAL_LEGACY_LOGS_REWRITE_STATE
|
||||
) => events.reduce(reduce, from)
|
||||
|
||||
const FAILED: LegacyLogsRewriteEvent = { type: 'rewriteFailed', message: 'boom' }
|
||||
|
||||
describe('legacyLogsRewriteReducer', () => {
|
||||
it('starts out offering the rewrite', () => {
|
||||
expect(INITIAL_LEGACY_LOGS_REWRITE_STATE).toEqual({ status: 'offered' })
|
||||
})
|
||||
|
||||
it('requesting a rewrite moves to rewriting, and a proposal returns to offered', () => {
|
||||
expect(run([{ type: 'rewriteRequested' }])).toEqual({ status: 'rewriting' })
|
||||
expect(run([{ type: 'rewriteRequested' }, { type: 'rewriteProposed' }])).toEqual({
|
||||
status: 'offered',
|
||||
})
|
||||
})
|
||||
|
||||
it('a failure lands in failed and keeps its message for the UI', () => {
|
||||
expect(run([{ type: 'rewriteRequested' }, FAILED])).toEqual({
|
||||
status: 'failed',
|
||||
message: 'boom',
|
||||
})
|
||||
})
|
||||
|
||||
it('a failure is recoverable — the same request retries it', () => {
|
||||
expect(run([{ type: 'rewriteRequested' }, FAILED, { type: 'rewriteRequested' }])).toEqual({
|
||||
status: 'rewriting',
|
||||
})
|
||||
})
|
||||
|
||||
it('an unchanged response waits for acknowledgement instead of retiring silently', () => {
|
||||
const noop = run([{ type: 'rewriteRequested' }, { type: 'rewriteNoop' }])
|
||||
expect(noop).toEqual({ status: 'noRewriteNeeded' })
|
||||
expect(run([{ type: 'dismissed' }], noop)).toEqual({ status: 'dismissed' })
|
||||
})
|
||||
|
||||
it('both outcomes can be dismissed, and neither can be retried into a new outcome', () => {
|
||||
const failed = run([{ type: 'rewriteRequested' }, FAILED])
|
||||
expect(run([{ type: 'dismissed' }], failed)).toEqual({ status: 'dismissed' })
|
||||
// noRewriteNeeded only accepts dismissal — no silent retry.
|
||||
const noop = run([{ type: 'rewriteRequested' }, { type: 'rewriteNoop' }])
|
||||
expect(run([{ type: 'rewriteRequested' }], noop)).toEqual({ status: 'noRewriteNeeded' })
|
||||
})
|
||||
|
||||
it('dismissal is terminal — nothing resurrects the offer', () => {
|
||||
const dismissed = run([{ type: 'dismissed' }])
|
||||
expect(dismissed).toEqual({ status: 'dismissed' })
|
||||
expect(
|
||||
run(
|
||||
[
|
||||
{ type: 'rewriteRequested' },
|
||||
{ type: 'rewriteProposed' },
|
||||
FAILED,
|
||||
{ type: 'rewriteNoop' },
|
||||
],
|
||||
dismissed
|
||||
)
|
||||
).toEqual({ status: 'dismissed' })
|
||||
})
|
||||
|
||||
it('cannot be dismissed mid-rewrite, so a settling request never resurrects it', () => {
|
||||
expect(run([{ type: 'rewriteRequested' }, { type: 'dismissed' }])).toEqual({
|
||||
status: 'rewriting',
|
||||
})
|
||||
})
|
||||
|
||||
it('ignores events that are invalid for the current state', () => {
|
||||
// No rewrite in flight to settle.
|
||||
expect(run([{ type: 'rewriteProposed' }])).toEqual({ status: 'offered' })
|
||||
expect(run([{ type: 'rewriteNoop' }])).toEqual({ status: 'offered' })
|
||||
expect(run([FAILED])).toEqual({ status: 'offered' })
|
||||
// Already rewriting; a second request is a no-op rather than a restart.
|
||||
expect(run([{ type: 'rewriteRequested' }, { type: 'rewriteRequested' }])).toEqual({
|
||||
status: 'rewriting',
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('useLegacyLogsRewrite — dismiss', () => {
|
||||
// No detectable source, so key discovery is skipped and the only outbound
|
||||
// request is the completion call we control below.
|
||||
const SQL_WITHOUT_SOURCE = 'select 1 from logs limit 5'
|
||||
|
||||
/** Keeps a requested rewrite in flight for the duration of the test. */
|
||||
function stallTheRewrite() {
|
||||
mswServer.use(
|
||||
http.post(`${API_URL}/ai/code/complete`, async () => {
|
||||
await delay(10_000)
|
||||
return HttpResponse.json('select 1 from logs')
|
||||
})
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Exposes the resolved project alongside the hook: `requestRewrite` no-ops
|
||||
* without a project ref, so tests must wait for that query before asking.
|
||||
*/
|
||||
async function renderDismissHarness() {
|
||||
const onDismissed = vi.fn()
|
||||
const utils = renderSqlEditorHook(() => {
|
||||
const { data: project } = useSelectedProjectQuery()
|
||||
const rewrite = useLegacyLogsRewrite({
|
||||
readSql: () => SQL_WITHOUT_SOURCE,
|
||||
onProposal: vi.fn(),
|
||||
onDismissed,
|
||||
})
|
||||
return { ...rewrite, projectRef: project?.ref }
|
||||
})
|
||||
await waitFor(() => expect(utils.result.current.projectRef).toBe('default'))
|
||||
return { ...utils, onDismissed }
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
setupSqlEditorMocks()
|
||||
})
|
||||
|
||||
it('dismisses from the offer and reports it', async () => {
|
||||
const { result, onDismissed } = await renderDismissHarness()
|
||||
|
||||
await act(async () => {
|
||||
result.current.dismiss()
|
||||
})
|
||||
|
||||
expect(result.current.state.status).toBe('dismissed')
|
||||
expect(onDismissed).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('does not report a dismissal the machine rejects mid-rewrite', async () => {
|
||||
stallTheRewrite()
|
||||
const { result, onDismissed } = await renderDismissHarness()
|
||||
|
||||
act(() => {
|
||||
void result.current.requestRewrite()
|
||||
})
|
||||
await waitFor(() => expect(result.current.state.status).toBe('rewriting'))
|
||||
|
||||
await act(async () => {
|
||||
result.current.dismiss()
|
||||
})
|
||||
|
||||
// Persisting this would suppress an offer that's still live.
|
||||
expect(onDismissed).not.toHaveBeenCalled()
|
||||
expect(result.current.state.status).toBe('rewriting')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,162 @@
|
||||
import { useReducer } from 'react'
|
||||
|
||||
import { constructHeaders } from '@/data/fetchers'
|
||||
import { rewriteLogsSqlWithAI } from '@/data/logs/logs-sql-rewrite'
|
||||
import { useLogsAttributeKeys } from '@/hooks/analytics/useLogsAttributeKeys'
|
||||
import { useSelectedOrganizationQuery } from '@/hooks/misc/useSelectedOrganization'
|
||||
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
|
||||
import { getErrorMessage } from '@/lib/get-error-message'
|
||||
|
||||
export type LegacyLogsRewriteState =
|
||||
| { status: 'offered' }
|
||||
| { status: 'rewriting' }
|
||||
| { status: 'failed'; message: string }
|
||||
| { status: 'noRewriteNeeded' }
|
||||
| { status: 'dismissed' }
|
||||
|
||||
export type LegacyLogsRewriteEvent =
|
||||
| { type: 'rewriteRequested' }
|
||||
| { type: 'rewriteProposed' }
|
||||
| { type: 'rewriteFailed'; message: string }
|
||||
| { type: 'rewriteNoop' }
|
||||
| { type: 'dismissed' }
|
||||
|
||||
export const INITIAL_LEGACY_LOGS_REWRITE_STATE: LegacyLogsRewriteState = { status: 'offered' }
|
||||
|
||||
/**
|
||||
* The events each state accepts. Anything absent is an invalid transition and
|
||||
* leaves the state untouched — notably `dismissed` is terminal, and the offer
|
||||
* can't be dismissed mid-rewrite.
|
||||
*/
|
||||
const VALID_EVENTS: {
|
||||
[S in LegacyLogsRewriteState['status']]: readonly LegacyLogsRewriteEvent['type'][]
|
||||
} = {
|
||||
offered: ['rewriteRequested', 'dismissed'],
|
||||
rewriting: ['rewriteProposed', 'rewriteFailed', 'rewriteNoop'],
|
||||
// A failure is recoverable: the same Rewrite control retries it.
|
||||
failed: ['rewriteRequested', 'dismissed'],
|
||||
noRewriteNeeded: ['dismissed'],
|
||||
dismissed: [],
|
||||
}
|
||||
|
||||
function targetState(event: LegacyLogsRewriteEvent): LegacyLogsRewriteState {
|
||||
switch (event.type) {
|
||||
case 'rewriteRequested':
|
||||
return { status: 'rewriting' }
|
||||
// The proposal is handed to the caller; the offer returns to idle behind it so
|
||||
// it's ready again if the user discards the proposal.
|
||||
case 'rewriteProposed':
|
||||
return { status: 'offered' }
|
||||
case 'rewriteFailed':
|
||||
return { status: 'failed', message: event.message }
|
||||
case 'rewriteNoop':
|
||||
return { status: 'noRewriteNeeded' }
|
||||
case 'dismissed':
|
||||
return { status: 'dismissed' }
|
||||
}
|
||||
}
|
||||
|
||||
export function legacyLogsRewriteReducer(
|
||||
state: LegacyLogsRewriteState,
|
||||
event: LegacyLogsRewriteEvent
|
||||
): LegacyLogsRewriteState {
|
||||
if (!VALID_EVENTS[state.status].includes(event.type)) return state
|
||||
return targetState(event)
|
||||
}
|
||||
|
||||
const CHANGED_WHILE_REWRITING_MESSAGE =
|
||||
'The query changed while the Assistant was working, so the rewrite no longer matches it.'
|
||||
|
||||
const NO_RESPONSE_MESSAGE = 'The Assistant did not respond. Try again.'
|
||||
|
||||
export type LegacyLogsRewriteProposal = { original: string; modified: string }
|
||||
|
||||
type UseLegacyLogsRewriteArgs = {
|
||||
/**
|
||||
* Reads the query to rewrite at the moment the user asks. A callback rather than
|
||||
* a value so the rewrite operates on exactly what the user sees, not on whatever
|
||||
* a surface last rendered.
|
||||
*/
|
||||
readSql: () => string
|
||||
/** Receives a rewrite worth reviewing. Each surface routes this to its own diff. */
|
||||
onProposal: (proposal: LegacyLogsRewriteProposal) => void
|
||||
/**
|
||||
* Called when the offer is dismissed, for surfaces that persist that. The
|
||||
* machine covers the current session only — a surface that remembers dismissals
|
||||
* across sessions layers that on top of its own visibility check, since a value
|
||||
* read from storage isn't available in time to seed the machine.
|
||||
*/
|
||||
onDismissed?: () => void
|
||||
}
|
||||
|
||||
/**
|
||||
* Owns the BigQuery → ClickHouse rewrite request end to end: key discovery, the
|
||||
* completion call, the stale-edit guard, no-op detection, and the resulting state.
|
||||
*/
|
||||
export function useLegacyLogsRewrite({
|
||||
readSql,
|
||||
onProposal,
|
||||
onDismissed,
|
||||
}: UseLegacyLogsRewriteArgs) {
|
||||
const { data: project } = useSelectedProjectQuery()
|
||||
const { data: organization } = useSelectedOrganizationQuery()
|
||||
const projectRef = project?.ref
|
||||
|
||||
const [state, dispatch] = useReducer(legacyLogsRewriteReducer, INITIAL_LEGACY_LOGS_REWRITE_STATE)
|
||||
|
||||
const { fetchAttributeKeys } = useLogsAttributeKeys()
|
||||
|
||||
const requestRewrite = async () => {
|
||||
if (!projectRef) return console.error('[useLegacyLogsRewrite] Project ref is required')
|
||||
|
||||
const currentSql = readSql()
|
||||
if (currentSql.trim().length === 0) return
|
||||
|
||||
dispatch({ type: 'rewriteRequested' })
|
||||
try {
|
||||
const [headerData, availableKeys] = await Promise.all([
|
||||
constructHeaders(),
|
||||
fetchAttributeKeys(currentSql),
|
||||
])
|
||||
const rewritten = await rewriteLogsSqlWithAI({
|
||||
sql: currentSql,
|
||||
projectRef,
|
||||
connectionString: project?.connectionString,
|
||||
orgSlug: organization?.slug,
|
||||
authorizationHeader: headerData.get('Authorization'),
|
||||
availableKeys,
|
||||
})
|
||||
|
||||
// The user may have kept typing while the model worked; a proposal built from
|
||||
// stale text would clobber those edits when accepted.
|
||||
if (readSql() !== currentSql) {
|
||||
dispatch({ type: 'rewriteFailed', message: CHANGED_WHILE_REWRITING_MESSAGE })
|
||||
return
|
||||
}
|
||||
|
||||
// An unchanged response means the query already runs on ClickHouse and the
|
||||
// dialect heuristic was over-eager. Proposing it would show an empty diff.
|
||||
if (rewritten.trim() === currentSql.trim()) {
|
||||
dispatch({ type: 'rewriteNoop' })
|
||||
return
|
||||
}
|
||||
|
||||
onProposal({ original: currentSql, modified: rewritten })
|
||||
dispatch({ type: 'rewriteProposed' })
|
||||
} catch (error) {
|
||||
dispatch({ type: 'rewriteFailed', message: getErrorMessage(error, NO_RESPONSE_MESSAGE) })
|
||||
}
|
||||
}
|
||||
|
||||
const dismiss = () => {
|
||||
const dismissed: LegacyLogsRewriteEvent = { type: 'dismissed' }
|
||||
// The transition table is the contract, not the UI that happens to disable the
|
||||
// control: never report a dismissal the machine rejected (mid-rewrite, say),
|
||||
// or a surface that persists it would suppress an offer that's still live.
|
||||
if (legacyLogsRewriteReducer(state, dismissed) === state) return
|
||||
dispatch(dismissed)
|
||||
onDismissed?.()
|
||||
}
|
||||
|
||||
return { state, requestRewrite, dismiss }
|
||||
}
|
||||
@@ -0,0 +1,124 @@
|
||||
import { act, waitFor } from '@testing-library/react'
|
||||
import { HttpResponse } from 'msw'
|
||||
import { beforeEach, describe, expect, it } from 'vitest'
|
||||
|
||||
import { useLogsAttributeKeys } from './useLogsAttributeKeys'
|
||||
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
|
||||
import { addAPIMock } from '@/tests/lib/msw'
|
||||
import { renderSqlEditorHook, setupSqlEditorMocks } from '@/tests/lib/sql-editor-test-utils'
|
||||
|
||||
const OTEL_ENDPOINT = '/platform/projects/:ref/analytics/endpoints/logs.all.otel'
|
||||
|
||||
const queryFor = (source: string) => `select 1 from logs where source = '${source}'`
|
||||
|
||||
/** Records the SQL of every key-discovery request so we can count them. */
|
||||
function mockKeyDiscovery({ fails = false }: { fails?: boolean } = {}) {
|
||||
const requests: string[] = []
|
||||
addAPIMock({
|
||||
method: 'post',
|
||||
path: OTEL_ENDPOINT,
|
||||
response: async ({ request }) => {
|
||||
const body = (await request.clone().json()) as { sql: string }
|
||||
requests.push(body.sql)
|
||||
if (fails) return HttpResponse.json({ message: 'boom' }, { status: 500 })
|
||||
return HttpResponse.json({ result: [{ key: 'request.method' }] })
|
||||
},
|
||||
})
|
||||
return requests
|
||||
}
|
||||
|
||||
/**
|
||||
* Exposes the resolved project alongside the hook. Discovery needs a project ref,
|
||||
* so tests must wait for that query before asking — otherwise a lookup no-ops and
|
||||
* the request counts below would pass for the wrong reason.
|
||||
*/
|
||||
function useKeysHarness() {
|
||||
const { data: project } = useSelectedProjectQuery()
|
||||
const { fetchAttributeKeys } = useLogsAttributeKeys()
|
||||
return { projectRef: project?.ref, fetchAttributeKeys }
|
||||
}
|
||||
|
||||
async function renderReadyHarness() {
|
||||
const utils = renderSqlEditorHook(useKeysHarness)
|
||||
await waitFor(() => expect(utils.result.current.projectRef).toBe('default'))
|
||||
return utils
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
setupSqlEditorMocks()
|
||||
})
|
||||
|
||||
describe('useLogsAttributeKeys', () => {
|
||||
it('makes no request until asked', async () => {
|
||||
const requests = mockKeyDiscovery()
|
||||
|
||||
await renderReadyHarness()
|
||||
|
||||
expect(requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('returns the discovered keys for the query source when asked', async () => {
|
||||
const requests = mockKeyDiscovery()
|
||||
const { result } = await renderReadyHarness()
|
||||
|
||||
let keys: string[] | undefined
|
||||
await act(async () => {
|
||||
keys = await result.current.fetchAttributeKeys(queryFor('edge_logs'))
|
||||
})
|
||||
|
||||
expect(keys).toEqual(['request.method'])
|
||||
expect(requests).toHaveLength(1)
|
||||
expect(requests[0]).toContain("source = 'edge_logs'")
|
||||
})
|
||||
|
||||
it('reuses the cached result for a source already looked up', async () => {
|
||||
const requests = mockKeyDiscovery()
|
||||
const { result } = await renderReadyHarness()
|
||||
|
||||
await act(async () => {
|
||||
await result.current.fetchAttributeKeys(queryFor('edge_logs'))
|
||||
await result.current.fetchAttributeKeys(queryFor('edge_logs'))
|
||||
})
|
||||
|
||||
expect(requests).toHaveLength(1)
|
||||
})
|
||||
|
||||
it('looks up a different source separately', async () => {
|
||||
const requests = mockKeyDiscovery()
|
||||
const { result } = await renderReadyHarness()
|
||||
|
||||
await act(async () => {
|
||||
await result.current.fetchAttributeKeys(queryFor('edge_logs'))
|
||||
await result.current.fetchAttributeKeys(queryFor('postgres_logs'))
|
||||
})
|
||||
|
||||
expect(requests).toHaveLength(2)
|
||||
expect(requests[1]).toContain("source = 'postgres_logs'")
|
||||
})
|
||||
|
||||
it('resolves undefined without a request when no source is detectable', async () => {
|
||||
const requests = mockKeyDiscovery()
|
||||
const { result } = await renderReadyHarness()
|
||||
|
||||
let keys: string[] | undefined
|
||||
await act(async () => {
|
||||
keys = await result.current.fetchAttributeKeys('select 1 from logs limit 5')
|
||||
})
|
||||
|
||||
expect(keys).toBeUndefined()
|
||||
expect(requests).toHaveLength(0)
|
||||
})
|
||||
|
||||
it('resolves undefined rather than throwing when discovery fails', async () => {
|
||||
mockKeyDiscovery({ fails: true })
|
||||
const { result } = await renderReadyHarness()
|
||||
|
||||
let keys: string[] | undefined
|
||||
await act(async () => {
|
||||
keys = await result.current.fetchAttributeKeys(queryFor('edge_logs'))
|
||||
})
|
||||
|
||||
// Keys are an enhancement — a failed lookup must not block the caller.
|
||||
expect(keys).toBeUndefined()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,43 @@
|
||||
import { useQueryClient } from '@tanstack/react-query'
|
||||
import { useCallback } from 'react'
|
||||
|
||||
import { detectLogSource } from '@/data/logs/logs-sql-rewrite'
|
||||
import { otelLogKeysQueryOptions } from '@/data/logs/otel-log-keys-query'
|
||||
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
|
||||
|
||||
/**
|
||||
* Looks up the real `log_attributes` keys for whichever source a logs query
|
||||
* targets. The AI flows pass these along so the model uses exact dotted paths
|
||||
* instead of inventing them.
|
||||
*
|
||||
* Deliberately imperative: discovery aggregates a week of logs, and the source is
|
||||
* derived from query text the user is editing, so anything reactive fires requests
|
||||
* for half-typed source names. Fetching at submit time means one request per
|
||||
* action the user actually took. It still goes through the query client, so a
|
||||
* result already cached for that source (by an earlier submit, or by a component
|
||||
* subscribed via `useOtelLogKeysQuery`) is reused rather than refetched.
|
||||
*
|
||||
* Keys are an enhancement, never a requirement — a failed or impossible lookup
|
||||
* resolves to `undefined` and the caller proceeds without them.
|
||||
*/
|
||||
export function useLogsAttributeKeys() {
|
||||
const queryClient = useQueryClient()
|
||||
const { data: project } = useSelectedProjectQuery()
|
||||
const projectRef = project?.ref
|
||||
|
||||
const fetchAttributeKeys = useCallback(
|
||||
async (sql: string): Promise<string[] | undefined> => {
|
||||
const source = detectLogSource(sql)
|
||||
if (!projectRef || source === undefined) return undefined
|
||||
|
||||
try {
|
||||
return await queryClient.fetchQuery(otelLogKeysQueryOptions({ projectRef, source }))
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
},
|
||||
[projectRef, queryClient]
|
||||
)
|
||||
|
||||
return { fetchAttributeKeys }
|
||||
}
|
||||
+46
-92
@@ -1,12 +1,34 @@
|
||||
import { BASE_PATH } from '@/lib/constants'
|
||||
/**
|
||||
* Everything the model needs to know about the ClickHouse-backed Supabase `logs`
|
||||
* table, in one place. Both ClickHouse completion flows — the inline "edit this
|
||||
* query" path and the whole-query BigQuery rewrite — are assembled from these,
|
||||
* server-side in `pages/api/ai/code/complete.ts`, so there is exactly one
|
||||
* description of the schema to keep current.
|
||||
*/
|
||||
|
||||
export const LOGS_SCHEMA_REFERENCE = `The logs table (ClickHouse) has these columns:
|
||||
/** System-prompt half: the dialect rules that hold for every ClickHouse request. */
|
||||
export const CLICKHOUSE_LOGS_COMPLETION_INSTRUCTIONS = `
|
||||
# Supabase logs SQL (ClickHouse)
|
||||
You are writing SQL for Supabase logs, which run on a ClickHouse-backed engine. This is NOT Postgres and NOT BigQuery. Output valid ClickHouse SQL only.
|
||||
- All logs are in a single table named \`logs\`, keyed by a \`source\` column. There are no per-service tables (no \`edge_logs\`, \`postgres_logs\`, and so on) and no \`unnest\` joins.
|
||||
- Per-source fields live in the \`log_attributes\` Map(String, String), read as \`log_attributes['key']\`. Map values are strings, so wrap numeric ones in \`toInt32OrZero(...)\`.
|
||||
- Use ClickHouse functions, not Postgres or BigQuery ones. Use \`match(col, 'regex')\` or \`col ILIKE '%text%'\` instead of \`regexp_contains\`, \`count()\` instead of \`count(*)\`, and select the \`timestamp\` column directly instead of \`cast(timestamp as datetime)\`.
|
||||
- Do not quote identifiers with double quotes and do not append a trailing semicolon.
|
||||
- Do not use \`select *\`, this is disallowed by the backend.
|
||||
`
|
||||
|
||||
/**
|
||||
* User-message half: the concrete shape of the table. Complements the dialect
|
||||
* rules above — this is the part that changes when a log source or its fields
|
||||
* change, and it is the ClickHouse counterpart to the Postgres DDL section.
|
||||
*/
|
||||
const CLICKHOUSE_LOGS_COLUMN_REFERENCE = `The logs table has these columns:
|
||||
- id (String)
|
||||
- timestamp (DateTime64, UTC) formatted like 2026-06-22T09:34:06.215000 (ISO 8601, microsecond precision, no trailing Z)
|
||||
- event_message (String): the raw log line
|
||||
- severity_text (String): log level when present
|
||||
- source (String): the service the log belongs to. Always filter by it, e.g. where source = 'edge_logs'.
|
||||
- log_attributes (Map(String, String)): structured per-source fields, read as log_attributes['key']. Values are strings, so wrap numeric ones in toInt32OrZero(...) for comparisons.
|
||||
- log_attributes (Map(String, String)): structured per-source fields, read as log_attributes['key']
|
||||
|
||||
Sources and their common log_attributes keys:
|
||||
- edge_logs: request.method, request.path, request.search, response.status_code, identifier
|
||||
@@ -17,7 +39,7 @@ Sources and their common log_attributes keys:
|
||||
- function_logs: event_type, function_id, execution_id, level
|
||||
- storage_logs, realtime_logs, postgrest_logs, supavisor_logs, pgbouncer_logs: mostly id, timestamp, event_message, with extra fields in log_attributes
|
||||
|
||||
Rules: always filter by source; the editor applies the selected time range so a timestamp filter is usually unnecessary; the old BigQuery unnest joins become log_attributes['key'] lookups (drop the metadata root).`
|
||||
The editor applies the user's selected time range as a request parameter, so an explicit timestamp filter is usually unnecessary.`
|
||||
|
||||
function renderAvailableKeys(availableKeys?: string[]): string {
|
||||
if (!availableKeys || availableKeys.length === 0) return ''
|
||||
@@ -26,10 +48,25 @@ function renderAvailableKeys(availableKeys?: string[]): string {
|
||||
${list}\n`
|
||||
}
|
||||
|
||||
export function buildClickhouseRewritePrompt(sql: string, availableKeys?: string[]): string {
|
||||
return `${LOGS_SCHEMA_REFERENCE}
|
||||
${renderAvailableKeys(availableKeys)}
|
||||
Convert the BigQuery logs query below to ClickHouse SQL for the logs table. There are no per-service tables and no unnest joins in ClickHouse. Follow these rules exactly:
|
||||
/**
|
||||
* The ClickHouse schema section of the user message — the counterpart to
|
||||
* `buildDatabaseSchemaSection` for Postgres. `availableKeys` are the real
|
||||
* `log_attributes` keys observed for the query's source, when the caller
|
||||
* discovered them.
|
||||
*/
|
||||
export function buildClickhouseLogsSchemaSection(availableKeys?: string[]): string {
|
||||
return `${CLICKHOUSE_LOGS_COLUMN_REFERENCE}\n${renderAvailableKeys(availableKeys)}`
|
||||
}
|
||||
|
||||
/**
|
||||
* The instruction for the whole-query BigQuery → ClickHouse rewrite. Used in
|
||||
* place of a user instruction when the request's intent is `rewrite`.
|
||||
*
|
||||
* It states that a rewrite is REQUIRED: the system prompt covers writing and
|
||||
* editing ClickHouse SQL generally, and without an explicit demand here the model
|
||||
* echoes the input back, which surfaces to the user as an empty diff.
|
||||
*/
|
||||
export const CLICKHOUSE_LOGS_REWRITE_INSTRUCTION = `Your task is to REWRITE the selected query. It is BigQuery SQL and will not run on ClickHouse, so returning it unchanged is likely wrong — every rule below that applies must be applied.
|
||||
|
||||
1. Replace the FROM table with the single logs table and filter by source. The old table name is the source value: "from postgres_logs as t" becomes "from logs where source = 'postgres_logs'". This is required, never select from a table like postgres_logs or edge_logs.
|
||||
2. Remove every join that unnests metadata or its structs. This includes "cross join unnest(...)" and "left join unnest(...) on true".
|
||||
@@ -56,87 +93,4 @@ where source = 'postgres_logs'
|
||||
and log_attributes['parsed.error_severity'] in ('ERROR', 'FATAL', 'PANIC')
|
||||
group by log_attributes['parsed.error_severity']
|
||||
order by count desc
|
||||
limit 100
|
||||
|
||||
Reply with ONLY the rewritten SQL query: no explanation, no comments, and no markdown code fences.
|
||||
|
||||
${sql}`
|
||||
}
|
||||
|
||||
export function stripSqlCodeFences(text: string): string {
|
||||
const trimmed = text.trim()
|
||||
const fenced = trimmed.match(/```(?:sql)?\s*\n?([\s\S]*?)\n?```/i)
|
||||
return (fenced ? fenced[1] : trimmed).trim()
|
||||
}
|
||||
|
||||
const SOURCE_ALIASES: Record<string, string> = {
|
||||
pg_cron_logs: 'postgres_logs',
|
||||
}
|
||||
|
||||
export function detectLogSource(sql: string): string | undefined {
|
||||
const bySource = sql.match(/source\s*=\s*'([^']+)'/i)
|
||||
if (bySource) {
|
||||
const source = bySource[1].toLowerCase()
|
||||
return SOURCE_ALIASES[source] ?? source
|
||||
}
|
||||
const byFrom = sql.match(/\bfrom\s+([a-z_][a-z0-9_]*)/i)
|
||||
if (byFrom) {
|
||||
const table = byFrom[1].toLowerCase()
|
||||
if (table === 'logs') return undefined
|
||||
return SOURCE_ALIASES[table] ?? table
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
export function looksLikeLegacyLogsQuery(sql: string): boolean {
|
||||
const lower = sql.toLowerCase()
|
||||
if (/\bunnest\s*\(/.test(lower)) return true
|
||||
if (/cast\s*\(\s*timestamp\s+as\s+datetime\s*\)/.test(lower)) return true
|
||||
const byFrom = lower.match(/\bfrom\s+([a-z_][a-z0-9_]*)/)
|
||||
return byFrom ? byFrom[1] !== 'logs' : false
|
||||
}
|
||||
|
||||
export interface RewriteLogsSqlArgs {
|
||||
sql: string
|
||||
projectRef: string
|
||||
connectionString?: string | null
|
||||
orgSlug?: string
|
||||
authorizationHeader?: string | null
|
||||
availableKeys?: string[]
|
||||
}
|
||||
|
||||
export async function rewriteLogsSqlWithAI(args: RewriteLogsSqlArgs) {
|
||||
const { sql, projectRef, connectionString, orgSlug, authorizationHeader, availableKeys } = args
|
||||
|
||||
const response = await fetch(`${BASE_PATH}/api/ai/code/complete`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...(authorizationHeader ? { Authorization: authorizationHeader } : {}),
|
||||
},
|
||||
body: JSON.stringify({
|
||||
projectRef,
|
||||
connectionString,
|
||||
language: 'sql',
|
||||
dialect: 'clickhouse',
|
||||
orgSlug,
|
||||
completionMetadata: {
|
||||
textBeforeCursor: '',
|
||||
textAfterCursor: '',
|
||||
language: 'pgsql',
|
||||
prompt: buildClickhouseRewritePrompt(sql, availableKeys),
|
||||
selection: sql,
|
||||
},
|
||||
}),
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text()
|
||||
throw new Error(errorText || 'Failed to rewrite the query')
|
||||
}
|
||||
|
||||
const raw = await response.json()
|
||||
const rewritten = stripSqlCodeFences(typeof raw === 'string' ? raw : String(raw))
|
||||
if (!rewritten) throw new Error('The assistant returned an empty query')
|
||||
return rewritten
|
||||
}
|
||||
limit 100`
|
||||
@@ -786,15 +786,6 @@ export const SQL_COMPLETION_INSTRUCTIONS = `
|
||||
Do not quote identifiers unless they actually require it (uppercase letters, reserved words, or special characters). Plain lowercase identifiers should not be quoted.
|
||||
`
|
||||
|
||||
export const CLICKHOUSE_LOGS_COMPLETION_INSTRUCTIONS = `
|
||||
# Supabase logs SQL (ClickHouse)
|
||||
You are writing SQL for Supabase logs, which run on a ClickHouse-backed engine. This is NOT Postgres and NOT BigQuery. Output valid ClickHouse SQL only.
|
||||
- All logs are in a single table named \`logs\`, keyed by a \`source\` column. There are no per-service tables (no \`edge_logs\`, \`postgres_logs\`, and so on) and no \`unnest\` joins.
|
||||
- Per-source fields live in the \`log_attributes\` Map(String, String), read as \`log_attributes['key']\`. Map values are strings, so wrap numeric ones in \`toInt32OrZero(...)\`.
|
||||
- Use ClickHouse functions, not Postgres or BigQuery ones. Use \`match(col, 'regex')\` or \`col ILIKE '%text%'\` instead of \`regexp_contains\`, \`count()\` instead of \`count(*)\`, and select the \`timestamp\` column directly instead of \`cast(timestamp as datetime)\`.
|
||||
- Do not quote identifiers with double quotes and do not append a trailing semicolon.
|
||||
`
|
||||
|
||||
export const LIMITATIONS_PROMPT = `
|
||||
# Limitations
|
||||
- You are to only answer Supabase, database, or edge function related questions. All other questions should be declined with a polite message.
|
||||
|
||||
@@ -3,44 +3,48 @@ import { describe, expect, it } from 'vitest'
|
||||
import { getErrorMessage } from './get-error-message'
|
||||
|
||||
describe('getErrorMessage', () => {
|
||||
it('returns null for null', () => {
|
||||
expect(getErrorMessage(null)).toBe(null)
|
||||
})
|
||||
|
||||
it('returns null for undefined', () => {
|
||||
expect(getErrorMessage(undefined)).toBe(null)
|
||||
})
|
||||
|
||||
it('returns the string for string errors', () => {
|
||||
expect(getErrorMessage('Something went wrong')).toBe('Something went wrong')
|
||||
expect(getErrorMessage('')).toBe('')
|
||||
})
|
||||
|
||||
it('returns the message for Error instances', () => {
|
||||
expect(getErrorMessage(new Error('Failed to load'))).toBe('Failed to load')
|
||||
expect(getErrorMessage(new TypeError('Invalid type'))).toBe('Invalid type')
|
||||
})
|
||||
|
||||
it('returns the message property for objects with message', () => {
|
||||
it('returns a string throw, trimmed', () => {
|
||||
expect(getErrorMessage('Something went wrong')).toBe('Something went wrong')
|
||||
expect(getErrorMessage(' boom ')).toBe('boom')
|
||||
})
|
||||
|
||||
it('reads a string message off a plain object', () => {
|
||||
expect(getErrorMessage({ message: 'Custom error' })).toBe('Custom error')
|
||||
expect(getErrorMessage({ message: 123 })).toBe('123')
|
||||
expect(getErrorMessage({ message: null })).toBe('null')
|
||||
})
|
||||
|
||||
it('converts other types to string', () => {
|
||||
expect(getErrorMessage(123)).toBe('123')
|
||||
expect(getErrorMessage(true)).toBe('true')
|
||||
expect(getErrorMessage(false)).toBe('false')
|
||||
expect(getErrorMessage({})).toBe('[object Object]')
|
||||
expect(getErrorMessage([])).toBe('')
|
||||
it('returns null when there is no usable message', () => {
|
||||
expect(getErrorMessage(null)).toBe(null)
|
||||
expect(getErrorMessage(undefined)).toBe(null)
|
||||
expect(getErrorMessage('')).toBe(null)
|
||||
expect(getErrorMessage(' ')).toBe(null)
|
||||
expect(getErrorMessage(123)).toBe(null)
|
||||
expect(getErrorMessage(true)).toBe(null)
|
||||
expect(getErrorMessage([])).toBe(null)
|
||||
})
|
||||
|
||||
it('handles objects without message property', () => {
|
||||
expect(getErrorMessage({ code: 500 })).toBe('[object Object]')
|
||||
expect(getErrorMessage({ error: 'test' })).toBe('[object Object]')
|
||||
it('never surfaces a stringified object as the message', () => {
|
||||
expect(getErrorMessage({})).toBe(null)
|
||||
expect(getErrorMessage({ code: 500 })).toBe(null)
|
||||
expect(getErrorMessage({ error: 'test' })).toBe(null)
|
||||
expect(getErrorMessage({ message: 123 })).toBe(null)
|
||||
expect(getErrorMessage({ message: null })).toBe(null)
|
||||
expect(getErrorMessage({ message: { nested: 'error' } })).toBe(null)
|
||||
})
|
||||
|
||||
it('handles nested error objects', () => {
|
||||
expect(getErrorMessage({ message: { nested: 'error' } })).toBe('[object Object]')
|
||||
it('returns the fallback instead of null when one is given', () => {
|
||||
expect(getErrorMessage(null, 'fallback')).toBe('fallback')
|
||||
expect(getErrorMessage({}, 'fallback')).toBe('fallback')
|
||||
expect(getErrorMessage({ message: ' ' }, 'fallback')).toBe('fallback')
|
||||
expect(getErrorMessage(123, 'fallback')).toBe('fallback')
|
||||
})
|
||||
|
||||
it('prefers a real message over the fallback', () => {
|
||||
expect(getErrorMessage(new Error('Rewrite failed'), 'fallback')).toBe('Rewrite failed')
|
||||
expect(getErrorMessage({ message: 'Bad request' }, 'fallback')).toBe('Bad request')
|
||||
})
|
||||
})
|
||||
@@ -1,12 +1,22 @@
|
||||
/**
|
||||
* Extracts a human-readable error message from various error types.
|
||||
* Extracts a human-readable message from an unknown thrown value, optionally
|
||||
* falling back when there isn't one.
|
||||
*/
|
||||
export function getErrorMessage(error: unknown): string | null {
|
||||
if (error === null || error === undefined) return null
|
||||
if (typeof error === 'string') return error
|
||||
if (error instanceof Error) return error.message
|
||||
if (typeof error === 'object' && error !== null && 'message' in error) {
|
||||
return String(error.message)
|
||||
export function getErrorMessage(error: unknown): string | null
|
||||
export function getErrorMessage(error: unknown, fallback: string): string
|
||||
export function getErrorMessage(error: unknown, fallback?: string): string | null {
|
||||
if (typeof error === 'string') {
|
||||
const trimmed = error.trim()
|
||||
if (trimmed.length > 0) return trimmed
|
||||
}
|
||||
return String(error)
|
||||
|
||||
if (typeof error === 'object' && error !== null && 'message' in error) {
|
||||
const { message } = error
|
||||
if (typeof message === 'string') {
|
||||
const trimmed = message.trim()
|
||||
if (trimmed.length > 0) return trimmed
|
||||
}
|
||||
}
|
||||
|
||||
return fallback ?? null
|
||||
}
|
||||
@@ -8,10 +8,14 @@ import z from 'zod'
|
||||
import { executeSql } from '@/data/sql/execute-sql-mutation'
|
||||
import { AiOptInLevel } from '@/hooks/misc/useOrgOptedIntoAi'
|
||||
import { getOrgAIDetails } from '@/lib/ai/ai-details'
|
||||
import {
|
||||
buildClickhouseLogsSchemaSection,
|
||||
CLICKHOUSE_LOGS_COMPLETION_INSTRUCTIONS,
|
||||
CLICKHOUSE_LOGS_REWRITE_INSTRUCTION,
|
||||
} from '@/lib/ai/clickhouse-logs'
|
||||
import { getModel } from '@/lib/ai/model'
|
||||
import { DEFAULT_COMPLETION_MODEL, LOGS_REWRITE_MODEL } from '@/lib/ai/model.utils'
|
||||
import {
|
||||
CLICKHOUSE_LOGS_COMPLETION_INSTRUCTIONS,
|
||||
COMPLETION_PROMPT,
|
||||
EDGE_FUNCTION_PROMPT,
|
||||
PG_BEST_PRACTICES,
|
||||
@@ -124,12 +128,24 @@ const requestBodySchema = z.object({
|
||||
textAfterCursor: z.string(),
|
||||
prompt: z.string(),
|
||||
selection: z.string(),
|
||||
/**
|
||||
* The real `log_attributes` keys observed for the query's source, when the
|
||||
* client discovered them. ClickHouse-only — there is no schema to fetch
|
||||
* server-side for the logs table the way there is for Postgres DDL.
|
||||
*/
|
||||
availableKeys: z.array(z.string()).optional(),
|
||||
}),
|
||||
projectRef: z.string(),
|
||||
connectionString: z.string().nullish(),
|
||||
orgSlug: z.string().optional(),
|
||||
language: z.string().optional(),
|
||||
dialect: z.enum(['postgres', 'clickhouse']).optional(),
|
||||
/**
|
||||
* What the caller wants done. `rewrite` swaps the user instruction for the
|
||||
* canonical BigQuery → ClickHouse rewrite instruction, so the client never has
|
||||
* to carry prompt text. ClickHouse-only; defaults to `edit`.
|
||||
*/
|
||||
intent: z.enum(['edit', 'rewrite']).optional(),
|
||||
})
|
||||
|
||||
async function handler(req: NextApiRequest, res: NextApiResponse) {
|
||||
@@ -150,8 +166,10 @@ async function handler(req: NextApiRequest, res: NextApiResponse) {
|
||||
return res.status(400).json({ error: 'Invalid request body', issues: parseError.issues })
|
||||
}
|
||||
|
||||
const { completionMetadata, projectRef, connectionString, orgSlug, language, dialect } = data
|
||||
const { textBeforeCursor, textAfterCursor, prompt, selection } = completionMetadata
|
||||
const { completionMetadata, projectRef, connectionString, orgSlug, language, dialect, intent } =
|
||||
data
|
||||
const { textBeforeCursor, textAfterCursor, prompt, selection, availableKeys } =
|
||||
completionMetadata
|
||||
const isClickhouse = dialect === 'clickhouse'
|
||||
|
||||
const authorization = req.headers.authorization
|
||||
@@ -232,8 +250,9 @@ async function handler(req: NextApiRequest, res: NextApiResponse) {
|
||||
|
||||
const system = isClickhouse
|
||||
? source`
|
||||
You rewrite SQL queries to ClickHouse SQL for the Supabase logs table.
|
||||
Output only the rewritten SQL query: no explanation, no markdown, and no code fences.
|
||||
You write and edit ClickHouse SQL for the Supabase logs table.
|
||||
Reply with ONLY the SQL that replaces the <selection> block below, keeping the
|
||||
surrounding query valid: no explanation, no comments, and no markdown code fences.
|
||||
${CLICKHOUSE_LOGS_COMPLETION_INSTRUCTIONS}
|
||||
${SECURITY_PROMPT}
|
||||
`
|
||||
@@ -243,23 +262,31 @@ async function handler(req: NextApiRequest, res: NextApiResponse) {
|
||||
${SECURITY_PROMPT}
|
||||
`
|
||||
|
||||
const userMessage = isClickhouse
|
||||
? prompt
|
||||
: source`
|
||||
## Database Schema
|
||||
const schemaSection = isClickhouse
|
||||
? { heading: 'Logs Schema', body: buildClickhouseLogsSchemaSection(availableKeys) }
|
||||
: {
|
||||
heading: 'Database Schema',
|
||||
body: buildDatabaseSchemaSection({ includeSchema, schemaListResult, schemaDDLResult }),
|
||||
}
|
||||
|
||||
${buildDatabaseSchemaSection({ includeSchema, schemaListResult, schemaDDLResult })}
|
||||
const instruction =
|
||||
isClickhouse && intent === 'rewrite' ? CLICKHOUSE_LOGS_REWRITE_INSTRUCTION : prompt
|
||||
|
||||
## Code
|
||||
const userMessage = source`
|
||||
## ${schemaSection.heading}
|
||||
|
||||
\`\`\`${language ?? ''}
|
||||
${textBeforeCursor}<selection>${selection}</selection>${textAfterCursor}
|
||||
\`\`\`
|
||||
${schemaSection.body}
|
||||
|
||||
## Instruction
|
||||
## Code
|
||||
|
||||
${prompt}
|
||||
`
|
||||
\`\`\`${language ?? ''}
|
||||
${textBeforeCursor}<selection>${selection}</selection>${textAfterCursor}
|
||||
\`\`\`
|
||||
|
||||
## Instruction
|
||||
|
||||
${instruction}
|
||||
`
|
||||
|
||||
// Note: these must be of type `CoreMessage` to prevent AI SDK from stripping `providerOptions`
|
||||
// https://github.com/vercel/ai/blob/81ef2511311e8af34d75e37fc8204a82e775e8c3/packages/ai/core/prompt/standardize-prompt.ts#L83-L88
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { useMonaco } from '@monaco-editor/react'
|
||||
import { useLocalStorage } from '@uidotdev/usehooks'
|
||||
import { useDebounce, useLocalStorage } from '@uidotdev/usehooks'
|
||||
import { IS_PLATFORM, LOCAL_STORAGE_KEYS, useFlag, useParams } from 'common'
|
||||
import dayjs from 'dayjs'
|
||||
import type { editor } from 'monaco-editor'
|
||||
@@ -8,11 +8,7 @@ import { useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { toast } from 'sonner'
|
||||
import { Button, ResizableHandle, ResizablePanel, ResizablePanelGroup } from 'ui'
|
||||
|
||||
import {
|
||||
detectLogSource,
|
||||
looksLikeLegacyLogsQuery,
|
||||
rewriteLogsSqlWithAI,
|
||||
} from '@/components/interfaces/Settings/Logs/logs-sql-rewrite'
|
||||
import { LegacyLogsRewriteAdmonition } from '@/components/interfaces/Settings/Logs/LegacyLogsRewriteAdmonition'
|
||||
import {
|
||||
EXPLORER_DATEPICKER_HELPERS,
|
||||
getDefaultHelper,
|
||||
@@ -36,7 +32,6 @@ import {
|
||||
buildLogQueryParams,
|
||||
resolveLogDateRange,
|
||||
} from '@/components/interfaces/Settings/Logs/logsDateRange'
|
||||
import { LogsExplorerOtelBanner } from '@/components/interfaces/Settings/Logs/LogsExplorerOtelBanner'
|
||||
import { LogsQueryPanel } from '@/components/interfaces/Settings/Logs/LogsQueryPanel'
|
||||
import { LogTable } from '@/components/interfaces/Settings/Logs/LogTable'
|
||||
import UpgradePrompt from '@/components/interfaces/Settings/Logs/UpgradePrompt'
|
||||
@@ -52,16 +47,20 @@ import {
|
||||
UpsertContentPayload,
|
||||
useContentUpsertMutation,
|
||||
} from '@/data/content/content-upsert-mutation'
|
||||
import { constructHeaders } from '@/data/fetchers'
|
||||
import { fetchOtelLogKeys } from '@/data/logs/otel-log-keys-query'
|
||||
import {
|
||||
LEGACY_LOGS_DIALECT_CHECK_DEBOUNCE_MS,
|
||||
shouldOfferLegacyLogsRewrite,
|
||||
} from '@/data/logs/logs-sql-rewrite'
|
||||
import { untrustedLogSql } from '@/data/logs/safe-analytics-sql'
|
||||
import {
|
||||
useLegacyLogsRewrite,
|
||||
type LegacyLogsRewriteProposal,
|
||||
} from '@/hooks/analytics/useLegacyLogsRewrite'
|
||||
import { useLogsQuery } from '@/hooks/analytics/useLogsQuery'
|
||||
import { useLogsUrlState } from '@/hooks/analytics/useLogsUrlState'
|
||||
import { useCustomContent } from '@/hooks/custom-content/useCustomContent'
|
||||
import { useCheckEntitlements } from '@/hooks/misc/useCheckEntitlements'
|
||||
import { useIsFeatureEnabled } from '@/hooks/misc/useIsFeatureEnabled'
|
||||
import { useSelectedOrganizationQuery } from '@/hooks/misc/useSelectedOrganization'
|
||||
import { useSelectedProjectQuery } from '@/hooks/misc/useSelectedProject'
|
||||
import { useUpgradePrompt } from '@/hooks/misc/useUpgradePrompt'
|
||||
import { uuidv4 } from '@/lib/helpers'
|
||||
import { useProfile } from '@/lib/profile'
|
||||
@@ -101,9 +100,6 @@ export const LogsExplorerPage: NextPageWithLayout = () => {
|
||||
const useOtelEndpoint = useFlag('otelLegacyLogs')
|
||||
const { logsShowMetadataIpTemplate } = useIsFeatureEnabled(['logs:show_metadata_ip_template'])
|
||||
|
||||
const { data: project } = useSelectedProjectQuery()
|
||||
const { data: organization } = useSelectedOrganizationQuery()
|
||||
|
||||
const allTemplates = useMemo(() => {
|
||||
const templates = getLogsTemplates(useOtelEndpoint)
|
||||
if (logsShowMetadataIpTemplate) return templates
|
||||
@@ -142,11 +138,7 @@ export const LogsExplorerPage: NextPageWithLayout = () => {
|
||||
const [warnings, setWarnings] = useState<LogsWarning[]>([])
|
||||
const [showMissingLimitError, setShowMissingLimitError] = useState<boolean>(false)
|
||||
const [selectedLog, setSelectedLog] = useState<LogData | null>(null)
|
||||
const [rewriteProposal, setRewriteProposal] = useState<{
|
||||
original: string
|
||||
modified: string
|
||||
} | null>(null)
|
||||
const [isRewriting, setIsRewriting] = useState<boolean>(false)
|
||||
const [rewriteProposal, setRewriteProposal] = useState<LegacyLogsRewriteProposal | null>(null)
|
||||
const [rewriteBannerDismissed, setRewriteBannerDismissed] = useLocalStorage<boolean>(
|
||||
`project-${projectRef}-logs-rewrite-banner-dismissed`,
|
||||
false
|
||||
@@ -189,7 +181,32 @@ export const LogsExplorerPage: NextPageWithLayout = () => {
|
||||
const results = logData
|
||||
const isLoading = logsLoading
|
||||
|
||||
const showRewriteCTA = useOtelEndpoint && looksLikeLegacyLogsQuery(editorValue)
|
||||
// Debounced so the dialect heuristics don't run on every keystroke, matching the
|
||||
// SQL editor's rewrite banner.
|
||||
const settledEditorValue = useDebounce(editorValue, LEGACY_LOGS_DIALECT_CHECK_DEBOUNCE_MS)
|
||||
const shouldShowRewriteCTA = useMemo(
|
||||
() =>
|
||||
shouldOfferLegacyLogsRewrite({
|
||||
sql: settledEditorValue,
|
||||
isClickhouseLogsEnabled: useOtelEndpoint,
|
||||
}),
|
||||
[settledEditorValue, useOtelEndpoint]
|
||||
)
|
||||
|
||||
const {
|
||||
state: rewriteState,
|
||||
requestRewrite,
|
||||
dismiss: dismissRewriteBanner,
|
||||
} = useLegacyLogsRewrite({
|
||||
// Read straight from the editor instance — `editorValue` state can lag the
|
||||
// most recent keystroke.
|
||||
readSql: () => editorRef.current?.getValue() ?? editorValue,
|
||||
onProposal: setRewriteProposal,
|
||||
onDismissed: () => setRewriteBannerDismissed(true),
|
||||
})
|
||||
const isRewriting = rewriteState.status === 'rewriting'
|
||||
const hasUnacknowledgedRewriteOutcome =
|
||||
rewriteState.status === 'failed' || rewriteState.status === 'noRewriteNeeded'
|
||||
|
||||
const { mutateAsync: upsertContent, isPending: isUpsertingContent } = useContentUpsertMutation({
|
||||
onError: (e) => {
|
||||
@@ -243,42 +260,6 @@ export const LogsExplorerPage: NextPageWithLayout = () => {
|
||||
addRecentLogSqlSnippet({ unchecked_sql: untrustedLogSql(template.searchString) })
|
||||
}
|
||||
|
||||
const handleRewrite = async () => {
|
||||
const currentSql = editorRef.current?.getValue() ?? editorValue
|
||||
if (!currentSql.trim()) {
|
||||
toast.info('Write a query to rewrite first')
|
||||
return
|
||||
}
|
||||
setIsRewriting(true)
|
||||
try {
|
||||
const headerData = await constructHeaders()
|
||||
const source = detectLogSource(currentSql)
|
||||
const availableKeys = source
|
||||
? await fetchOtelLogKeys({ projectRef: projectRef!, source }).catch(() => undefined)
|
||||
: undefined
|
||||
const rewritten = await rewriteLogsSqlWithAI({
|
||||
sql: currentSql,
|
||||
projectRef: projectRef!,
|
||||
connectionString: project?.connectionString,
|
||||
orgSlug: organization?.slug,
|
||||
authorizationHeader: headerData.get('Authorization'),
|
||||
availableKeys,
|
||||
})
|
||||
// The editor may have changed while awaiting key discovery and the AI call;
|
||||
// don't offer a proposal that would clobber intervening edits.
|
||||
const latestSql = editorRef.current?.getValue() ?? editorValue
|
||||
if (latestSql !== currentSql) {
|
||||
toast.info('The query changed while rewriting. Please try again.')
|
||||
return
|
||||
}
|
||||
setRewriteProposal({ original: currentSql, modified: rewritten })
|
||||
} catch (error) {
|
||||
toast.error(`Couldn't rewrite the query: ${(error as Error).message}`)
|
||||
} finally {
|
||||
setIsRewriting(false)
|
||||
}
|
||||
}
|
||||
|
||||
const acceptRewrite = () => {
|
||||
if (!rewriteProposal) return
|
||||
editorRef.current?.setValue(rewriteProposal.modified)
|
||||
@@ -504,15 +485,16 @@ export const LogsExplorerPage: NextPageWithLayout = () => {
|
||||
templates={allTemplates.filter((template) => template.mode === 'custom')}
|
||||
onSelectTemplate={onSelectTemplate}
|
||||
warnings={warnings}
|
||||
showRewriteAction={showRewriteCTA && rewriteBannerDismissed}
|
||||
showRewriteAction={shouldShowRewriteCTA && rewriteBannerDismissed}
|
||||
isRewriting={isRewriting}
|
||||
onRewrite={handleRewrite}
|
||||
onRewrite={requestRewrite}
|
||||
/>
|
||||
{showRewriteCTA && !rewriteBannerDismissed && (
|
||||
<LogsExplorerOtelBanner
|
||||
isRewriting={isRewriting}
|
||||
onRewrite={handleRewrite}
|
||||
onDismiss={() => setRewriteBannerDismissed(true)}
|
||||
{(hasUnacknowledgedRewriteOutcome ||
|
||||
(shouldShowRewriteCTA && !rewriteBannerDismissed)) && (
|
||||
<LegacyLogsRewriteAdmonition
|
||||
state={rewriteState}
|
||||
onRewrite={requestRewrite}
|
||||
onDismiss={dismissRewriteBanner}
|
||||
/>
|
||||
)}
|
||||
<ShimmerLine active={isLoading} />
|
||||
|
||||
Reference in new issue
Block a user