mirror of
https://github.com/supabase/supabase.git
synced 2026-10-09 11:25:06 +03:00
## Problem Moving the Logs Explorer to ClickHouse means users' saved BigQuery queries no longer run. <img width="2430" height="1010" alt="CleanShot 2026-06-29 at 11 36 04@2x" src="https://github.com/user-attachments/assets/ae0ab155-7d3d-4ae9-81c3-22bf3a88cf8c" /> ## Fix Rewrite the query with AI instead of a SQL transpiler. AI handles the long tail of nested fields and dialect differences far better than a rule-based rewriter, and it needs no extra runtime dependency. - `rewriteLogsSqlWithAI` posts the current query to `/api/ai/code/complete` with `dialect: 'clickhouse'`. The endpoint skips the Postgres schema and best-practices for that dialect and uses logs-specific instructions and model so the output is ClickHouse logs SQL (FROM `logs` + `source` filter, no `unnest` joins, nested fields read from `log_attributes['...']`). - The query's `source` is detected and its real `log_attributes` keys are fetched and passed to the model, so it maps to exact paths instead of guessing. - The rewrite runs in the background and is proposed as a side-by-side accept/discard diff in the editor. The AI Assistant panel is not opened. - Entry points: a banner shown only for legacy-looking queries (dismissal persisted), and a "Fix Query" button next to Field Reference. - The Field Reference drawers discover `log_attributes` keys from real data so the listed fields match what the source actually emits. ## Dependencies Built on top of #47265 (Logs Explorer -> OTEL endpoint) — that is the base branch of this PR. Merge #47265 first. Behind `otelLegacyLogs` (off by default). Part of DEBUG-145 (split from #47087). ## How to test - Open the Logs Explorer with a BigQuery logs query (the templates have some), click "Fix Query", and confirm the diff shows valid ClickHouse SQL. Accept it and confirm the applied query runs. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added an OTEL legacy logs workflow (behind a feature flag) with an interactive banner and a “Fix Query” ClickHouse rewrite action, including an accept/discard diff review overlay. * Introduced OTEL-aware field reference rendering with dynamic discovery of `log_attributes` keys and updated OTEL source insertion behavior. * Enabled dialect-aware SQL completion for ClickHouse logs, using logs-specific instructions and output constraints. * **Bug Fixes** * Improved rewrite flow validation and handling, including log source detection and cleanup of AI-generated SQL formatting. * **Tests** * Added Vitest coverage for rewrite prompt generation, detection/classification utilities, SQL fence stripping, OTEL field mapping, and OTEL log attribute key discovery. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
46 lines
1.5 KiB
TypeScript
46 lines
1.5 KiB
TypeScript
import { useQuery } from '@tanstack/react-query'
|
|
|
|
import { executeAnalyticsSql } from './execute-analytics-sql'
|
|
import { logsAllEndpointUrl } from './logs-endpoint'
|
|
import { analyticsLiteral, safeSql } from './safe-analytics-sql'
|
|
|
|
const LOOKBACK_HOURS = 24 * 7
|
|
|
|
export async function fetchOtelLogKeys({
|
|
projectRef,
|
|
source,
|
|
signal,
|
|
}: {
|
|
projectRef: string
|
|
source: string
|
|
signal?: AbortSignal
|
|
}): Promise<string[]> {
|
|
const end = new Date()
|
|
const start = new Date(end.getTime() - LOOKBACK_HOURS * 60 * 60 * 1000)
|
|
const sql = safeSql`SELECT arrayJoin(mapKeys(log_attributes)) AS key, count() AS n FROM logs WHERE source = ${analyticsLiteral(source)} GROUP BY key ORDER BY n DESC LIMIT 500`
|
|
const data = await executeAnalyticsSql({
|
|
projectRef,
|
|
endpoint: logsAllEndpointUrl(true),
|
|
sql,
|
|
iso_timestamp_start: start.toISOString(),
|
|
iso_timestamp_end: end.toISOString(),
|
|
method: 'post',
|
|
signal,
|
|
})
|
|
const rows = (data?.result ?? []) as { key: string }[]
|
|
return rows.map((r) => r.key).filter(Boolean)
|
|
}
|
|
|
|
export function useOtelLogKeysQuery(
|
|
{ projectRef, source }: { projectRef?: string; source?: string },
|
|
{ enabled = true }: { enabled?: boolean } = {}
|
|
) {
|
|
return useQuery({
|
|
queryKey: ['projects', projectRef, 'otel-log-keys', source],
|
|
queryFn: ({ signal }) =>
|
|
fetchOtelLogKeys({ projectRef: projectRef ?? '', source: source ?? '', signal }),
|
|
enabled: enabled && Boolean(projectRef) && Boolean(source),
|
|
staleTime: 5 * 60 * 1000,
|
|
})
|
|
}
|