Commit Graph
5 Commits
Author SHA1 Message Date
Charis 8b38e0d1ed feat(studio): ClickHouse dialect for logs snippet AI + rewrite to ClickHouse (#48501)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Feature, plus a refactor of the shared logs-rewrite flow.

PR 8 of the SQL editor query-source series. Stacked on #48457 — review
that one first, and merge this after it.

## What is the current behavior?

A `log_sql` snippet runs against the ClickHouse-backed analytics
endpoint, but the SQL editor's AI still writes Postgres: inline edits
get Postgres system prompts, and the result is run through
`sql-formatter`, which mangles ClickHouse backticks and `log_attributes`
map lookups.

Legacy Logs Explorer saved queries open in the editor as `log_sql`
snippets. Those are BigQuery dialect and error against the ClickHouse
endpoint the editor runs them on, with no in-editor way out — only the
Logs Explorer offered a rewrite.

The completion route was also asymmetric. It assembled a
schema/code/instruction message for Postgres but forwarded `prompt`
verbatim for ClickHouse, so a client wanting ClickHouse had to
hand-build the equivalent string.

## What is the new behavior?

**Inline AI speaks ClickHouse for logs snippets.** `sqlSourceToDialect`
maps a snippet's source to `postgres`/`clickhouse` and
`buildCompletionRequestBody` threads it through. For ClickHouse,
`useSqlEditorAi` strips code fences from the response and skips
`formatSql`. Execution and dialect both follow the snippet type, so a
snippet's valid dialect never flips.

**Rewrite to ClickHouse in the editor.** A banner offers the rewrite for
a logs snippet whose text trips `looksLikeLegacyLogsQuery`, and proposes
the result through the editor's existing AI diff view rather than
replacing the snippet, so it's accepted or discarded like any other AI
edit. Gated on `otelLegacyLogs`: on a non-migrated org the BigQuery text
is still correct, so rewriting it would break a working query.

The offer is a state machine (`offered` / `rewriting` / `failed` /
`noRewriteNeeded` / `dismissed`) with a declarative table of valid
transitions, so the states are mutually exclusive by construction and
dismissal is terminal. A failure keeps its message and offers a retry; a
response identical to the input is reported rather than opening an empty
diff.

**One place assembles completion prompts.** The route now uses a single
template for both dialects, branching only the schema section and — for
`intent: 'rewrite'` — the instruction. `lib/ai/clickhouse-logs.ts` is
the single home for ClickHouse-logs prompt content, replacing two
independently maintained descriptions of the same table. Clients carry
no prompt text.

**The rewrite flow is shared with the Logs Explorer.** Both surfaces
previously hand-rolled the same sequence and had drifted: only one
detected a no-op rewrite, they sourced `log_attributes` keys
differently, and the Explorer formatted errors with an `as Error` cast.
Both now use `useLegacyLogsRewrite` and the same state-driven banner, so
the Explorer picks up no-op detection and typed error extraction.

**Attribute keys are fetched on submit, not while typing.** The detected
source would otherwise feed a reactive query key, making every edit that
changed it cost another network call. `useLogsAttributeKeys` is
imperative and goes through `queryClient.fetchQuery`, so a source
already cached — including by the Explorer header and query panel, which
subscribe reactively — is reused. This also closes a gap where inline
edits never received keys at all, unlike full rewrites.

`getErrorMessage` gains an optional typed fallback and no longer
stringifies a bare object into `'[object Object]'`; every existing
caller already hand-rolled a fallback, except `QueueSettings`, which
interpolated the raw result and now passes one.

Nothing here is user-visible until the `sqlEditorLogsSource` flag is
enabled.

Tests: dialect selection and request-body shape, the ClickHouse prompt
content (including that the schema section does not restate the dialect
rules), the reducer's valid and invalid transitions,
`shouldOfferLegacyLogsRewrite`, on-submit key discovery with cache
reuse, and `getErrorMessage`.

## Additional context

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added an Assistant banner to help rewrite legacy BigQuery-style logs
queries into ClickHouse SQL.
* SQL assistance now adapts to the selected query type, including
relevant log attribute context.
* Rewrite suggestions can be reviewed as editor diffs before being
applied.

* **Bug Fixes**
* Improved rewrite failure handling, retry options, dismissal behavior,
and “no rewrite needed” messaging.
* Error notifications now provide a clearer fallback message when
details are unavailable.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-04 09:02:40 -04:00
Charis 3b7052b5a9 cleanup: fix import order and prefixes for studio/data (#44501) 2026-04-03 09:15:57 +02:00
Joshen Lim 22f937c954 Chore/async filters for unified logs (#37200)
* Refactor retrieval of log counts

* Async filters

* Clean up

* Clean up

* Fix
2025-07-17 17:33:17 +08:00
Jonathan Summers-MuirandAlaister Young 1e3b5063bb Feat/unified logs inspection panel (#36895)
* Update QueryOptions.ts

* Create UnifiedLogs_InspectionPanel_Architecture.md

* init new queries

* Create UnifiedLogs_ServiceFlow_Implementation_Plan.md

* Add service flow blocks to UnifiedLogs panel

Introduces ServiceFlowBlocks components to visually represent the service flow in UnifiedLogs, including Origin, Network, PostgREST, Postgres, and Response steps. Adds utility formatters and updates ServiceFlowPanel to render the new blocks with enriched data and loading/error states.

* Enable filterable fields in ServiceFlow blocks

Refactored ServiceFlow blocks to support filterable fields by passing filterFields and table props, updating BlockField to render filterable values as clickable actions, and aligning field configs with filterable IDs. Updated ServiceFlowPanel to use new block names and pass required props. This enhances interactivity and consistency in the UnifiedLogs service flow UI.

* Update ServiceFlowBlocks.tsx

* Enhance Service Flow with enriched fields and filtering

Expanded the Service Flow SQL query to extract 35+ flattened, meaningful fields including request, response, client location, network, headers, and JWT data. Updated the implementation plan to reflect new block architecture, filtering integration, and future enhancements. Added debug logging for enriched and raw log data in ServiceFlowPanel.

* Start adding inspection panel

* Moar logs

* Add Postgres service flow support to Unified Logs

Introduces Postgres as a first-class service flow type in Unified Logs, including SQL query, type definitions, UI blocks, and filtering. Updates ServiceFlow queries, types, and UI components to display enriched Postgres log details and event messages. Also refines filter fields, log type ordering, and minor UI adjustments for clarity and consistency.

* Lots of tidying up, moving blocks into their own files.

* Remove UnifiedLogs architecture and service flow docs

Deleted documentation files related to UnifiedLogs inspection panel architecture, service flow field mapping, implementation plan, and integration example. This cleans up outdated or redundant design and implementation reference materials.

* Refactor service flow blocks to use unified config system

Replaces individual service block components with a generic, config-driven system. Introduces a shared Block component and centralizes block configuration in blockConfigs.ts, reducing duplication and improving maintainability. Updates imports and removes now-unnecessary block files. Also refactors ServiceFlowPanel to use the new block structure and simplifies layer icon logic.

* Refactor ServiceFlow blocks and utilities structure

Consolidates edge service flow SQL queries to reduce duplication, removes ServiceFlowBlocks.tsx in favor of more modular block/component structure, and moves storage-specific utilities to a new utils/storageUtils.ts file. Updates imports in config files and ServiceFlowPanel to reflect the new structure. Type definitions are clarified and extended in types.ts.

* Update ServiceFlow.sql.ts

* Refactor ServiceFlow types for stronger type safety

Replaces generic 'any' types with specific types such as ColumnSchema, QuerySearchParamsType, and SearchParamsType in ServiceFlow components and ServiceFlowPanel. This improves type safety and code clarity, and removes unused metadata and workaround variables from ServiceFlowPanel.

* Remove debug flags and console logs from ServiceFlow components

Eliminated unused DEBUG_SERVICE_FLOW flags and related console logging from ServiceFlow.sql.ts and ServiceFlowPanel.tsx to clean up the codebase and reduce unnecessary output.

* Remove unused log query and filtering functions

Deleted unused functions related to log level filtering, base condition building, and the edge/supavisor logs queries. This cleanup simplifies the UnifiedLogs.queries.ts file by removing legacy or currently unused code.

* revert

* Remove ServiceFlow panel and types, update DataTableToolbar

Deleted the ServiceFlowPanel component and its associated types from the UnifiedLogs interface. Updated DataTableToolbar to toggle between PanelLeftClose and PanelLeftOpen icons based on the open state.

* Remove unused totalRows prop from DataTable

Eliminates the totalRows prop from UnifiedLogs and its context interface, as it is no longer used. This helps clean up the component API and related type definitions.

* Refactor log field configs and unify event message component

Replaces specialized PostgresEventMessage with a generic EventMessage component and updates references accordingly. Consolidates all log field configuration files into serviceFlowFields.ts, removing separate config files for each service. Cleans up unused types and utility imports, and deletes redundant formatters.

* Show empty state for Postgres block with no data

Added logic to display an empty state in the Postgres block when no Postgres data is available in non-Postgres logs. Also updated ServiceFlowPanel to render both PostgREST and Postgres blocks, ensuring the empty state is shown when appropriate.

* Remove Log Details tab from ServiceFlowPanel

The Log Details tab and its associated content have been removed from the ServiceFlowPanel component. This simplifies the UI by focusing on the Overview and Raw JSON tabs.

* Update UnifiedLogs.queries.ts

* Update UnifiedLogs.queries.ts

* Update apps/studio/components/interfaces/UnifiedLogs/ServiceFlow/components/shared/BlockField.tsx

* Refactor UnifiedLogs layout and improve service type formatting

Refactored UnifiedLogs.tsx to use a more flexible ResizablePanelGroup layout, replaced inline function logs panel with a new LogsListPanel component, and removed unused imports and code. Updated ServiceFlow.sql.ts to simplify JWT payload extraction and added a dedicated query for edge function service flow. Added a utility to format service type strings for display in UnifiedLogs.utils.ts.

* Refactor logs UI and improve service flow handling

Renamed FunctionLogsTab to LogsList and added a new LogsListPanel for displaying function logs in a side panel. Improved ServiceFlowPanel with better panel sizing, conditional query enabling, and UI adjustments. Enhanced EventMessage with badge styling and service type formatting. Updated CollapsibleSection for cleaner layout. Enabled debug mode in unified-log-inspection-query. Updated FilterSideBar to use ResizablePanel for improved layout flexibility.

* Refactor UnifiedLogs UI and ServiceFlow query fields

Removes unused skeletonClassName from BlockFieldConfig and related UI, simplifies ServiceFlow SQL query by returning null for unused JWT fields, and updates UnifiedLogs and LogsListPanel for improved layout and collapsible logs panel. Also adjusts table and log list row heights for better visual consistency.

* Update ServiceFlowHeader.tsx

* Update ServiceFlowHeader.tsx

* Refactor ServiceFlowPanel and header tooltip logic

Replaces custom tooltip implementation in ServiceFlowHeader with ButtonTooltip for navigation buttons. Simplifies serviceFlowType logic in ServiceFlowPanel by removing the getServiceFlowType helper and using direct mapping, reducing code complexity.

* Update UnifiedLogs.constants.tsx

* Remove commented code and add doc comment in UnifiedLogs

Removed an unused commented-out span in UnifiedLogs.fields.tsx and added a documentation comment for the Supabase storage logs query fragment in UnifiedLogs.queries.ts.

* Update UnifiedLogs.tsx

* Remove 'live' level from DataTable components

Eliminated the 'live' level from LEVELS constant and related color utility logic. Also removed commented code and references to 'live' in DataTableFilterControls and LiveRow, streamlining the DataTable component's status handling.

* Update UnifiedLogs.constants.tsx

* cleanup

---------

Co-authored-by: Alaister Young <a@alaisteryoung.com>
2025-07-08 20:00:23 +08:00
Joshen Lim 2808931f92 Chore/refactor unified logs infinite queries (#36598)
* Split unified logs count into its own infinite query and refactor main UI to use that new query

* Split unified logs into its own infinite query and refactor main UI to use that new query

* Shift useChartData into unified-logs-chart-query, and refactor main UI to use that new query

* Rename unified-logs-query to unified-logs-infinite-query

* Remove uuid and live params from query string in react queries for unified logs
2025-06-24 16:28:00 +08:00