21 Commits
Author SHA1 Message Date
Jordi Enric 6434c48999 feat(studio): migrate Auth reports to OTEL (#50469)
## Problem

Auth observability charts always queried the legacy logs.all endpoint,
even when the OTEL reports rollout was enabled. The existing OTEL SQL
also had ClickHouse correctness and parity gaps around timestamp
aliasing, JSON types, provider paths, missing values, and error-code
attributes.

## Fix

Route the ten Auth-specific charts through the OTEL query builders and
logs.all.otel endpoint when otelReports is enabled. Preserve the
BigQuery fallback, partition React Query caches by backend, and leave
the shared API gateway charts on the legacy endpoint.

Correct the OTEL queries by qualifying source timestamps, using typed
and nullable JSON extraction, preserving missing actor and duration
semantics, selecting the right provider path for each event shape,
preferring the canonical Auth error-code attribute with a legacy
fallback, and applying bounded result limits. Two-minute report
intervals now use minute-level SQL buckets instead of falling through to
hourly buckets.

## How to test

- Run `CI=1 pnpm --filter studio exec vitest run
data/reports/v2/auth.config.otel.test.ts
hooks/misc/__tests__/useReportDateRange.test.ts`
- Run `pnpm --filter studio run lint:ratchet`
- Run `pnpm --filter studio run typecheck`
- Expected result: all checks pass and generated OTEL SQL preserves
legacy report semantics.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Auth observability charts can now use OpenTelemetry data when enabled,
while retaining the existing reporting source otherwise.
- Switching the data source automatically refreshes the relevant charts.

- **Bug Fixes**
- Improved Auth observability accuracy for provider, duration, actor,
and error-code reporting.
- Added safeguards to keep report queries within the supported result
limit.
- Corrected minute-level grouping for two-minute analytics intervals and
three-hour date ranges.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-17 13:44:39 +02:00
Jordi Enric a28214c236 feat(studio): add ClickHouse OTEL SQL for the Auth report's v2 metrics (#48750)
## Problem

Part of DEBUG-73 (migrate reports queries to the OTEL/ClickHouse
endpoint).

The Auth report's v2 metrics (ActiveUsers, SignInAttempts,
PasswordResetRequests, TotalSignUps, sign-in/sign-up processing time,
error breakdowns) currently only query the BigQuery-backed logs.all
endpoint.

## Fix

Adds `AUTH_REPORT_SQL_OTEL`, a ClickHouse-dialect mirror of the existing
`AUTH_REPORT_SQL`, covering all 10 metrics. Threads a `useOtel`
parameter through `fetchLogs` and the three report config creators
(`createUsageReportConfig`, `createErrorsReportConfig`,
`createLatencyReportConfig`), defaulting to `false` everywhere.

This PR is inert on its own: nothing yet passes `useOtel: true`, so it
changes no runtime behavior. The follow-up PR (stacked on this one)
wires the `otelReports` feature flag through the Auth report page to
actually select the OTEL SQL.

Also includes: one dataProvider now validates its raw rows with a Zod
schema instead of casting to `any`, and removal of a few functions in
this file that had zero callers (`AUTH_ERROR_CODE_VALUES`,
`createAuthReportConfig`, an exact duplicate of a status-code color map,
an unused hook).

## How to test

- Unit: `cd apps/studio && npx vitest run
data/reports/v2/auth.config.otel.test.ts`
- No manual testing needed for this PR alone since it changes no runtime
behavior (useOtel defaults to false, unwired). The follow-up PR covers
manual testing of the actual flag-gated behavior.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added OpenTelemetry-backed authentication reports.
* Authentication reports now include usage, errors, latency, sign-ins,
sign-ups, and edge-log metrics.
  * Added filtering by provider, status code, action, and time range.
  * Added selectable telemetry sources for retrieving report logs.
* **Bug Fixes**
  * Improved validation of authentication error codes.
* Improved handling of missing report data with consistent empty
results.
  * Improved report formatting for more consistent attribute display.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-05 18:25:41 +02:00
Jordi Enric aed332b663 feat: migrate Edge Functions report to ClickHouse OTEL logs (#48756)
## Problem

The Edge Functions report (`observability/edge-functions`) only queries
BigQuery. As part of the broader Reports→ClickHouse OTEL migration
(DEBUG-73), we need each report migrated one at a time behind the
`otelReports` flag.

## Fix

Adds a ClickHouse OTEL SQL variant (`METRIC_SQL_OTEL`) for the 4 Edge
Functions metrics (TotalInvocations, ExecutionStatusCodes,
InvocationsByRegion, ExecutionTime), querying the unified `logs` table
filtered to `source = 'function_edge_logs'`, with fields read from
`log_attributes` (`function_id`, `response.status_code`,
`response.headers.x_sb_edge_region`, `execution_time_ms`) — the same
mapping already used by `edge-functions-last-hour-stats-query.ts`.

`edgeFunctionReports()` now takes a `useOtel` flag that picks between
the BQ and OTEL query sets and forwards it to `fetchLogs`. The page
wires this up via `useFlag('otelReports')`, matching the pattern used
for the Auth report. No behavior change while the flag is off — report
still fetches from BigQuery.

Also removed two pieces of dead code spotted in `report.utils.ts` while
touching it: the unused `useEdgeFnIdToName` hook and a
`STATUS_CODE_COLORS` map that was an exact duplicate of
`REPORT_STATUS_CODE_COLORS` (the one actually imported elsewhere).

This PR is standalone — no dependency on the in-flight Auth report OTEL
stack.

## How to test

- `pnpm vitest run data/reports/v2/edge-functions.config.otel.test.ts` —
9 new tests covering the OTEL SQL shape (single logs table,
unix-microsecond timestamp bucketing, field mapping, filters).
- `pnpm vitest run data/reports` and `pnpm vitest run
data/edge-functions components/interfaces/Reports` — existing suites (46
+ 54 tests) still pass, confirming no regression to the BQ path.
- Manually: with `otelReports` flag enabled, visit a project's Edge
Functions observability report and confirm charts render from the
ClickHouse endpoint.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added OpenTelemetry support for Edge Functions observability metrics,
including invocations, status codes, regional activity, and execution
time.
* Reports can now dynamically use either the standard or OpenTelemetry
logs source.

* **Bug Fixes**
* Improved filtering and timestamp handling for OpenTelemetry-based Edge
Functions metrics.
* Added coverage for status, execution time, function, and region
filters.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-05 17:13:34 +02:00
kemal.earth bc95a2f19a fix(studio): edge func exec time formatting in reports (#48539)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Fixes Edge Function Execution Time chart within our observability
reports time formatting. This also fixes the non-hovered state which
would lose the `ms` formatting.

| Before | After |
|--------|--------|
| <img width="2160" height="652" alt="cleanshot_2026-07-29_at_02 15
53_2x"
src="https://github.com/user-attachments/assets/cfd6dbc2-f283-4379-a133-581c76990cb5"
/> | <img width="797" height="314" alt="Screenshot 2026-07-31 at 14 30
48"
src="https://github.com/user-attachments/assets/f5f1ace5-b6ef-43db-aebd-e10d31631013"
/> |



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Improved execution-time chart formatting with clearer millisecond
values, thousands separators, and configurable precision.
* Chart highlights now support custom value formatting alongside
existing number, percentage, and byte formats.

* **Bug Fixes**
  * Non-finite execution-time values now display safely as `0ms`.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-31 16:20:37 +01:00
Charis fd1f437eca feat(logs): brand remaining analytics SQL callers with SafeLogSqlFragment (#46476)
## Summary

PR 10 of the analytics SQL safety series. Migrates the last surface of
analytics queries that flowed through plain
`get(.../analytics/endpoints/logs.all, { query: { sql } })` or the
`fetchLogs(projectRef, sql: string, ...)` helper over to
`executeAnalyticsSql` with branded `SafeLogSqlFragment` inputs.

After this PR, every analytics SQL call site builds its query through
the safe-analytics-sql helpers and hits the wire through the single
`executeAnalyticsSql` boundary. User-controlled values (filter
operators, numeric thresholds, function IDs, regions, provider names)
all flow through `analyticsLiteral` / branded operator maps; static
fragments are wrapped in `safeSql`. PR 11 (ESLint / vitest rule
forbidding direct analytics-endpoint POST/GET outside
`executeAnalyticsSql`) is the next and final step.

## Changes

- **`hooks/analytics/useProjectUsageStats.tsx`** — route the
already-branded `genChartQuery` output through `executeAnalyticsSql`
(parallels `useLogsPreview`).
- **`data/reports/report.utils.ts`** — tighten `fetchLogs(sql)` from
`string` to `SafeLogSqlFragment`; the wire boundary is now the same
single `executeAnalyticsSql` wrapper used by the rest of the analytics
path. Adds two pre-branded fragment maps reused by the report configs:
- `SAFE_GRANULARITY_SQL` — closed set returned by
`analyticsIntervalToGranularity`.
- `SAFE_COMPARISON_OPERATOR_SQL` — closed set on
`NumericFilter.operator`.
- **`components/interfaces/Auth/Overview/OverviewErrors.constants.ts`**
— wrap the two static `AUTH_TOP_*_SQL` fragments in `safeSql` (no
interpolation, but the type now flows).
- **`data/reports/v2/edge-functions.config.ts`** — `filterToWhereClause`
and every entry in `METRIC_SQL` now return `SafeLogSqlFragment`.
User-controlled values (`status_code.value`, `execution_time.value`,
function IDs, regions) pass through `analyticsLiteral`; operators look
up the branded map; the granularity uses the branded map. The
wire-format strings are unchanged, so the existing
`edge-functions.test.tsx` exact-string expectations still hold.
- **`data/reports/v2/auth.config.ts`** — same shape applied to all ten
`AUTH_REPORT_SQL` entries. The legacy `whereClause.replace(/^WHERE\s+/,
'')` pattern is replaced by two helpers that emit `AND`-prefixed
predicate fragments directly (`authFiltersToAndPredicates`,
`edgeLogsFiltersToAndPredicates`). Static provider SELECT / GROUP BY
fragments are pre-branded.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Enhanced security for analytics and reporting queries by updating
query construction methods across auth, edge functions, and project
usage reports.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/supabase/supabase/pull/46476?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-29 09:26:06 -04:00
Joshen Lim 7f5865872a Enforce noUnusedLocals and noUnusedParameters in tsconfig.json + fix all related issues (#45264)
## Context

Enforce `noUnusedLocals` and `noUnusedParameters` in tsconfig.json + fix
all related issues
2026-04-27 17:42:34 +08:00
Charis 3b7052b5a9 cleanup: fix import order and prefixes for studio/data (#44501) 2026-04-03 09:15:57 +02:00
Ignacio Dobronich 60a2a5e092 chore: observability dashboard entitlements (#43502)
## Changes
- Replaces plan ID-based chart gating with entitlement-based checks
across all observability pages (database, auth, realtime,
edge-functions)
- Adds requiredPlan display field so the upsell shows the correct target
plan (e.g. "Upgrade to Team" for disk throughput, not just "Upgrade")
- Decouples database-charts.ts from Organization Plan, now accepts
pre-computed entitlement flags instead

### Note on `requiredPlan`
The entitlement system currently tells us whether a user has access, but
not what plan they need to get access. `requiredPlan` is a frontend-only
display hint. Ideally the backend entitlement response would include a
minimumPlan field so we can remove this hardcoded mapping. For now, this
is a pragmatic workaround to avoid misleading upsell copy (e.g. a Free
user seeing "Upgrade" for a Team-only feature and landing on Pro).


## Testing 

### Database
- Head to `/project/_/observability/database` with an Org on the Free
Plan and assert that the Disk throughput is gated.

<img width="1402" height="682" alt="image"
src="https://github.com/user-attachments/assets/b783d866-774d-44dc-88d1-797d26502f02"
/>

### Auth
- Head to `/project/_/observability/auth` with an Org on the Free Plan
and assert that the following metrics are gated:

<img width="1402" height="682" alt="image"
src="https://github.com/user-attachments/assets/cb062dd7-5864-4e40-a11f-ad6be1f7fd41"
/>

### Edge Functions
- Head to `/project/_/observability/edge-functions` with an Org on the
Free Plan and assert that the following metrics are gated:

<img width="1402" height="320" alt="image"
src="https://github.com/user-attachments/assets/517b5483-4575-493c-9401-2c67ac9e684c"
/>


### Realtime
- Head to `/project/_/observability/realtime` with an Org on the Free
Plan and assert that the following metrics are gated:

<img width="1402" height="722" alt="image"
src="https://github.com/user-attachments/assets/0c2757a2-8482-43f4-a65c-c0dde3d878c7"
/>
2026-03-11 16:57:53 -03:00
Raúl Barroso 09df6e8373 refactor: infra queries to use attributes (#41018)
* refactor: infra queries to use `attributes`

This PR refactors the infrastructure monitoring query code reducing duplication and unifying the API request to always be `attributes`:

•  Removed the separate useInfraMonitoringQuery hook and getInfraMonitoring function that handled a single monitoring query
•  Consolidated all infrastructure monitoring queries into a unified useInfraMonitoringAttributesQuery hook that handles multi-attribute requests
•  Moved interval selection logic from the query layer to the consumer (InfrastructureActivity.tsx), where it can be computed dynamically based on user-selected date ranges
•  Simplified query types by removing intermediate InfraMonitoringData and InfraMonitoringVariables types
•  Interval is now computed in the component (defaults to 1d, switches to 1h for date ranges ≤48 hours) rather than hardcoded in the query layer
•  All queries now use the unified multi-attribute endpoint with explicit parameter passing

* fix: handle single-attribute response format
2025-12-04 16:16:12 +01:00
Kevin Grüneberg 915a08812d feat: support new platform plan (#40890) (#41046)
Forward compatible changes to support new platform plan (similar handling to Enterprise)
2025-12-04 17:31:27 +08:00
Joshen Lim 888b1794c6 Revert "feat: support new platform plan" (#40980)
Revert "feat: support new platform plan (#40890)"

This reverts commit ae4fe1b740.
2025-12-03 10:41:53 +08:00
Kevin Grüneberg ae4fe1b740 feat: support new platform plan (#40890)
Forward compatible changes to support new platform plan (similar handling to Enterprise)
2025-12-02 15:35:39 +08:00
Eduardo Gurgel 206512d5a4 fix(studio): realtime metrics (#40824)
* Connected clients is now showing the number of connections at the time. 
* Broadcast events 
* Presence events 
* Postgres changes events
* Rate of Channel joins (unchanged)
* Message payload size : median of payload size 
* Broadcast From Database Replication Lag:  median replication from commit to broadcast
* (Read) Private Channel Subscription RLS Execution Time: median time RLS execution to subscribe
* (Write) Private Channel Subscription RLS Execution Time: median time RLS execution to publish
2025-11-28 10:44:32 +13:00
038249088a feat: update realtime report (#40252)
* feat: update realtime report

Co-authored-by: Jordi Enric <jordi.err@gmail.com>

* fix: misc bug fixes

* fixup! fix: misc bug fixes

---------

Co-authored-by: Jordi Enric <jordi.err@gmail.com>
Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Co-authored-by: Ali Waseem <waseema393@gmail.com>
2025-11-12 10:33:27 -05:00
Jordi Enric 37500328a2 Rename Active Users to Auth Activity (#39805)
rename active users to auth activity
2025-10-23 15:22:06 +02:00
Jordi Enric b5a638bfc6 auth report improvements (#39013) 2025-10-09 09:58:32 +02:00
Jordi Enric 82559c5b8c fix auth report where (#38951)
fix
2025-09-23 15:56:44 +00:00
Jordi EnricandCharis c75bb1b60d Auth Report tests and improvements (#38550)
* move to new ReportConfig

* separate types

* add auth report config

* add auth report transofmration test

* add fetchlogs util for reports

* rm auth-charts

* rm auth-report-query

* rm dupped code

* simplify upsell compo

* rm unnecessary hook

* use upsell in database report instead of old component

* rip ReportChart you wont be missed

* more tests

* rm unnecessary import

* Fix SQL in active users

* support hide inside report chart component

* add max bar size to avoid having big ass bars when theres little data

* fix incorrect timestamp key passed on to fillTimeseries

* add tooltip to auth report

* fix bug with timestamp key inside tooltips

* add report settings

* rm debugging check

* Update apps/studio/data/reports/v2/auth.config.ts

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

* safely access result

* fix sign in latency query

* add toggleable attributes

* handle toggleable attributes

* pass on hideHighlightedValue

* fix signup latency

* pass hideHighlighted

* report chart pass on hide highlighted value

* cleanup finalChartData

* simplify tests, use timestamp for everything

* fix test

* rm dupped attribute

* add tooltip to auth errors

* Update apps/studio/data/reports/v2/auth.config.ts

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

* Update apps/studio/components/ui/Charts/ComposedChart.tsx

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

* prettify

---------

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>
2025-09-17 17:41:10 +02:00
Jordi Enric dee5ff1194 edge function report: cleanup and url filters (#38446)
* rm dupped value

* use zod in reportsnumericfilter

* use zod in reportsselectfilter

* use nuqs in filters

* validate data before change

* unnecessary changes

* fix some type errs update tests

* use command to add search to select filter

* update styles / clear btn in num filter

* add search to edge fn report

* fix clickable area

* fix test

* default true

* fix availablein

* sum paddin

* fix default values in tests
2025-09-05 11:49:08 +02:00
Jordi EnricandJoshen Lim 3dd065efce Add Edge Function Report Filters (#38369)
* add regions

* update chart

* add numeric filter

* add select filter

* update types

* update edge fn ui

* update report config to use filters

* remove unused event

* export wherefn

* add where fn tests

* rm unnecessary memo

* make type optional

* rm any

* fix keys, add missing flag, add flags to region filter

* bye div

* bye div2

* pass refresh state

* type err

* Minor style fix

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-09-04 14:03:40 +07:00
f65884f341 Reports consolidation 1: Edge Functions (#38129)
* wip

* add formatTimestamp for reports

* add tests for formatTimestamp

* fix type err

* rm old chart logic

* do not retry replication sources

* update report config schema

* use report config

* fix charts, simplify config

* sync charts

* rm comment

* usefilltimeseries

* move chart upsell to its own component

* cleanup v2 chart component

* fix chartheader function format bug

* make attributes dynamic

* move types to its own file

* update reportv2chart types

* update reportv2 chart type

* use new edgefn report

* rm mock

* rm old code

* fix type err

* fix type err

* rename hook

* return sql in dataprovider

* add button to open in log explorer in chart header

* fix type

* fix chart type toggle

* fix type error

* export data transformation for testing

* add tests for data transformation

* rm dupped code

* add report settings to edge fn report

* rm mocked data

* rename

* fix type err

* Apply suggestion from @joshenlim

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>

* rm unnecessary div

* rm unnecessary props from report upsell

* cleaner query keys

* move loadin/error/empty states to inside panel content to prevent layout shift

* use card instead of panel

* use buttontooltip in chartheader

* use buttontooltips in buttons with tooltips

* rm export default

* rm export defualt

* rm i from key

* fix status code transformation

* fix filltimeseries

* add tests for filltimeseries

* fix data transform

* pass xaxis key to chart

* refactor hover state to use timestamp not index

* fix tests

* fix areachart

* fix barchart

* fix chartheader

* fix composedchart

* fix stacked

* fix type missing xaxiskey

* type fix

* Revert "refactor hover state to use timestamp not index"

This reverts commit cfd7886460.

* undo charthover changes, fix data aggregation

* Revert "fix tests"

This reverts commit e5200a2035.

* move transforms to testable functions, add tests

* nit refactors and clean up

* feat: add aws nimbus cloud provider (#38312)

* feat: add aws nimbus cloud provider

* Fix return type of useCustomContent hook

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>

* add utc plugin to dayjs

* move plugins to vitest setup

* rm customparser plugin

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: Alaister Young <alaister@users.noreply.github.com>
2025-09-01 11:10:30 +02:00