mirror of
https://github.com/supabase/supabase.git
synced 2026-10-05 09:25:06 +03:00
Andrew ValleteauandClaude Fable 5
6c6a721cb7
## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Bug fix (performance), follow-up to #47894, plus regression-guard tests. ## What is the current behavior? #47894 scoped the Table Editor and entity-definition introspection queries, but four more `@supabase/pg-meta` query families still do O(catalog) work per request. On a production project with a very large catalog (hundreds of schemas, ~465K `pg_constraint` rows) they run 5 to 55 seconds each, trip the 58s `statement_timeout`, and spill sorts to temp files. During a recent "DB CPU > 85%" incident on such a project, 24 of 27 active backends were running these queries concurrently. 1. **`tables.retrieve()` (single-table lookup by name+schema or id)**: the `tables`/`columns` CTEs scan the whole catalog (`pg_class`, `pg_constraint`, `pg_index`, all of `pg_attribute`, per-table sizes) and the one-table predicate is applied only on the outer select. Same bug class #47894 fixed for the OID-based table editor query; this sibling path never got the treatment. It accounted for 94 of the 96 statement-timeout cancellations in the incident. 2. **Types listing**: the `t_enums` and `t_attributes` subqueries aggregate the entire `pg_enum` and every composite relation before the wrapper's schema filter applies. 3. **Table privileges**: `aclexplode` + double `pg_roles` join + GROUP BY over every relation in the database; schema/OID filters applied only after aggregation, in both `list()` and `retrieve()`. 4. **Row counts**: `getTableRowsCountSql` treats `reltuples = -1` (never-analyzed table) as "small table, run exact count(*)". A freshly bulk-loaded multi-million-row table times out on every Table Editor pagination render. Two Studio-side amplifiers turned one slow query into a sustained load storm: - `useTableQuery` (behind `tables.retrieve()`) mounts once per visible foreign-key grid cell via `ForeignKeyFormatter`, so a single Table Editor view fires ~20 concurrent copies against the FK target table. A timed-out query caches nothing, and TanStack retries errored no-data queries on every observer mount by default, so scrolling kept re-issuing the 58s scan. - `useTableApiAccessQuery` fetched table privileges for the entire database and filtered down to one schema client-side. ## What is the new behavior? **pg-meta (all behind the existing `pgMetaScopedIntrospection` flag, same rollout mechanism as #47894; `scoped: false` keeps serving the current SQL):** - `tables.retrieve()`: the identifier is resolved to a scalar `targetOid` init-plan and pushed into the base scan, primary-key, relationships (both FK directions kept: `conrelid` or `confrelid`) and columns CTEs. A materialized `target` CTE was deliberately avoided: it acts as an optimization barrier and forces the very seq scans being removed. - Types: filter `pg_type`/`pg_namespace` first, then compute enums/attributes per surviving row via correlated index-scan subqueries (`pg_enum(enumtypid, enumsortorder)`, `pg_attribute(attrelid, attnum)`). - Table privileges: schema/OID predicates injected into the base WHERE before `aclexplode`/GROUP BY for `list()` and `retrieve()`. - Row counts: `reltuples = -1` is treated as "unknown" and gated on physical size via `pg_relation_size` (a cheap stat call; `relpages` is equally stale pre-vacuum). At or below `THRESHOLD_ESTIMATE_BYTES` (~10MB, derived from `THRESHOLD_COUNT` at a conservative ~200 bytes/row) the exact count runs as before: fast by construction, and it avoids bogus estimates since Postgres floors never-vacuumed heaps at 10 pages, so an empty table would otherwise report ~2K estimated rows. Above the gate the count routes through the EXPLAIN-based `pg_temp.count_estimate`, or returns `-1`/`is_estimate = true` in read-only contexts where the temp function cannot be created. The scoped branch embeds the estimated select via `literal()` instead of legacy's apostrophe-only escaping, so it stays correct under `standard_conforming_strings = off`. `enforceExactCount` unchanged. **Studio:** - The flag decision is contained in the data layer instead of prop-drilled: a small imperative accessor (`apps/studio/data/scoped-introspection.ts`) is hydrated from `useFlag` via a one-line `useSyncScopedIntrospection()` call in `DefaultLayout`, and the query functions read it internally when building the pg-meta SQL. `DefaultLayout` is shared by both the Next and TanStack router trees; hydrating from `_app.tsx` alone would leave TanStack-served pages permanently unscoped since `routes/__root.tsx` mounts its own flag provider. Cold loads cannot race the flag: the query functions await a readiness promise that resolves only after the sync hook has hydrated the accessor with a loaded flag store (immediately on self-hosted where flags are disabled; a 5s safety net armed lazily on the first `ready()` call - not at module import, which would let the timer expire before a project page ever mounts - bounds genuine ConfigCat outages). No component threading, no query-key changes (remaining tradeoff, documented in the module: a mid-session flag flip can serve stale-keyed caches until refetch, fine for a session-stable rollout flag). #47894's existing threading is left as-is and gets deleted together with the flag in the cleanup PR. Also fixes the previously-missing `scoped` pass-through in `getTableRowsCount`. - Flag-independent hardening: `useTableQuery` now sets `retryOnMount: false`, `refetchOnWindowFocus: false` and `staleTime: 5min`. Errored (timed-out) queries no longer refire on every grid cell remount, while stale successful metadata still revalidates on mount after `staleTime`. - `useTableApiAccessQuery` now passes `includedSchemas: [schemaName]`; the client-side filter stays as a safety net. - The rows-count query is `enabled`-gated on the permission check settling, so a transiently-false `canSQLAdminWrite` can no longer cache a read-only `-1` count for a writable user (read replicas short-circuit synchronously as before). **Regression guards (extending the #47894 infrastructure):** - Execution-based scoped-vs-legacy equivalence tests for all four queries: both variants run against the test database and are compared with raw `toEqual` - no normalization, ids included (types across 6 option combos, privileges incl. multi-grantee + PUBLIC, `tables.retrieve` for both identifier branches, row counts for every case where the two paths must agree). Two documented exceptions where only the LEGACY side is sorted, because a de-normalized diagnostic run proved legacy emits genuinely plan-dependent order there (an adversarial-FK fixture shows it is neither oid, name, nor creation order): the `types.list` outer row order (scoped adds `order by t.oid`; legacy has no ORDER BY) and the `tables.retrieve` relationships array (scoped orders by `constraint_name` + column-name tie-breakers - a composite two-column FK expands to 4 entries sharing one constraint_name). Everything else (privileges via `aclexplode` over the same relacl, columns by `ordinal_position`, primary keys by `indkey` order, enums by `enumsortorder`) is byte-identical between the two paths with no test-side help. The one intentional value divergence, never-analyzed tables above the size gate where legacy's exact count is the timeout bug itself, is asserted explicitly as a divergence. - Plan-guard budgets for every scoped query against the stress catalog (extended with 200 enums + 200 composite types). Residual seq scans are justified in-budget: `pg_constraint` max 2 (no index on `confrelid`), `pg_attrdef` max 1, `pg_authid` max 2 (scales with role count, not schema count). - Legacy templates carry a FROZEN do-not-edit marker (they must keep matching production behavior until the flag cleanup deletes them); the ordinary test suite runs against the legacy default, so behavioral drift there fails regular tests. ### Validation - pg-meta: typecheck clean; the affected suites (types, table-privileges, tables, rows-count, catalog-plan-guard) pass in full. - Cross-version: the scoped-vs-legacy equivalence and rows-count behavioral suites were validated on PostgreSQL 14, 15, and 17 (identical results on all three). Two version-marginal planner choices surfaced on 17 (`pg_type` / `pg_class` seq scan vs full-index bitmap for per-schema listings, both structurally unavoidable without an index leading on the namespace column) and are carried as justified plan-guard budget entries. A full 468-test suite run sequentially: 452 passed, 16 failures verified environmental (13 timeouts in an untouched file that passes 27/27 in isolation on the marathon-run cluster, 3 cluster-global role collisions from container reuse). - Studio: `pnpm --filter studio typecheck` clean; 39/39 tests across the touched data hooks; eslint clean on touched files. ### Rollout Same staged ConfigCat rollout as #47894 via `pgMetaScopedIntrospection` (user-email targeting first, then percentage, then 100%). The `useTableQuery` hardening and the API-access schema scoping ship unflagged (behavior-safe). Gate before percentage rollout: functionally verify the FK popover/selector UX under the new `staleTime`/`retryOnMount` settings (a just-edited FK target must not look stale anywhere Studio does not already refetch on save). Once fully rolled out, the legacy templates and flag get deleted together with #47894's in one cleanup PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Supabase
Supabase is the Postgres development platform. We're building the features of Firebase using enterprise-grade open source tools.
- Hosted Postgres Database. Docs
- Authentication and Authorization. Docs
- Auto-generated APIs.
- Functions.
- File Storage. Docs
- AI + Vector/Embeddings Toolkit. Docs
- Dashboard
Watch "releases" of this repo to get notified of major updates.
Documentation
For full documentation, visit supabase.com/docs
To see how to Contribute, visit Getting Started
Community & Support
- Community Forum. Best for: help with building, discussion about database best practices.
- GitHub Issues. Best for: bugs and errors you encounter using Supabase.
- Email Support. Best for: problems with your database or infrastructure.
- Discord. Best for: sharing your applications and hanging out with the community.
How it works
Supabase is a combination of open source tools. We’re building the features of Firebase using enterprise-grade, open source products. If the tools and communities exist, with an MIT, Apache 2, or equivalent open license, we will use and support that tool. If the tool doesn't exist, we build and open source it ourselves. Supabase is not a 1-to-1 mapping of Firebase. Our aim is to give developers a Firebase-like developer experience using open source tools.
Architecture
Supabase is a hosted platform. You can sign up and start using Supabase without installing anything. You can also self-host and develop locally.
- Postgres is an object-relational database system with over 30 years of active development that has earned it a strong reputation for reliability, feature robustness, and performance.
- Realtime is an Elixir server that allows you to listen to PostgreSQL inserts, updates, and deletes using websockets. Realtime polls Postgres' built-in replication functionality for database changes, converts changes to JSON, then broadcasts the JSON over websockets to authorized clients.
- PostgREST is a web server that turns your PostgreSQL database directly into a RESTful API.
- GoTrue is a JWT-based authentication API that simplifies user sign-ups, logins, and session management in your applications.
- Storage a RESTful API for managing files in S3, with Postgres handling permissions.
- pg_graphql a PostgreSQL extension that exposes a GraphQL API.
- postgres-meta is a RESTful API for managing your Postgres, allowing you to fetch tables, add roles, and run queries, etc.
- Kong is a cloud-native API gateway.
Client libraries
Our approach for client libraries is modular. Each sub-library is a standalone implementation for a single external system. This is one of the ways we support existing tools.
| Language | Client | Feature-Clients (bundled in Supabase client) | ||||
|---|---|---|---|---|---|---|
| Supabase | PostgREST | GoTrue | Realtime | Storage | Functions | |
| ⚡️ Official ⚡️ | ||||||
| JavaScript (TypeScript) | supabase-js | postgrest-js | auth-js | realtime-js | storage-js | functions-js |
| Flutter | supabase-flutter | postgrest-dart | gotrue-dart | realtime-dart | storage-dart | functions-dart |
| Swift | supabase-swift | postgrest-swift | auth-swift | realtime-swift | storage-swift | functions-swift |
| Python | supabase-py | postgrest-py | gotrue-py | realtime-py | storage-py | functions-py |
| 💚 Community 💚 | ||||||
| C# | supabase-csharp | postgrest-csharp | gotrue-csharp | realtime-csharp | storage-csharp | functions-csharp |
| Go | - | postgrest-go | gotrue-go | - | storage-go | functions-go |
| Java | - | - | gotrue-java | - | storage-java | - |
| Kotlin | supabase-kt | postgrest-kt | auth-kt | realtime-kt | storage-kt | functions-kt |
| Ruby | supabase-rb | postgrest-rb | - | - | - | - |
| Rust | - | postgrest-rs | - | - | - | - |
| Godot Engine (GDScript) | supabase-gdscript | - | - | - | - | - |
Badges
[](https://supabase.com)
<a href="https://supabase.com">
<img
width="168"
height="30"
src="https://supabase.com/badge-made-with-supabase.svg"
alt="Made with Supabase"
/>
</a>
[](https://supabase.com)
<a href="https://supabase.com">
<img
width="168"
height="30"
src="https://supabase.com/badge-made-with-supabase-dark.svg"
alt="Made with Supabase"
/>
</a>
Translations
- Arabic | العربية
- Albanian / Shqip
- Bangla / বাংলা
- Bulgarian / Български
- Catalan / Català
- Croatian / Hrvatski
- Czech / čeština
- Danish / Dansk
- Dutch / Nederlands
- English
- Estonian / eesti keel
- Finnish / Suomalainen
- French / Français
- German / Deutsch
- Greek / Ελληνικά
- Gujarati / ગુજરાતી
- Hebrew / עברית
- Hindi / हिंदी
- Hungarian / Magyar
- Nepali / नेपाली
- Indonesian / Bahasa Indonesia
- Italiano / Italian
- Japanese / 日本語
- Korean / 한국어
- Lithuanian / lietuvių
- Latvian / latviski
- Malay / Bahasa Malaysia
- Norwegian (Bokmål) / Norsk (Bokmål)
- Persian / فارسی
- Polish / Polski
- Portuguese / Português
- Portuguese (Brazilian) / Português Brasileiro
- Romanian / Română
- Russian / Pусский
- Serbian / Srpski
- Sinhala / සිංහල
- Slovak / slovenský
- Slovenian / Slovenščina
- Spanish / Español
- Simplified Chinese / 简体中文
- Swedish / Svenska
- Thai / ไทย
- Traditional Chinese / 繁體中文
- Turkish / Türkçe
- Ukrainian / Українська
- Vietnamese / Tiếng Việt
- List of translations