From ad0ed2cdbc90ca72e5cf260834e3ee8028c0a492 Mon Sep 17 00:00:00 2001 From: Steven Eubank <47563310+smeubank@users.noreply.github.com> Date: Tue, 29 Sep 2026 20:55:02 -0500 Subject: [PATCH] Update docs based on SRE Agent findings (#50910) ## Problem SRE Agent running against a project which is read-only due to disk being full. ## Solution The SRE agent struggled to find the information which is now included in this PR. ## Review instructions - https://supabase.com/docs/guides/api/rest/postgrest-error-codes - https://supabase.com/docs/guides/observability/advanced-log-filtering - https://supabase.com/docs/guides/platform/database-size ## Checklist Check all before review: - [x] I have read [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) - [x] If I wrote a new docs topic or edited an existing topic, I used the `/write-the-docs` or `/edit-the-docs` skill, which references [WORD_LIST](https://github.com/supabase/supabase/blob/master/apps/docs/WORD_LIST.md) and the docs [CONTRIBUTING](https://github.com/supabase/supabase/blob/master/apps/docs/CONTRIBUTING.md) guide ## Summary by CodeRabbit * **Documentation** * Added guidance for recognizing platform-related PostgreSQL errors, including read-only mode, disk exhaustion, connection-pool limits, and database restarts or failovers. * Added SQL queries for grouping PostgreSQL errors and reviewing recent error events while filtering out selected platform-level codes. * Clarified that read-write transaction settings apply only to the current session, and that background writes resume automatically after read-only mode ends. --- .../guides/api/rest/postgrest-error-codes.mdx | 13 +++++++ .../observability/advanced-log-filtering.mdx | 35 +++++++++++++++++++ .../content/guides/platform/database-size.mdx | 27 ++++++++++++-- 3 files changed, 73 insertions(+), 2 deletions(-) diff --git a/apps/docs/content/guides/api/rest/postgrest-error-codes.mdx b/apps/docs/content/guides/api/rest/postgrest-error-codes.mdx index 19288d73d08..8c91fbe3694 100644 --- a/apps/docs/content/guides/api/rest/postgrest-error-codes.mdx +++ b/apps/docs/content/guides/api/rest/postgrest-error-codes.mdx @@ -67,6 +67,19 @@ Here's the text formatted as a proper markdown table: | 42501 | if authenticated 403, else 401 | insufficient privileges | | other | 400 | | + + +Some codes in this table are triggered by platform conditions rather than application code. Seeing them in high volume usually points to an infrastructure event rather than a bug in your queries: + +- **25006** — The database has entered read-only mode due to disk quota. See [Read-only mode](/docs/guides/platform/database-size#read-only-mode) for causes and recovery steps. +- **53100** — Disk is full. Supabase emits this alongside 25006 during severe disk exhaustion. +- **53300** — Too many connections. The connection pool has reached its limit; check your [connection pool settings](/docs/guides/database/connection-management). +- **57P03** — The database cannot accept connections, typically during a restart or failover. + +When you see these codes in volume, check the [Database dashboard](/dashboard/project/_/observability/database) before debugging application code. + + + ## API level errors ### Connection errors diff --git a/apps/docs/content/guides/observability/advanced-log-filtering.mdx b/apps/docs/content/guides/observability/advanced-log-filtering.mdx index fc46a5c8b22..ecf286ca5d4 100644 --- a/apps/docs/content/guides/observability/advanced-log-filtering.mdx +++ b/apps/docs/content/guides/observability/advanced-log-filtering.mdx @@ -95,6 +95,41 @@ limit 100; Combine predicates with `and`, `or`, and `not`. Select only the fields needed for the investigation. To correlate sources, use an identifier present in both; a shared timestamp alone does not establish that events belong to the same request. +## Filter by Postgres error code [#sqlstate-filtering] + +Filter `postgres_logs` by SQLSTATE to surface errors at a specific category rather than by message text. The code lives in `log_attributes['parsed.sql_state_code']`. + +To see which error codes are occurring across a time window: + +```sql +select + log_attributes['parsed.sql_state_code'] as sqlstate, + count() as occurrences +from logs +where source = 'postgres_logs' + and log_attributes['parsed.sql_state_code'] != '' +group by sqlstate +order by occurrences desc +limit 50; +``` + +During a platform-level incident (such as the database entering read-only mode), codes like `25006` (`read_only_sql_transaction`) and `53100` (`disk_full`) will dominate the results. To focus on application errors while a platform incident is active, exclude the known platform codes: + +```sql +select + timestamp, + log_attributes['parsed.sql_state_code'] as sqlstate, + event_message +from logs +where source = 'postgres_logs' + and log_attributes['parsed.sql_state_code'] not in ('25006', '53100', '57P03') + and log_attributes['parsed.error_severity'] = 'ERROR' +order by timestamp desc +limit 50; +``` + +SQLSTATE codes starting with `25` or `53` usually indicate platform-level resource events rather than application bugs. See the [PostgREST error codes reference](/docs/guides/api/rest/postgrest-error-codes#database-level-errors) and [Database size guide](/docs/guides/platform/database-size#read-only-mode) for context. + ## Query limits [#limit-and-result-row-limitations] Use an explicit `limit` and narrow time range. The logs query surface rejects `select *` and `count(*)`; list columns and use `count()`. A result limit bounds returned rows, not the time range scanned. diff --git a/apps/docs/content/guides/platform/database-size.mdx b/apps/docs/content/guides/platform/database-size.mdx index bf8532ebd3d..65466517d3e 100644 --- a/apps/docs/content/guides/platform/database-size.mdx +++ b/apps/docs/content/guides/platform/database-size.mdx @@ -124,7 +124,24 @@ To resolve it, upgrade your plan or disable your Spend Cap to lift the restricti In some cases Supabase may put your database into read-only mode to prevent your database from exceeding the billing or disk limitations. -In read-only mode, clients will encounter errors such as `cannot execute INSERT in a read-only transaction`. Regular operation (read-write mode) is automatically re-enabled once usage is below 95% of the disk size, +In read-only mode, clients will encounter errors such as `cannot execute INSERT in a read-only transaction`. The Postgres SQLSTATE code for this error is **25006** (`read_only_sql_transaction`). During severe disk exhaustion, **SQLSTATE 53100** (`disk_full`) may also appear. Regular operation (read-write mode) is automatically re-enabled once usage is below 95% of the disk size. + +While in read-only mode, all write operations are blocked — including background jobs, scheduled tasks, and internal monitoring writes. Data that those jobs write (such as disk usage snapshots) will be stale until the database returns to read-write mode. + +To confirm that your database has entered read-only mode, query Postgres logs for the relevant SQLSTATE codes using the [Logs Explorer](/dashboard/project/_/logs/explorer) with the query source set to **Logs**: + +```sql +select + timestamp, + log_attributes['parsed.error_severity'] as severity, + log_attributes['parsed.sql_state_code'] as sqlstate, + event_message +from logs +where source = 'postgres_logs' + and log_attributes['parsed.sql_state_code'] in ('25006', '53100', '57P03') +order by timestamp desc +limit 50; +``` ### Disabling read-only mode @@ -136,7 +153,7 @@ First, change the [transaction access mode](https://www.postgresql.org/docs/curr set session characteristics as transaction read write; ``` -This allows you to delete data from within the session. After deleting data, consider running a vacuum to reclaim as much space as possible: +This allows you to delete data from within the current session. After deleting data, consider running a vacuum to reclaim as much space as possible: ```sql vacuum; @@ -148,6 +165,12 @@ Once you have reclaimed space, you can run the following to disable [read-only]( set default_transaction_read_only = 'off'; ``` + + +`SET SESSION CHARACTERISTICS` applies only to the current session. Background jobs and scheduled tasks resume writes automatically once the platform exits read-only mode — no manual step is needed. + + + ### Disk size distribution You can check the distribution of your disk size on your [project's Infrastructure page](/dashboard/project/_/settings/infrastructure).