When Assistant tries to schedule crons in read-only mode, it succeeds
but creates the jobs under the `supabase_read_only_user`. This causes
permission errors when user try to delete or unschedule them from the
Cron dashboard. The root fix will be to enforce read-only transactions
for that user. In the meantime, this PR steers Assistant to avoid the
mistake.
**Changes**
- Prompts `execute_sql` to treat side-effecting function calls such as
`cron.schedule()` as write queries.
- Adds tool input assertions for "Tool Usage" scorer and a focused cron
regression eval.
- Updates eval mocks to show pg_cron extension as installed so it can
call `cron.schedule()`
**Verification**
See [this
trace](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=experiment&object_id=4a9e8c0e-83b7-4555-8502-365662c3ec8e&r=e041e69b-b70f-41d1-b88c-e8f7888c3de5&s=e041e69b-b70f-41d1-b88c-e8f7888c3de5)
from Braintrust where the new eval passes "Tool Usage", correctly using
`isWriteQuery` for the `cron.schedule()`
Closes AI-737
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Tool evaluation now validates tool inputs (including exact and
substring matches) in addition to tool presence.
* **Tests**
* Added a test confirming cron-scheduling behavior and that SQL
scheduling/enqueue calls are treated as write operations.
* **Chores**
* Added pg_cron to mock extension data.
* Clarified description that SQL calls with side effects should be
treated as writes.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/supabase/supabase/pull/45997?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Adds prompt guardrails and evals to prevent the AI assistant from asking
users to share sensitive data (API keys, `.env` contents, etc.) and to
warn when credentials are shared.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Stronger safety behavior: assistant now refuses requests to share full
environment files, asks for variable names only, and directs users to
secure secret-management tooling.
* Immediate warning and guidance if credentials or other sensitive
values are pasted in chat, without repeating exposed secrets.
* **Behavior**
* Clarified evaluation rules so responses more consistently follow the
new safety guidance.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Prevents the AI assistant from helping with local git/filesystem
operations, and adds explicit warnings before irreversible database
operations (DROP TABLE, DELETE without WHERE, etc.).
Adds a `safetyScorer` and eval cases to cover these behaviours.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a Safety metric to evaluations so assistant responses are scored
for safe handling of destructive or risky requests
* Assistant guidance updated to refuse destructive local VCS/filesystem
actions and require clear warnings for irreversible database operations
* **Tests**
* Added evaluation cases covering safe refusals, clear warnings, and
correct handling of destructive or risky prompts
* **Chores**
* Enabled Safety metric in online evaluation manifests/handlers
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Updates the RLS knowledge loaded by the dashboard AI assistant to
explain the new secure-by-default functionality.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Clarified PostgreSQL/RLS guidance in Studio: tables are now "secure by
default"—SQL-created tables aren’t exposed via the Data API unless
explicit grants are given to anon/authenticated/service_role and RLS is
enabled; added an “Exposing a Table to the Data API” workflow,
strengthened RLS prerequisites in best practices, and improved
troubleshooting/error-recovery guidance.
* **Tests / Evaluations**
* Added an evaluation case validating guidance for non-RLS tables
requiring explicit grants and RLS policies.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Ali Waseem <waseema393@gmail.com>
Moves knowledge (RLS, Edge Functions, PostgreSQL best practices,
Realtime) out of the static system prompt and into a `load_knowledge`
tool the model calls on demand, reducing prompt bloat. This is a
temporary stopgap until the [standard Supabase
agent-skills](https://github.com/supabase/agent-skills) are ready for
integration in Assistant.
- New always-available `load_knowledge` tool added to
`rendering-tools.ts`
- Updated `Message.Parts.tsx` so the "Ran load_knowledge" chip renders
in chat
- System prompt replaces the four knowledge blobs with an `## Available
Knowledge` block and is hardened to load knowledge for given topics
- New "Knowledge Usage" scorer and `requiredKnowledge` assertions check
that knowledge loads as expected in test scenarios
- Filters GraphQL error responses out of `output.docs` before
faithfulness scoring to reduce noise
See "Knowledge Usage" scoring 100% in evals with no major regressions:
https://github.com/supabase/supabase/pull/44296#issuecomment-4145760236
Sample trace showing the tool in action
([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=project_logs&object_id=5a8d02e5-b3b6-40cc-ba76-ecee286478f4&r=351a11c8-9cb7-4945-93ad-d11e8cc2e3e1&s=351a11c8-9cb7-4945-93ad-d11e8cc2e3e1))
<img width="2192" height="1730" alt="CleanShot 2026-03-30 at 13 53
59@2x"
src="https://github.com/user-attachments/assets/f483767c-34e0-401c-8089-5b9834fe696a"
/>
**References**
- https://ai-sdk.dev/cookbook/guides/agent-skills
Closes AI-508
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added dynamic knowledge loading capability enabling the AI assistant
to retrieve on-demand information about PostgreSQL best practices, Row
Level Security, Edge Functions, and Realtime.
* **Bug Fixes**
* Improved search results filtering to exclude error responses in tool
outputs.
* **Tests**
* Enhanced evaluation metrics with knowledge usage scoring.
* Expanded test dataset cases to validate knowledge requirement
handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Logic changes**
- Adds function in `helpers.ts` to extract URLs from text via regex
- I also considering using a library like
[linkify-it](https://www.npmjs.com/package/linkify-it) for this but
figured it's not worth the extra dep
- Adds associated tests in `helpers.test.ts`
- Adds "URL Validity" scorer which performs a HEAD request for links in
Assistant response text and determins what portion of links have `.ok`
responses
- Adds eval case to check correctness of support ticket URL answers
**Prompt changes**
- Informs Assistant of https://supabase.com/dashboard/support/new being
the URL to create support tickets
- Encourages Assistant to "self-debug" issues before directing users to
create support tickets
See [Eval
Report](https://github.com/supabase/supabase/pull/42227#issuecomment-3807772871)
and
[Correctness](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=experiment&object_id=1ad0f9b0-5adb-436c-9812-a87aac62c036&r=1ef13459-a98c-4904-925e-6d81276cebb2&s=dbe5c607-a560-462b-8745-41d430744431)
analysis for new support ticket test case.
Resolves AI-384
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added URL validity scoring to evaluations and helper utilities for
extracting/cleaning URLs.
* Added evaluation cases for support-ticket URL handling and OAuth
callback guidance.
* **Documentation**
* Updated assistant guidance to prefer self-resolution, include
support-ticket direction, clarified data-recovery search steps, and
added template-URL notation.
* **Tests**
* Expanded URL extraction and related utility tests to cover many
formats and edge cases.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* feat: SQL correctness scorer, override mock tables
* feat: replace "SQL Correctness" with "SQL Identifier Quoting" scorer
* fix(prompt): discourage simulating confirmation of execute_sql tool
this is already handled at the UI layer
* fix(prompt): encourage quotes on identifiers with caps
* feat: move extractIdentifiers to own file, add tests
* chore: shorten tests
* feat: extract ColumnDef column names in extractIdentifiers
* refactor: sqlIdentifierQuotingScorer with more thorough checks
* refactor: consolidate into `sql-identifier-quoting.ts`
* feat: support mocking schemas, eval test case with case sensitive schema
* fix: test cases that don't match default mock schema
* chore: format
* feat(prompt): mention special characters and reserved words
* feat: optional description in metadata, test with special characters
* feat: consolidated comprehensive test case
* fix(prompt): revert conflicting instruction
Matt Rossman
