Commit Graph
27 Commits
Author SHA1 Message Date
Joshen Lim 4901f081e5 Migrate remaining requests to pg-meta API to use query endpoint (#47758)
## Context

Migrates the remaining API requests to the pg-meta endpoint to use the
query endpoint directly with the SQL from the pg-meta package. This
touches the following:
- policies
- publications
- triggers
- views
- materialized views
- types

## To test
Just need to verify that we're still fetching the data correctly on
these pages
- Database policies
- Database publications
- Database triggers
- Database tables (views + materialized views)
- Database types

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved and stabilized loading of database metadata (views, triggers,
RLS policies, publications, materialized views, and enum types),
including more reliable schema-scoped filtering.
* Updated policy loading behavior and related UI queries to consistently
use schema arrays, improving cache correctness and consistency.
* **Tests**
* Updated end-to-end test synchronization to wait for the correct
metadata responses using more specific request identifiers.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-09 17:09:03 +08:00
Carel de WaalandAlaister Young 0acc0eb8b3 feat: Support Form - Sync AI assistant conversation to Front (#46778)
# Sync AI assistant conversation to Front

## What & why

When a user submits a support ticket, an AI assistant chat opens so they
get help
immediately while waiting for a human agent. This PR mirrors every turn
of that chat into
the Front conversation the support form already created, so the support
team sees the full
context and Front automations (routing, emails, CSAT) can act on it.

Studio holds no Front credentials — it calls the platform endpoints (see
the platform PR)
to do the syncing. The assistant card is gated behind the
`supportAssistantFollowUp`
ConfigCat flag.

## How it works

1. **Submit** — `SupportFormV3` generates a stable `threadRef` (via the
`uuid` package —
`crypto.randomUUID()` is `undefined` in insecure contexts like
non-localhost HTTP and
would throw, silently aborting the submit) and sends it on
`/platform/feedback/send`.
The response returns the Front `conversationId`. Both are stored on
`SubmittedSupportRequest`.
2. **Open chat** — `SupportAssistantSuccessCardContent` opens a chat
seeded with
`supportMetadata` (`threadRef`, `frontConversationId`, subject,
category, severity, …).
   The first message is a `<support>…</support>` XML block.
3. **First user message** — the chat is tagged `isSupportChat = true`;
the `onFinish` hook
   fires `syncSupportChatToFront`.
4. **Subsequent turns** — each `onFinish` slices the unsynced delta,
strips the XML
metadata block from the seed message, and posts to the platform messages
endpoint.
5. **Escalation / resolve** — the `escalate_to_human` /
`resolve_support_conversation` tools
(and manual **Escalate**/**Resolve** buttons in the assistant input)
flip lifecycle status
via `setSupportLifecycleStatus` → `syncSupportLifecycleToFront`, which
calls the
escalation/resolve endpoints. Front rules act on `ai_support_status`.
The assistant only
   resolves after the user explicitly confirms the issue is fixed.

## Key design decisions

- **`threadRef` as the shared key** — one UUID travels as `threadRef` on
submit and as
`chatId` on every sync, so all messages thread into a single Front
conversation.
- **`conversationId` from the form response** — passed to all
sync/lifecycle calls so the
  platform skips lazy derivation and PATCHes custom fields directly.
- **Delta-only sync** — `lastSyncedMessageCount` tracks what's been
sent; the boundary is
snapshotted before the async call to avoid skipping messages that arrive
mid-flight.
- **Server-side de-dup** — stable `external_id` (`chatId:msg.id`) means
retries don't
  duplicate in Front.
- **Fire-and-forget** — sync failures log to Sentry, never break the
chat; `isSyncing`
resets on rehydration so the next `onFinish` retries the same delta.
Message and lifecycle
syncs use separate guards (`isSyncing` / `isLifecycleSyncing`) so an
in-flight message
  sync can't drop an escalate/resolve.
- **Lifecycle queued until the conversation exists** — if a lifecycle
transition is requested
before the initial message sync has returned a `frontConversationId`,
it's stored as
`pendingLifecycleStatus` and flushed once the id is assigned, rather
than dropped.
- **Tools return immediately** — the lifecycle tools return a stub to
the AI SDK; the real
Front call happens in `onFinish`, keeping async I/O out of the tool
execute path.
- **XML seed stripped before sync** — only the user's actual `<message>`
is sent to Front
  (or dropped entirely if the form already created the conversation).

## Changes

| Area | File(s) |
| --- | --- |
| Support form state | `SupportForm.state.ts` — `threadRef` /
`frontConversationId` on `SubmittedSupportRequest` |
| Support form submit | `support-ticket-send.ts` — sends `threadRef`,
reads `conversationId` |
| Support form UI | `SupportFormV3.tsx` — generates `threadRef`, stores
`conversationId` |
| AI assistant state | `ai-assistant-state.tsx` — `SupportChatMetadata`,
`setSupportLifecycleStatus`, `onFinish` wiring, tool handling |
| Message sync | `state/ai-chat-front-sync.ts` — delta tracking, message
filtering, initial vs. incremental |
| API data layer | `data/feedback/ai-chat-front-sync.ts` — typed
platform-client wrappers for the three conversation endpoints |
| Support tools | `lib/ai/tools/support-tools.ts` — `escalate_to_human`,
`resolve_support_conversation` |
| Tool integration | `lib/ai/tool-filter.ts`, `tools/index.ts`,
`generate-assistant-response.ts` |
| Success card | `SupportAssistantSuccessCardContent.tsx` — tags chat on
first engagement |
| Assistant panel UI | `AIAssistant.tsx` — Escalate/Resolve buttons,
disabled input on closed chats, support placeholders |



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

- **New Features**
  - Support chats now include “Escalate to human” and “Resolve” actions.
- Support submissions can be associated with a stable Front thread via a
generated `threadRef`, preserving linkage across follow-ups.
- AI assistant responses and input hints adapt when support mode is
active.

- **Bug Fixes**
- Improved support chat state management and lifecycle handling to keep
conversation metadata and message history synchronized more reliably
with Front.

- **Chores**
- Added/updated coverage to reflect the new support-chat state and
syncing behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com>
2026-07-08 12:41:53 +02:00
Pedro RodriguesandClaude Opus 4.8 c4c213ce3d feat(studio): switch dashboard assistant to remote MCP server (#47479)
## I have read the
[CONTRIBUTING.md](<https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md>)
file.

YES

## What kind of change does this PR introduce?

Feature / refactor.

## What is the current behavior?

The dashboard assistant runs `@supabase/mcp-server-supabase` in-process
over an in-memory transport (`lib/ai/supabase-mcp.ts`).

## What is the new behavior?

The assistant connects to the **remote MCP server** over HTTP
(`@ai-sdk/mcp`), forwarding the dashboard session token as a bearer. URL
comes from `NEXT_PUBLIC_MCP_URL` with a local-dev fallback;
platform-only, and Nimbus works via the same env var.

* **Tool model unchanged:** UI-controlled `execute_sql` (with
`needsApproval`) and `deploy_edge_function` still come from Studio; the
allowlist (`TOOL_CATEGORY_MAP`) remains the gate keeping the remote's
write tools away from the assistant (`read_only` is defense-in-depth).
* **Attribution:** sends `x-source-name: supabase-studio` (+
`x-source-version`) → logged as `source_name`/`client_name`.
* **Connection lifecycle:** the HTTP client is closed via the request's
`AbortSignal` (tools execute later during streaming); `signal` is
required on `getTools`/`getMcpTools`.
* **Resilience:** a remote-MCP failure degrades to the remaining tools
instead of failing the assistant.
* **Drift protection:** relied-upon tools are typed against `keyof
typeof supabaseMcpToolSchemas`, so a package bump that renames/removes
one fails `pnpm typecheck`; a runtime check also warns if the deployed
server returns fewer tools.
* Adds unit tests for the above.

## Additional context

* Verified end-to-end against a local remote MCP server with a dashboard
token: `initialize` 200, tools listed, a tool executed, client closed
cleanly.
* The remote MCP (mgmt-api) already accepts dashboard session tokens
(GoTrue-JWT auth path) — no backend change needed. `NEXT_PUBLIC_MCP_URL`
must point at each env's `/mcp`.
* `@supabase/mcp-server-supabase` is kept — still used by the
self-hosted `/api/mcp` routes.

Closes
[AI-137](https://linear.app/supabase/issue/AI-137/switch-dashboard-assistant-to-remote-mcp)

## Rollout

* **Rollout:** merges with `USE_REMOTE_MCP` off (in-process); flip it to
`true` per environment (staging → prod → Nimbus) once each one's
prerequisites land.
* **Rollback:** unset `USE_REMOTE_MCP` and redeploy to fall back to the
in-process client — no revert needed.

## Summary by CodeRabbit

* **Bug Fixes**
* Improved AI request handling so tool loading and generation clean up
properly when a request is cancelled or the browser connection closes.
* Added safer fallback behavior when remote tool loading fails, so AI
features can continue with available tools instead of stopping entirely.
* Updated remote tool access to use the current project reference and
preserve the correct access headers.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* AI tools now connect more reliably to remote services and stop cleanly
when requests end or are canceled.
* Tool loading is more resilient, continuing with available tools if
remote access is unavailable.

* **Bug Fixes**
* Improved cleanup to prevent lingering connections during SQL
generation and policy workflows.
  * Added safer handling for remote tool changes and invalid responses.

* **Tests**
* Expanded automated coverage for remote tool setup, cancellation, and
fallback behavior.


<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 19:38:21 +01:00
Gildas Garcia 0e3364bbad Chore/cleanup studio deps (#47399)
## Problem

Knip reported some unused dependencies. Some are actually used in
builds, etc but others are not.

## Solution

Remove the really unused dependencies

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Cleaned up unused dependencies and removed some obsolete test/support
files.
* Updated project ignore rules to better match current app structure and
generated files.
* **Bug Fixes**
* No user-facing behavior changed; this release is focused on
maintenance and cleanup.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-06-30 14:03:40 +02:00
Joshen Lim 1baaded0bb Consolidate execute-sql-query into execute-sql-mutation (#46944)
## Context

Just some clean up as I was going through stuff
- `useExecuteSqlQuery` is deprecated and not used at all
- As such `execute-sql-query` is technically irrelevant, the more
relevant file is `execute-sql-mutation`
- Hence opting to consolidate `execute-sql-query` into
`execute-sql-mutation`
- Also removing `ExecuteSqlError` since its just re-exporting the
`ResponseError` type

There's a lot of file changes but its essentially just updating the
importing statements across the files
2026-06-16 00:07:16 +08:00
Matt Rossman 65fab30935 feat(ai): judge tool inputs, add storage guidance and permissive RLS evals (#46168)
Adding broad RLS policies to public buckets can cause users to expose
more than they expected, like the ability to list all profile pictures
on an app. This patches Assistant with knowledge to follow our latest
guidance on restrictive RLS policies for storage buckets
https://github.com/supabase/supabase/pull/46172

**Changes**
- Adds Storage bucket evals for public website assets and avatar access
patterns to distinguish public vs private bucket use cases
- Adds eval for overly permissive table policies
- Adds `storage` knowledge so Assistant distinguishes public buckets,
private buckets, object reads, and object listing.
- Adds `includeToolCallInputs` option for scorer transcripts so LLM
judges can evaluate proposed SQL/tool actions.
- Bumps max step count to 10 since storage knowledge may incur another
tool call (also 10 is recommended
[here](https://vercel.com/academy/ai-sdk/multi-step-and-generative-ui#why-multi-step-is-required)
for complex multi-tool scenarios)

**References**
-
https://supabase.com/docs/guides/storage/buckets/fundamentals#public-buckets
- https://supabase.com/docs/guides/storage/security/access-control
- https://github.com/supabase/supabase/pull/46172

**Notes:**
- These prompt tweaks are not meant to be exhaustive fixes, they are
mainly hotfixes intended to hold us out until these cases can be
addressed more deeply in skills/docs and tracked in a central evals

Closes AI-676
Closes AI-756

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added Storage knowledge resource for the assistant covering Supabase
Storage access patterns and RLS guidance.
* Added three evaluation cases: two for Storage (marketing assets,
avatars) and one for RLS policy generation for user profiles.

* **Improvements**
  * Evaluators now include tool call inputs when judging conversations.
* Assistant prompts and generation enhanced with richer Storage/RLS
guidance and extended streaming limits.

* **Tests**
* Added test ensuring tool call inputs are included in serialized thread
context.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/supabase/supabase/pull/46168?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-29 09:55:23 -04:00
Saxon Fletcher 80da153450 Fix for AI Assistant query and deploy confirmation (#46052)
When Assistant requests confirmation to run a query or deploy an edge
function if the user doesn't skip or run and instead sends a follow-up
message it errors out. This allows follow-up messages and treats them as
"skips" which means adjusting confirmation message state as part of the
follow-up. This also uses toModelOutput to cleanse data based on
permissions.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Enhanced tool approval workflow: pending approvals are now
automatically resolved as denied when submitting new messages
* Improved chat input state management with better handling of approval
states
  * Customizable loading messages for tool operations

* **Bug Fixes**
  * Fixed chat input availability during pending tool approval states
  * Improved tool execution feedback during approval workflows

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/supabase/supabase/pull/46052?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-20 09:09:28 +10:00
Matt Rossman 212ccf8135 fix(ai): contextualize cron schedule as SQL writes, score in "Tool Usage" (#45997)
When Assistant tries to schedule crons in read-only mode, it succeeds
but creates the jobs under the `supabase_read_only_user`. This causes
permission errors when user try to delete or unschedule them from the
Cron dashboard. The root fix will be to enforce read-only transactions
for that user. In the meantime, this PR steers Assistant to avoid the
mistake.

**Changes**

- Prompts `execute_sql` to treat side-effecting function calls such as
`cron.schedule()` as write queries.
- Adds tool input assertions for "Tool Usage" scorer and a focused cron
regression eval.
- Updates eval mocks to show pg_cron extension as installed so it can
call `cron.schedule()`

**Verification**

See [this
trace](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=experiment&object_id=4a9e8c0e-83b7-4555-8502-365662c3ec8e&r=e041e69b-b70f-41d1-b88c-e8f7888c3de5&s=e041e69b-b70f-41d1-b88c-e8f7888c3de5)
from Braintrust where the new eval passes "Tool Usage", correctly using
`isWriteQuery` for the `cron.schedule()`

Closes AI-737


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Tool evaluation now validates tool inputs (including exact and
substring matches) in addition to tool presence.

* **Tests**
* Added a test confirming cron-scheduling behavior and that SQL
scheduling/enqueue calls are treated as write operations.

* **Chores**
  * Added pg_cron to mock extension data.
* Clarified description that SQL calls with side effects should be
treated as writes.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/supabase/supabase/pull/45997?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)

<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-18 12:11:11 -04:00
Charis 12869fcd9f studio: flip executeSql signature to SafeSqlFragment (7/7) (#46007)
## Summary

Final PR in the SafeSql migration stack. Stacked on top of #46006.

Tightens `executeSql`'s `sql` parameter from `string` to
`SafeSqlFragment`. Any future raw-string caller is now a compile error —
the SafeSql safety property becomes structural rather than
convention-based.

Also adapts the AI `execute_sql` tool to promote AI-generated SQL via
`acceptUntrustedSql(untrustedSql(sql))` inside the `execute` callback.
The tool's existing \`needsApproval: true\` gate ensures `execute` only
runs after the user has explicitly approved — that approval is the
gesture that promotes untrusted to safe.

## Test plan

- [x] `pnpm typecheck` passes
- [x] Grep for any remaining raw-string `executeSql` calls in
`apps/studio` returns nothing
- [x] Dev-server smoke: AI tool approval flow executes SQL
2026-05-15 20:07:54 +00:00
Charis 39ec777d22 studio: SafeSql stragglers + remaining tests (6/7) (#46006)
## Summary

Part 6 of 7 in the SafeSql migration stack. Picks up the few remaining
files that didn't fit cleanly into earlier batches:

- `components/Docs/Description.tsx` — comment statements built via
`safeSql`.
-
`components/Integrations/IntegrationOverviewTabV2/InstallIntegrationSheet.test.tsx`
— test fixture updated to `SafeSqlFragment`.
- `lib/ai/tools/studio-tools.test.ts` — test fixture updated.
- `lib/api/generate-v4.test.ts` — test fixture updated.

Sets up PR 7, which flips the `executeSql` signature itself.

## Test plan

- [x] `pnpm typecheck` passes
- [x] Specific Studio unit tests run on top of the stack
(`Policies.utils.test.ts`, `SidePanelEditor.utils.createTable.test.ts`,
`useQueryInsightsIssues.utils.test.ts`)
- [x] Dev-server smoke: Docs panel renders / accepts edits

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Tests**
  * Updated test suites to reflect internal type definition changes.

* **Refactor**
* Internal code improvements to enhance type safety and consistency
across the codebase.

<!-- review_stack_entry_start -->

[![Review Change
Stack](https://storage.googleapis.com/coderabbit_public_assets/review-stack-in-coderabbit-ui.svg)](https://app.coderabbit.ai/change-stack/supabase/supabase/pull/46006)

<!-- review_stack_entry_end -->

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-15 15:56:57 -04:00
Matt Rossman d143571586 feat(assistant): trace-level scorers + server-side tool execution with needsApproval (#45654)
## Motivation

When Assistant runs a potentially destructive tool like `execute_sql`,
it stops the LLM request and prompts for client-side approval and
execution of the tool. After approval, a second request kicks off under
a separate trace. This has made scoring and
[Topics](https://www.braintrust.dev/blog/topics) classification
challenging, as the generated `output` is split across stateless
requests. The [span-level
scoring](https://www.braintrust.dev/docs/evaluate/custom-code#score-spans)
approach we've used thusfar (after the LLM call, we massage the result
into an `output` payload that's stuck onto the root span) has been
cumbersome and led to invalid scores / topics where only part of the
assistant response is considered. It's also inefficient, as we're
duplicating potentially large info (like the `search_docs` output) that
already exists within the trace.

An alternative to scoring spans is to [score
traces](https://www.braintrust.dev/docs/evaluate/custom-code#score-traces).
Braintrust [best
practices](https://www.braintrust.dev/docs/evaluate/score-online#best-practices)
advise:

> Use span scope for evaluating individual operations or outputs. Use
trace scope for evaluating multi-turn conversations, overall workflow
completion, or when your scorer needs access to the full execution
context.

We've also received [direct
guidance](https://supabase.slack.com/archives/C05QYJBLX89/p1777925770927149?thread_ts=1777905716.911979&cid=C05QYJBLX89)
from their team to use this approach.

## Changes

Migrates eval scorers from custom `AssistantEvalOutput` shape to
trace-level scoring via `trace.getThread()` / `trace.getSpans()`, with
thread parsing that scores the full latest Assistant turn and passes
prior conversation separately where relevant.

Moves `execute_sql` and `deploy_edge_function` from client-side
execution after approval to AI SDK `needsApproval` + server-side
`execute()`. SQL results returned to the model are gated by AI opt-in
level, so row data is only included with `schema_and_log_and_data`;
otherwise the tool returns the no-data-permissions sentinel.

Adds `metadata.isFinalStep` to disambiguate multiple LLM requests within
an "assistant" turn due to tool call requests/responses. For online
evals, this means we should configure automations to only score traces
with `metadata.isFinalStep = true` to ensure we're judging the complete
generated response.

Other minor kaizen changes:
- Renamed `promptProviderOptions` to `systemProviderOptions` to clarify
that this is associated with the "system" message and disambiguate from
the root `providerOptions`
- Adds `evals/trace-utils.ts` to handle Zod validation of the `unknown`
span shapes from Braintrust, to more easily access typed inputs/output
on tool spans.
- Bumps AI SDK floor version `^6.0.116` → `^6.0.174`
- Tweaked the "Conciseness" scorer to not unfairly dock points for the
new `[called tool_name]` labels in serialized assistant response

## Verification

In the studio staging build, I asked Assistant to create a todos table
with 3 sample todos. I manually approved the `execute_sql` call and saw
Assistant generate text before & after the call.

In Braintrust I verified two traces were produced (see [filtered
logs](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs?v=Staging&tvt=trace&search={%22filter%22:[{%22text%22:%22metadata.environment%2520%253D%2520%27staging%27%22,%22label%22:%22metadata.environment%2520%253D%2520%27staging%27%22,%22originType%22:%22btql%22},{%22text%22:%22%2560Chat%2520ID%2560%2520%253D%2520%25221cb2ac45-e5e7-458c-9da4-3bf6863b8842%2522%22,%22label%22:%22Chat%2520ID%2520equals%25201cb2ac45-e5e7-458c-9da4-3bf6863b8842%22,%22originType%22:%22form%22}]})),
the first with `metadata.isFinalStep = false` and the second with
`metadata.isFinalStep = true`.

In the Braintrust staging scorers, I ran the preview Completeness scorer
on the second trace and verified it sees the complete Assistant response
including markers for tool calls ([link to
trace](https://www.braintrust.dev/app/supabase.io/p/Assistant%20(Staging%20Scorers)/trace?object_type=project_logs&object_id=b5214b62-ad1e-4929-9d5b-40b1daebe948&r=0ed0a4f8-8aff-4a34-bb1d-1df1d88a5070&s=ff9015f8-6bf7-4ab3-83a9-ca4e69e27e82))

<img width="1193" height="960" alt="CleanShot 2026-05-07 at 11 27 10@2x"
src="https://github.com/user-attachments/assets/509d4858-c3a1-4068-986d-3aa4d5617d1a"
/>

I also tested the `deploy_edge_function` workflow and verified it still
prompts for permission and warns on deployment of existing functions.

**References**
- https://www.braintrust.dev/docs/evaluate/custom-code#score-traces
-
https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#tool-execution-approval

Supercedes https://github.com/supabase/supabase/pull/45556 and
https://github.com/supabase/supabase/pull/45339

Closes AI-473

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Tool actions (SQL execution, edge-function deploy) now require
explicit user Approve/Deny before proceeding.

* **Improvements**
* Assistant pauses for approval responses before sending follow-ups,
giving clearer control over risky actions.
  * Deploy/replace flows show confirmation and clearer replace warnings.
* Evaluation/scoring updated to use richer trace data for more accurate
assistant performance signals.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-12 15:24:21 -04:00
Joshen Lim 7f5865872a Enforce noUnusedLocals and noUnusedParameters in tsconfig.json + fix all related issues (#45264)
## Context

Enforce `noUnusedLocals` and `noUnusedParameters` in tsconfig.json + fix
all related issues
2026-04-27 17:42:34 +08:00
Manuel RubioandAndrey A. 667be50982 fix(self-hosted): local AI assistant unable to read schemas and interpret/execute SQL (#44757)
## I have read the
[CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md)
file.

YES

## What kind of change does this PR introduce?

Bug fix for #42574 

## What is the current behavior?

If you setup a self-hosted Supabase environment and ask local AI
assistant it is unable to interpret the results for executed SQL
queries.

## What is the new behavior?

Local AI assistant can now read schemas and interpret the results for
executed SQL queries.

## Additional context

I tested it using ollama with a smaller model which does not support
reasoning mode so It would be useful if someone can test it using an
actual OpenAI API KEY.

The main issue was that `executeSql` returns a wrapped `{ result: {...}
}` on `fallback-tools.ts` making the model think there's no result.
The solution is just to unwrap it like it's done at:
`apps/studio/data/database-functions/database-functions-query.ts`:

```
export async function getDatabaseFunctions(...) {
...
  const { result } = await executeSql(...)
...
  return result as DatabaseFunction[]
}
```

Also added a `false` default value in `execute_sql` tool schema so zod
validation does not reject a valid LLM tool call if the model omits the
field (which happened with the smaller model)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
  * Optimized internal schema metadata query handling in AI tools.

* **Chores**
* Enhanced SQL execution tool robustness by providing a sensible default
for query type specification when not explicitly provided.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Andrey A. <56412611+aantti@users.noreply.github.com>
2026-04-20 12:41:23 -04:00
Alaister YoungandAlaister Young 1b1d05ff96 chore: upgrade vite to v8 and vitest to v4 (#44833)
Upgrade vite and vitest to their latest major versions across the
monorepo, along with related packages.

**Changed:**
- `vite` catalog: `^7.3.2` → `^8.0.8` (Rolldown replaces esbuild/Rollup)
- `vitest` catalog: `^3.2.0` → `^4.1.4`
- `@vitejs/plugin-react`: `^4.3.4` → `^6.0.1`
- `@vitest/coverage-v8`: `^3.2.0` → `^4.1.4`
- `@vitest/ui`: `^3.2.0` → `^4.1.4`
- `vite-tsconfig-paths`: `^4.3.2` / `^5.1.4` → `^6.1.1`

**Pinned to vite 7:**
- `apps/lite-studio` — `@react-router/dev` hasn't declared vite 8
support yet
- `blocks/vue` — Nuxt plugins (`vite-plugin-inspect`, `vite-dev-rpc`,
`vite-hot-client`, `vite-plugin-vue-tracer`) haven't declared vite 8
support yet

**Test fixes for vitest 4 breaking changes:**
- **`apps/studio/lib/api/snippets.utils.test.ts`** — Replaced
`vi.mock('fs/promises')` automock with an explicit factory. Vitest 4's
automocking doesn't create mock functions for getter-based exports on
Node built-ins, so `mockedFS.access.mockResolvedValue` etc. were
`undefined`.
- **`apps/studio/lib/api/self-hosted/functions/index.test.ts`** —
Changed `mockReturnValue` to `mockImplementation(function() { ... })`
for a constructor mock. Vitest 4 no longer allows `mockReturnValue` when
the mock is called with `new`.
- **`apps/studio/tests/pages/api/mcp/index.test.ts`** — Changed arrow
function to regular `function` in `mockImplementation` for
`StreamableHTTPServerTransport`. Arrow functions can't be constructors,
and vitest 4 now enforces this.
- **`packages/ui-patterns/vitest.setup.ts`** — Changed `ResizeObserver`
mock from arrow function to regular `function` for the same constructor
enforcement reason. This was crashing Radix popover rendering in jsdom.

## To test

- `pnpm test:studio` — all 226 test files should pass
- `pnpm --filter ui-patterns vitest run` — all 183 tests should pass
- `pnpm --filter www test -- --run` — all 19 tests should pass
- `pnpm --filter ui vitest run` — all tests should pass
- `pnpm --filter dev-tools vitest run` — all tests should pass
- `pnpm --filter ai-commands vitest run` — all tests should pass

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Standardized and updated development tooling versions and version
sources for consistent installs across the repo (Vite, Vitest,
vite-tsconfig-paths and related plugins/catalog entries).
* **Tests**
* Improved test mocks and typings (updated mock
factories/implementations and tightened spy/type assertions) to increase
test reliability and compatibility with updated tooling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Alaister Young <10985857+alaister@users.noreply.github.com>
2026-04-16 00:13:48 +09:00
Charis 205cbe7d26 chore(studio}: enforce import order, remove bare import specifiers (#44585) 2026-04-07 20:34:10 -04:00
Matt Rossman 82deff37de feat(assistant): lazy load topic knowledge via load_knowledge tool (#44296)
Moves knowledge (RLS, Edge Functions, PostgreSQL best practices,
Realtime) out of the static system prompt and into a `load_knowledge`
tool the model calls on demand, reducing prompt bloat. This is a
temporary stopgap until the [standard Supabase
agent-skills](https://github.com/supabase/agent-skills) are ready for
integration in Assistant.

- New always-available `load_knowledge` tool added to
`rendering-tools.ts`
- Updated `Message.Parts.tsx` so the "Ran load_knowledge" chip renders
in chat
- System prompt replaces the four knowledge blobs with an `## Available
Knowledge` block and is hardened to load knowledge for given topics
- New "Knowledge Usage" scorer and `requiredKnowledge` assertions check
that knowledge loads as expected in test scenarios
- Filters GraphQL error responses out of `output.docs` before
faithfulness scoring to reduce noise


See "Knowledge Usage" scoring 100% in evals with no major regressions:
https://github.com/supabase/supabase/pull/44296#issuecomment-4145760236

Sample trace showing the tool in action
([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=project_logs&object_id=5a8d02e5-b3b6-40cc-ba76-ecee286478f4&r=351a11c8-9cb7-4945-93ad-d11e8cc2e3e1&s=351a11c8-9cb7-4945-93ad-d11e8cc2e3e1))

<img width="2192" height="1730" alt="CleanShot 2026-03-30 at 13 53
59@2x"
src="https://github.com/user-attachments/assets/f483767c-34e0-401c-8089-5b9834fe696a"
/>


**References**
- https://ai-sdk.dev/cookbook/guides/agent-skills

Closes AI-508

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added dynamic knowledge loading capability enabling the AI assistant
to retrieve on-demand information about PostgreSQL best practices, Row
Level Security, Edge Functions, and Realtime.

* **Bug Fixes**
* Improved search results filtering to exclude error responses in tool
outputs.

* **Tests**
  * Enhanced evaluation metrics with knowledge usage scoring.
* Expanded test dataset cases to validate knowledge requirement
handling.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-02 16:09:06 -04:00
Joshen Lim be26feb9ba Chore/shift manual queries into pg meta 03 (#43951)
## Context

Shifting more dashboard queries into pg-meta so that we centralize all
manually written queries in one place
Having them in packages/pg-meta also allows us to write tests for them

## To test

Just needs a smoke test on
- Table Editor
  - Fetching entities
  - Viewing definition 
- SQL Editor
  - View ongoing queries
  - Abort queries
- Integrations
  - Queues
- Database
  - Migrations
  -Triggers (Updating)
2026-03-19 18:31:46 +08:00
Matt Rossman 25036af80e fix(assistant): sanitize backslash-escaped apostrophes in SQL (#43728)
Fix for the LLM occasionally generating MySQL-style `\'` escapes in SQL,
which are invalid in PostgreSQL.

Example trace where this happened in the wild:
([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Assistant/review?tab=experiment&r=5fcf1b12-8584-455c-9e9a-bdc0fa3ed21c&s=5fcf1b12-8584-455c-9e9a-bdc0fa3ed21c&o=0627ada8-b567-4117-9fe8-49d847cb73a7&review=1))

**Changes**
- Adds `fixSqlBackslashEscapes` to convert `\'` → `''` before SQL is
executed
- Unit tests + adversarial eval dataset case

Compare the results of the adversarial test case:
- `master`: 0% SQL Validity
([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Dev%20(mattrossman%2FAssistant)/trace?object_type=experiment&object_id=b469cbf7-4d6f-429c-9819-6c4099294123&r=dce5a29b-2fde-44c3-80f8-4e14d1f657c0&s=dce5a29b-2fde-44c3-80f8-4e14d1f657c0))
- This branch: 100% SQL Validity
([Braintrust](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=experiment&object_id=160e9ce0-e320-4f6d-8aa7-c5ad7e01fbd2&r=d75ef0e3-90ed-42a7-9ef3-8bf69592f193&s=0eeca492-dbe6-451e-8d81-127caff30320))

Closes AI-400
2026-03-17 13:44:14 -04:00
Jordi Enric ac64a902c1 chore: adds tests (#42653) 2026-02-11 09:50:11 +01:00
Matt Rossman eb259f1364 feat(assistant): score and improve SQL identifier quoting (#42122)
* feat: SQL correctness scorer, override mock tables

* feat: replace "SQL Correctness" with "SQL Identifier Quoting" scorer

* fix(prompt): discourage simulating confirmation of execute_sql tool

this is already handled at the UI layer

* fix(prompt): encourage quotes on identifiers with caps

* feat: move extractIdentifiers to own file, add tests

* chore: shorten tests

* feat: extract ColumnDef column names in extractIdentifiers

* refactor: sqlIdentifierQuotingScorer with more thorough checks

* refactor: consolidate into `sql-identifier-quoting.ts`

* feat: support mocking schemas, eval test case with case sensitive schema

* fix: test cases that don't match default mock schema

* chore: format

* feat(prompt): mention special characters and reserved words

* feat: optional description in metadata, test with special characters

* feat: consolidated comprehensive test case

* fix(prompt): revert conflicting instruction
2026-01-27 09:28:56 -05:00
Danny WhiteandCharis Lam 05dc676d36 feat(studio): incident-aware AI assistant (#41603)
* handling and mock data support

* admonition

* only show in empty state

* rabbit

* mock without local overrides

* remove admonition

* make incident banner more prominent

* remove mock data

* move to tool

* prettier

* fix(studio): get_active_incidents tool

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
2026-01-21 13:52:29 +11:00
072883bcec feat: assistant evals (#41311)
* chore: bump `supabase` CLI

* chore: stricter message types in `generate-v4.ts`

* feat: tutorial eval

https://www.braintrust.dev/docs/evaluation

* feat: project ID for eval

* refactor: `generateAssistantResponse` out of `handlePost`

* refactor: generateAssistantResponse to lib/ai

* feat: factuality eval with assistant response

* chore: upgrade braintrust to v1.0.1

* chore: silence tsconfig warning

* feat: assertion scorer

* fix: aggregate tools across all steps

* refactor: strict tool names, remove need for `as const`

* refactor: generic tool name type in assertions

* feat: transfer mocks from `feature/braintrust`

* feat: LLM criteria assertion

* feat: braintrust evals workflow

* fix: BRAINTRUST_PROJECT_ID

* feat: `sql_similar` assertion

* fix: `OPENAI_API_KEY` in workflow env

* feat: split AssertionScorer into separate scorers

* feat: remove tutorial eval

* feat: 20 minute CI timeout

* feat: category in test case metadata

* feat: score with gpt-5

* refactor: dataset to own file, colocate scorers

* feat: "gpt-5.2-2025-12-11" for llm as a judge

* feat: SQL syntax scorer with `libpg-query`

* feat: `evals:setup` and `evals:run` scripts

* feat: `evals:setup` in CI

* feat: human readable scorer names

* chore: rename to "SQL Validity"

* feat: add 2 "sql_generation" test cases

* feat: update requiredTools in test cases

* chore: ignore Cursor MCP config

* feat: "Conciseness" score

* feat: "Completeness" scorer

* fix: generate-v4 test mocks

* feat: serialize "steps" for scorer inputs

* updated node mem options for typecheck

* updated runner

* remove ram update as actions handle this

* feat: read `BRAINTRUST_PROJECT_ID` from secrets

* feat: score helpfulness, remove old scorers

* feat: separate `evals:run` and `evals:upload` scripts

* feat: passthrough entire classifier result

* feat: use live `search_docs` impl, store docs result in metadata

* feat: reduce classifier options

* feat: filter workflow by `run-evals` PR label or `master` branch

* chore: cleanup stubbed mock tools

* fix: checkout actual branch with `ref:`

* fix: capture search_docs results from all content parts

* feat: simplify sql syntax score calculation

* feat: use AI SDK's UI message validator

* docs: justification for relative `extends`

* fix: cleanup leftover validatedMessages

* doc: note mock token isn't secret for snyk

* fix: mock ui message to pass validation

* feat: revert ignoring Cursor MCP config

Using `.git/info/exclude` instead until we have an opinion on this

* feat: add "tsconfig" as shared-data devDependency, revert relative path in tsconfig

* refactor: tool call parsing into function

* Update apps/studio/evals/assistant.eval.ts

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* refactor: organize mock schemas and tool factories

---------

Co-authored-by: Ali Waseem <waseema393@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-12-22 23:45:48 -05:00
Saxon Fletcher 1ae6f4f44f re-add chart config (#39149) 2025-10-02 12:41:29 +10:00
626eb30e77 Assistant action orientated approach (#38806)
* update onboarding

* update model and fix part issue

* action orientated assistant

* fix tool

* lock

* remove unused filter

* fix tests

* fix again

* update package

* update container

* fix tests

* refactor(ai assistant): break out message markdown and profile picture

* wip

* refactor(ai assistant): break up message component

* refactor: break ai assistant message down into multiple files

* refactor: simplify ReportBlock state

* fix: styling of draggable report block header

When the drag handle is showing, it overlaps with the block header.
Decrease the opacity of the header so the handle can be seen and the two
can be distinguished.

* fix: minor tweaks to tool ui

* refactor: simplify DisplayBlockRenderer state

* fix: remove double deploy button in edge function block

When the confirm footer is shown, the deploy button on the top right should be
hidden (not just disabled) to avoid confusion.

* refactor, test: message sanitization by opt-in level

Refactor the message sanitization to have more type safety and be more testable.
Add tests to ensure:

- Message sanitization always runs on generate-v4
- Message sanitization correctly works by opt-in level

* Fix conflicts in pnpm lock

* Couple of nits and refactors

* Revert casing for report block snippet

* adjust sanitised prompt

* Fix tests

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-09-29 03:57:36 +00:00
Greg Richardson 9820707d71 feat: local mcp server (#38797)
* feat: local mcp server

* feat(local-mcp): implement migrations

* fix: remove unsupported mcp args

* feat(local-mcp): tests

* fix(local-mcp): packages to adhere to minimumReleaseAge

* fix(mcp): import path for createSupabaseApiPlatform

* fix(local-mcp): move tests out of pages/api dir

* refactor: self-hosted execute sql logic

* fix: deps
2025-09-23 12:00:08 -06:00
Saxon Fletcher b3a687e7a4 Fix filtering issue (#38613)
filter issue
2025-09-11 16:32:26 +10:00
d60aceb562 Prompt and tool refactoring (#37500)
* try a really long context window to maximize caching

* update examples

* attempt to update packages and useChat

* update endpoints

* update zod

* zod

* update to v5

* message update

* Revert "zod"

This reverts commit ec39bac6b6.

* revert zod

* zod i

* fix complete endpoints

* remove async

* change to content

* type cleanup

* Revert the package bumps to rebuild them.

* Bump zod to 2.25.76 in all packages.

* Bump openai in all packages.

* Bump ai and ai-related packages.

* Remove unneeded files.

* Fix the rest of the migration stuff.

* Prettier fixes.

* add policy list tool

* refactor

* ai sdk 5 fixes

* refactor complete endpoint

* edge function prompt

* remove example

* slight prompt change

* Minor clean up

* More clean up

---------

Co-authored-by: Jordi Enric <jordi.err@gmail.com>
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-08-08 15:25:57 +07:00