## Motivation
When Assistant runs a potentially destructive tool like `execute_sql`,
it stops the LLM request and prompts for client-side approval and
execution of the tool. After approval, a second request kicks off under
a separate trace. This has made scoring and
[Topics](https://www.braintrust.dev/blog/topics) classification
challenging, as the generated `output` is split across stateless
requests. The [span-level
scoring](https://www.braintrust.dev/docs/evaluate/custom-code#score-spans)
approach we've used thusfar (after the LLM call, we massage the result
into an `output` payload that's stuck onto the root span) has been
cumbersome and led to invalid scores / topics where only part of the
assistant response is considered. It's also inefficient, as we're
duplicating potentially large info (like the `search_docs` output) that
already exists within the trace.
An alternative to scoring spans is to [score
traces](https://www.braintrust.dev/docs/evaluate/custom-code#score-traces).
Braintrust [best
practices](https://www.braintrust.dev/docs/evaluate/score-online#best-practices)
advise:
> Use span scope for evaluating individual operations or outputs. Use
trace scope for evaluating multi-turn conversations, overall workflow
completion, or when your scorer needs access to the full execution
context.
We've also received [direct
guidance](https://supabase.slack.com/archives/C05QYJBLX89/p1777925770927149?thread_ts=1777905716.911979&cid=C05QYJBLX89)
from their team to use this approach.
## Changes
Migrates eval scorers from custom `AssistantEvalOutput` shape to
trace-level scoring via `trace.getThread()` / `trace.getSpans()`, with
thread parsing that scores the full latest Assistant turn and passes
prior conversation separately where relevant.
Moves `execute_sql` and `deploy_edge_function` from client-side
execution after approval to AI SDK `needsApproval` + server-side
`execute()`. SQL results returned to the model are gated by AI opt-in
level, so row data is only included with `schema_and_log_and_data`;
otherwise the tool returns the no-data-permissions sentinel.
Adds `metadata.isFinalStep` to disambiguate multiple LLM requests within
an "assistant" turn due to tool call requests/responses. For online
evals, this means we should configure automations to only score traces
with `metadata.isFinalStep = true` to ensure we're judging the complete
generated response.
Other minor kaizen changes:
- Renamed `promptProviderOptions` to `systemProviderOptions` to clarify
that this is associated with the "system" message and disambiguate from
the root `providerOptions`
- Adds `evals/trace-utils.ts` to handle Zod validation of the `unknown`
span shapes from Braintrust, to more easily access typed inputs/output
on tool spans.
- Bumps AI SDK floor version `^6.0.116` → `^6.0.174`
- Tweaked the "Conciseness" scorer to not unfairly dock points for the
new `[called tool_name]` labels in serialized assistant response
## Verification
In the studio staging build, I asked Assistant to create a todos table
with 3 sample todos. I manually approved the `execute_sql` call and saw
Assistant generate text before & after the call.
In Braintrust I verified two traces were produced (see [filtered
logs](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs?v=Staging&tvt=trace&search={%22filter%22:[{%22text%22:%22metadata.environment%2520%253D%2520%27staging%27%22,%22label%22:%22metadata.environment%2520%253D%2520%27staging%27%22,%22originType%22:%22btql%22},{%22text%22:%22%2560Chat%2520ID%2560%2520%253D%2520%25221cb2ac45-e5e7-458c-9da4-3bf6863b8842%2522%22,%22label%22:%22Chat%2520ID%2520equals%25201cb2ac45-e5e7-458c-9da4-3bf6863b8842%22,%22originType%22:%22form%22}]})),
the first with `metadata.isFinalStep = false` and the second with
`metadata.isFinalStep = true`.
In the Braintrust staging scorers, I ran the preview Completeness scorer
on the second trace and verified it sees the complete Assistant response
including markers for tool calls ([link to
trace](https://www.braintrust.dev/app/supabase.io/p/Assistant%20(Staging%20Scorers)/trace?object_type=project_logs&object_id=b5214b62-ad1e-4929-9d5b-40b1daebe948&r=0ed0a4f8-8aff-4a34-bb1d-1df1d88a5070&s=ff9015f8-6bf7-4ab3-83a9-ca4e69e27e82))
<img width="1193" height="960" alt="CleanShot 2026-05-07 at 11 27 10@2x"
src="https://github.com/user-attachments/assets/509d4858-c3a1-4068-986d-3aa4d5617d1a"
/>
I also tested the `deploy_edge_function` workflow and verified it still
prompts for permission and warns on deployment of existing functions.
**References**
- https://www.braintrust.dev/docs/evaluate/custom-code#score-traces
-
https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#tool-execution-approval
Supercedes https://github.com/supabase/supabase/pull/45556 and
https://github.com/supabase/supabase/pull/45339
Closes AI-473
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Tool actions (SQL execution, edge-function deploy) now require
explicit user Approve/Deny before proceeding.
* **Improvements**
* Assistant pauses for approval responses before sending follow-ups,
giving clearer control over risky actions.
* Deploy/replace flows show confirmation and clearer replace warnings.
* Evaluation/scoring updated to use richer trace data for more accurate
assistant performance signals.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What kind of change does this PR introduce?
Feature
## What is the current behavior?
Functions page on self-hosted differs from Platform
## What is the new behavior?
> [!NOTE]
> This PR only add readonly operations. Function edit and deploy should
be implemented in a future one.
Adds the possibility to download and see function code in Self-Host
version.
<details>
<img width="1465" height="944" alt="image"
src="https://github.com/user-attachments/assets/4bbf8f5c-3390-4de6-9e8b-8ec9cd59ebad"
/>
</details>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* API endpoint to stream function files/artifacts as
multipart/form-data.
* New function file entry type and server-side file listing for
functions.
* **Improvements**
* Edge Functions "Code" navigation item always visible.
* Download popover reworked: ZIP download always available; CLI section
shown only on supported platforms.
* Editor set to read-only and file actions disabled on unsupported
environments.
* **Editor**
* Added JavaScript, TypeScript, and Markdown language modules for the
embedded editor.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Functions page on self-hosted differs from Platform Adds the possibility to see function details page in Self-Host version.
## Summary by CodeRabbit
* **New Features**
* Edge function detail lookup added for self‑hosted deployments (new
retrieval endpoint & store method).
* Consistent navigation to function pages from the functions list.
* **Improvements**
* UI tabs, download, and test controls adapt to deployment type.
* Region, JWT verification, local development, and delete controls shown
only on the platform.
* Edit/save/delete controls enable/disable correctly based on deployment
and permissions.
* Function details load reliably across deployments.
## What is the current behaviour?
Functions page on self-hosted differs from Platform
## What is the new behaviour?
Adds the possibility to see a list of edge-functions in Self-Host
version.
## Summary by CodeRabbit
* **New Features**
* Added self-hosted edge functions support with filesystem-based
artifact listing and a new API endpoint to list local functions.
* Improved functions UI: unified view, enhanced search and adjacent sort
controls, and clearer local-hosting guidance.
* **Chores**
* Added environment configuration and docker volume for edge functions
management.
* Updated build/task config to pass through edge functions env and
include contentlayer outputs.
Co-authored-by: Ali Waseem <waseema393@gmail.com>
Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Co-authored-by: Terry Sutton <saltcod@gmail.com>
* fix: correctly set entrypoint, import map path and static patterns for new functions
* Smol refactor
* Use TimestampInfo for last updated column in edge functions
---------
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
* Add custom types for queries, mutations and infinite queries.
* Migrate all queries to use the new type.
* Migrate all infinite queries to useCustomInfiniteQueryOptions.
* Migrate all mutations to use useCustomMutationOptions.
* Add type to all imports in `types` folder.
* feat: use mgmt-api's function body endpoint
* skip json files from static patterns
* set the default content for deno.json
* feat: add drag and drop file support to FileExplorerAndEditor
- Add drag and drop functionality to accept files
- Dropped files are automatically read and added to the files list
- Visual feedback with drag overlay during drag operations
- Files maintain existing data format with id, name, content, and selected state
- Last dropped file is automatically selected for editing
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: add binary file handling to FileExplorerAndEditor
- Add binary file detection for common file extensions (.wasm, images, executables, etc.)
- Show "Cannot Edit Selected File" error message when trying to edit binary files
- Binary files dropped via drag-and-drop retain their original binary content
- Only show error in editor view, files remain accessible in file list
- Text files continue to work normally with full editing capabilities
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* do not ignore empty files
* exclude wasm files from static patterns
* Remove eszip parser test as dependency has been removed
* Fix TS issues
* Fix TS issues
* Fix pnpm-lock
* Fix
* Fix
* Nit
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
* Migrate all uses of invalidateQueries to use object syntax.
* Migrate the remainder of useInfiniteQuery.
* Migrate all setQueriesData.
* Migrate all fetchQuery uses.
* Migrate some leftover functions from RQ.
* Fix issues found by Charis.
* fix(edge-functions): parse eszip made from edge-runtime v1.69.x correctly
* stamp: update test
* stamp: format
* stamp: meow
* stamp: meowmeow
* stamp: format
* Add custom error handler for fetch error 'Failed to fetch'
* Refactor EdgeFunctionTesterSheet to use react query instead of manually calling fetch
* Use new fetchHandler for where we're calling fetch
* Revert
* deploy edge function mutaton
* update url
* use openapi-fetch POST in the deploy mutation.
---------
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
* Update the design of the sonner toasts. Add the close button by default.
* Migrate studio and www apps to use the SonnerToaster.
* Migrate all toasts from studio.
* Migrate all leftover toasts in studio.
* Add a new toast component with progress. Use it in studio.
* Migrate the design-system app.
* Refactor the consent toast to use sonner.
* Switch docs to use the new sonner toasts.
* Remove toast examples from the design-system app.
* Remove all toast-related components and old code.
* Fix the progress bar in the toast progress component. Also make the bottom components vertically centered.
* Fix the width of the toast progress.
* Use text-foreground-lighter instead of muted for ToastProgress text
* Rename ToastProgress to SonnerProgress.
* Shorten the text in sonner progress.
* Use the correct classes for the close button. Add a const var for the default toast duration. Remove the custom width class from sonner.
* Set the position for all progress toasts to bottom right. Set the duration for all toasts to the default (when reusing a toast id from loading/progress toast, the duration is set to infinity).
* Fix the playwright tests.
* Refactor imports to use ui instead of @ui.
* Change all imports of react-hot-toast with sonner. These components were merged since the last commit to this branch.
* Remove react-hot-toast lib.
---------
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: Jonathan Summers-Muir <MildTomato@users.noreply.github.com>
* Move all studio files from /studio to /apps/studio.
* Move studio specific prettier ignores.
* Fix the ui references from studio.
* Fix the css imports.
* Fix all package.json issues.
* Fix the prettier setup for the studio app.
* Add .turbo folder to prettierignore.
* Fix the github workflows.
Matt Rossman