supabase

38,771 Commits 1,784 Branches 62 Tags

2 Commits

Include renames

Author	SHA1	Message	Date
Pedro RodriguesandClaude Sonnet 5	22d7bc0cfd	feat(studio-evals): custom search_docs tool for the eval harness (no token / no PAT) (#50092 ) - Eval harness's only live tool, `search_docs`, no longer needs the in-process MCP client or its dummy token — it now calls the public docs GraphQL API (`https://supabase.com/docs/api/graphql`) directly. Low risk as this is an eval-harness change only. Production assistant path (`mcp-tools.ts`) untouched. Update: per [@mattrossman's review](https://github.com/supabase/supabase/pull/50092#discussion_r3980396341), the eval tool's description embeds the Content API's own GraphQL schema (fetched via a `{ schema }` query and minified with `gqlmin`), mirroring how `@supabase/mcp-server-supabase`'s `docs-tools.ts`/`loadSchema` populates production's `search_docs` description. Without it, the model had no schema to work from and issued malformed queries, which caused the 218 `search_docs` errors and the -25pp Docs Faithfulness regression in the first eval run on this PR. Schema loading is required: `createSearchDocsTool()` rejects if the schema fetch fails, so preflight and the gated eval job fail loudly instead of producing untrustworthy fallback results. `createSearchDocsTool` is async because the `ai` package's `tool()` only accepts a plain string `description`, unlike the MCP SDK's async description support; both callers (`getMockTools`, `evals/preflight.ts`) await it. `gqlmin` is a direct `apps/studio` dependency and was already transitive via `@supabase/mcp-server-supabase`. ### Verification - `pnpm -C apps/studio exec -- tsc --noEmit` reaches the compiler; it reports only the pre-existing unrelated `packages/ui-patterns/src/McpUrlBuilder/components/InstructionBlocks.tsx` `StaticImageData` error. - `pnpm -C apps/studio exec -- vitest run lib/ai/tools/mock-tools.test.ts lib/ai/tools/mcp-tools.test.ts` — 21/21 passed. - `pnpm exec tsx evals/preflight.ts` — live docs API schema fetch and search_docs call passed. - `NEXT_PUBLIC_CONTENT_API_URL=http://127.0.0.1:1/graphql pnpm -C apps/studio exec -- tsx evals/preflight.ts` — failed fast as expected, proving schema/API failures gate evals. - Fresh `run-evals` pass: Docs Faithfulness 55.7% (0pp), with no systemic `search_docs` regression. Risk: eval-harness-only; schema/API outage now fails the eval job before scoring rather than allowing fallback descriptions. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * New Features * Added documentation search powered by the public Supabase documentation GraphQL API. * Documentation search results now include live schema information and clearer error handling for failed or invalid requests. * Bug Fixes * Improved evaluation tooling reliability by removing unnecessary connection-abort behavior. * Updated validation to detect missing search tools and malformed documentation responses. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-14 12:51:14 +02:00
Pedro RodriguesandClaude Opus 4.8	c4c213ce3d	feat(studio): switch dashboard assistant to remote MCP server (#47479 ) ## I have read the [CONTRIBUTING.md](<https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md>) file. YES ## What kind of change does this PR introduce? Feature / refactor. ## What is the current behavior? The dashboard assistant runs `@supabase/mcp-server-supabase` in-process over an in-memory transport (`lib/ai/supabase-mcp.ts`). ## What is the new behavior? The assistant connects to the remote MCP server over HTTP (`@ai-sdk/mcp`), forwarding the dashboard session token as a bearer. URL comes from `NEXT_PUBLIC_MCP_URL` with a local-dev fallback; platform-only, and Nimbus works via the same env var. * Tool model unchanged: UI-controlled `execute_sql` (with `needsApproval`) and `deploy_edge_function` still come from Studio; the allowlist (`TOOL_CATEGORY_MAP`) remains the gate keeping the remote's write tools away from the assistant (`read_only` is defense-in-depth). * Attribution: sends `x-source-name: supabase-studio` (+ `x-source-version`) → logged as `source_name`/`client_name`. * Connection lifecycle: the HTTP client is closed via the request's `AbortSignal` (tools execute later during streaming); `signal` is required on `getTools`/`getMcpTools`. * Resilience: a remote-MCP failure degrades to the remaining tools instead of failing the assistant. * Drift protection: relied-upon tools are typed against `keyof typeof supabaseMcpToolSchemas`, so a package bump that renames/removes one fails `pnpm typecheck`; a runtime check also warns if the deployed server returns fewer tools. * Adds unit tests for the above. ## Additional context * Verified end-to-end against a local remote MCP server with a dashboard token: `initialize` 200, tools listed, a tool executed, client closed cleanly. * The remote MCP (mgmt-api) already accepts dashboard session tokens (GoTrue-JWT auth path) — no backend change needed. `NEXT_PUBLIC_MCP_URL` must point at each env's `/mcp`. * `@supabase/mcp-server-supabase` is kept — still used by the self-hosted `/api/mcp` routes. Closes [AI-137](https://linear.app/supabase/issue/AI-137/switch-dashboard-assistant-to-remote-mcp) ## Rollout * Rollout: merges with `USE_REMOTE_MCP` off (in-process); flip it to `true` per environment (staging → prod → Nimbus) once each one's prerequisites land. * Rollback: unset `USE_REMOTE_MCP` and redeploy to fall back to the in-process client — no revert needed. ## Summary by CodeRabbit * Bug Fixes * Improved AI request handling so tool loading and generation clean up properly when a request is cancelled or the browser connection closes. * Added safer fallback behavior when remote tool loading fails, so AI features can continue with available tools instead of stopping entirely. * Updated remote tool access to use the current project reference and preserve the correct access headers. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * New Features * AI tools now connect more reliably to remote services and stop cleanly when requests end or are canceled. * Tool loading is more resilient, continuing with available tools if remote access is unavailable. * Bug Fixes * Improved cleanup to prevent lingering connections during SQL generation and policy workflows. * Added safer handling for remote tool changes and invalid responses. * Tests * Expanded automated coverage for remote tool setup, cancellation, and fallback behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-07 19:38:21 +01:00

Author

SHA1

Message

Date

Pedro RodriguesandClaude Sonnet 5

22d7bc0cfd

feat(studio-evals): custom search_docs tool for the eval harness (no token / no PAT) (#50092 )

- Eval harness's only live tool, `search_docs`, no longer needs the
in-process MCP client or its dummy token — it now calls the public docs
GraphQL API (`https://supabase.com/docs/api/graphql`) directly. Low risk
as this is an eval-harness change only. Production assistant path
(`mcp-tools.ts`) untouched.

**Update:** per [@mattrossman's
review](https://github.com/supabase/supabase/pull/50092#discussion_r3980396341),
the eval tool's description embeds the Content API's own GraphQL schema
(fetched via a `{ schema }` query and minified with `gqlmin`), mirroring
how `@supabase/mcp-server-supabase`'s `docs-tools.ts`/`loadSchema`
populates production's `search_docs` description. Without it, the model
had no schema to work from and issued malformed queries, which caused
the 218 `search_docs` errors and the -25pp Docs Faithfulness regression
in the first eval run on this PR. Schema loading is required:
`createSearchDocsTool()` rejects if the schema fetch fails, so preflight
and the gated eval job fail loudly instead of producing untrustworthy
fallback results. `createSearchDocsTool` is async because the `ai`
package's `tool()` only accepts a plain string `description`, unlike the
MCP SDK's async description support; both callers (`getMockTools`,
`evals/preflight.ts`) await it. `gqlmin` is a direct `apps/studio`
dependency and was already transitive via
`@supabase/mcp-server-supabase`.

### Verification
- `pnpm -C apps/studio exec -- tsc --noEmit` reaches the compiler; it
reports only the pre-existing unrelated
`packages/ui-patterns/src/McpUrlBuilder/components/InstructionBlocks.tsx`
`StaticImageData` error.
- `pnpm -C apps/studio exec -- vitest run
lib/ai/tools/mock-tools.test.ts lib/ai/tools/mcp-tools.test.ts` — 21/21
passed.
- `pnpm exec tsx evals/preflight.ts` — live docs API schema fetch and
search_docs call passed.
- `NEXT_PUBLIC_CONTENT_API_URL=http://127.0.0.1:1/graphql pnpm -C
apps/studio exec -- tsx evals/preflight.ts` — failed fast as expected,
proving schema/API failures gate evals.
- Fresh `run-evals` pass: Docs Faithfulness 55.7% (0pp), with no
systemic `search_docs` regression.

Risk: eval-harness-only; schema/API outage now fails the eval job before
scoring rather than allowing fallback descriptions.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added documentation search powered by the public Supabase
documentation GraphQL API.
* Documentation search results now include live schema information and
clearer error handling for failed or invalid requests.

* **Bug Fixes**
* Improved evaluation tooling reliability by removing unnecessary
connection-abort behavior.
* Updated validation to detect missing search tools and malformed
documentation responses.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

2026-09-14 12:51:14 +02:00

Pedro RodriguesandClaude Opus 4.8

c4c213ce3d

feat(studio): switch dashboard assistant to remote MCP server (#47479 )

## I have read the
[CONTRIBUTING.md](<https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md>)
file.

YES

## What kind of change does this PR introduce?

Feature / refactor.

## What is the current behavior?

The dashboard assistant runs `@supabase/mcp-server-supabase` in-process
over an in-memory transport (`lib/ai/supabase-mcp.ts`).

## What is the new behavior?

The assistant connects to the **remote MCP server** over HTTP
(`@ai-sdk/mcp`), forwarding the dashboard session token as a bearer. URL
comes from `NEXT_PUBLIC_MCP_URL` with a local-dev fallback;
platform-only, and Nimbus works via the same env var.

* **Tool model unchanged:** UI-controlled `execute_sql` (with
`needsApproval`) and `deploy_edge_function` still come from Studio; the
allowlist (`TOOL_CATEGORY_MAP`) remains the gate keeping the remote's
write tools away from the assistant (`read_only` is defense-in-depth).
* **Attribution:** sends `x-source-name: supabase-studio` (+
`x-source-version`) → logged as `source_name`/`client_name`.
* **Connection lifecycle:** the HTTP client is closed via the request's
`AbortSignal` (tools execute later during streaming); `signal` is
required on `getTools`/`getMcpTools`.
* **Resilience:** a remote-MCP failure degrades to the remaining tools
instead of failing the assistant.
* **Drift protection:** relied-upon tools are typed against `keyof
typeof supabaseMcpToolSchemas`, so a package bump that renames/removes
one fails `pnpm typecheck`; a runtime check also warns if the deployed
server returns fewer tools.
* Adds unit tests for the above.

## Additional context

* Verified end-to-end against a local remote MCP server with a dashboard
token: `initialize` 200, tools listed, a tool executed, client closed
cleanly.
* The remote MCP (mgmt-api) already accepts dashboard session tokens
(GoTrue-JWT auth path) — no backend change needed. `NEXT_PUBLIC_MCP_URL`
must point at each env's `/mcp`.
* `@supabase/mcp-server-supabase` is kept — still used by the
self-hosted `/api/mcp` routes.

Closes
[AI-137](https://linear.app/supabase/issue/AI-137/switch-dashboard-assistant-to-remote-mcp)

## Rollout

* **Rollout:** merges with `USE_REMOTE_MCP` off (in-process); flip it to
`true` per environment (staging → prod → Nimbus) once each one's
prerequisites land.
* **Rollback:** unset `USE_REMOTE_MCP` and redeploy to fall back to the
in-process client — no revert needed.

## Summary by CodeRabbit

* **Bug Fixes**
* Improved AI request handling so tool loading and generation clean up
properly when a request is cancelled or the browser connection closes.
* Added safer fallback behavior when remote tool loading fails, so AI
features can continue with available tools instead of stopping entirely.
* Updated remote tool access to use the current project reference and
preserve the correct access headers.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* AI tools now connect more reliably to remote services and stop cleanly
when requests end or are canceled.
* Tool loading is more resilient, continuing with available tools if
remote access is unavailable.

* **Bug Fixes**
* Improved cleanup to prevent lingering connections during SQL
generation and policy workflows.
  * Added safer handling for remote tool changes and invalid responses.

* **Tests**
* Expanded automated coverage for remote tool setup, cancellation, and
fallback behavior.


<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-07 19:38:21 +01:00