mirror of
https://github.com/supabase/supabase.git
synced 2026-10-05 09:25:06 +03:00
Pedro RodriguesandClaude Sonnet 5
22d7bc0cfd
- Eval harness's only live tool, `search_docs`, no longer needs the in-process MCP client or its dummy token — it now calls the public docs GraphQL API (`https://supabase.com/docs/api/graphql`) directly. Low risk as this is an eval-harness change only. Production assistant path (`mcp-tools.ts`) untouched. **Update:** per [@mattrossman's review](https://github.com/supabase/supabase/pull/50092#discussion_r3980396341), the eval tool's description embeds the Content API's own GraphQL schema (fetched via a `{ schema }` query and minified with `gqlmin`), mirroring how `@supabase/mcp-server-supabase`'s `docs-tools.ts`/`loadSchema` populates production's `search_docs` description. Without it, the model had no schema to work from and issued malformed queries, which caused the 218 `search_docs` errors and the -25pp Docs Faithfulness regression in the first eval run on this PR. Schema loading is required: `createSearchDocsTool()` rejects if the schema fetch fails, so preflight and the gated eval job fail loudly instead of producing untrustworthy fallback results. `createSearchDocsTool` is async because the `ai` package's `tool()` only accepts a plain string `description`, unlike the MCP SDK's async description support; both callers (`getMockTools`, `evals/preflight.ts`) await it. `gqlmin` is a direct `apps/studio` dependency and was already transitive via `@supabase/mcp-server-supabase`. ### Verification - `pnpm -C apps/studio exec -- tsc --noEmit` reaches the compiler; it reports only the pre-existing unrelated `packages/ui-patterns/src/McpUrlBuilder/components/InstructionBlocks.tsx` `StaticImageData` error. - `pnpm -C apps/studio exec -- vitest run lib/ai/tools/mock-tools.test.ts lib/ai/tools/mcp-tools.test.ts` — 21/21 passed. - `pnpm exec tsx evals/preflight.ts` — live docs API schema fetch and search_docs call passed. - `NEXT_PUBLIC_CONTENT_API_URL=http://127.0.0.1:1/graphql pnpm -C apps/studio exec -- tsx evals/preflight.ts` — failed fast as expected, proving schema/API failures gate evals. - Fresh `run-evals` pass: Docs Faithfulness 55.7% (0pp), with no systemic `search_docs` regression. Risk: eval-harness-only; schema/API outage now fails the eval job before scoring rather than allowing fallback descriptions. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added documentation search powered by the public Supabase documentation GraphQL API. * Documentation search results now include live schema information and clearer error handling for failed or invalid requests. * **Bug Fixes** * Improved evaluation tooling reliability by removing unnecessary connection-abort behavior. * Updated validation to detect missing search tools and malformed documentation responses. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Supabase Studio
A dashboard for managing your self-hosted Supabase project, and used on our hosted platform. Built with:
What's included
Studio is designed to work with existing deployments - either the local hosted, docker setup, or our CLI. It is not intended for managing the deployment and administration of projects - that's out of scope.
As such, the features exposed on Studio for existing deployments are limited to those which manage your database:
- Table & SQL editors
- Saved queries are unavailable
- Database management
- Policies, roles, extensions, replication
- API documentation
Managing Project Settings
Project settings are managed outside of the Dashboard. If you use docker compose, you should manage the settings in your docker-compose file. If you're deploying Supabase to your own cloud, you should store your secrets and env vars in a vault or secrets manager.
How to contribute?
- Branch from
masterand name your branches with the following structure{type}/{branch_name}- Type:
chore | fix | feature - The branch name is arbitrary — just make sure it summarizes the work.
- Type:
- When you send a PR to
master, it will automatically tag members of the frontend team for review. - Review the main contributing guide to help test your feature before sending a PR.
- The Dashboard is under active development. You should run
git pullfrequently to make sure you're up to date.
Developer Quickstart
Note
Supabase internal use: To develop on Studio locally with the backend services, see the instructions in the internal
infrastructurerepo.
# You'll need to be on Node v22
# in /studio
## For external contributors
pnpm install # install dependencies
pnpm run dev # start dev server
## For internal contributors
## First clone the private supabase/platform repo and follow instructions for setting up mise
mise studio # Run from supabase/platform alongside `mise infra`
## For all
pnpm run test # run tests
pnpm run test -- --watch # run tests in watch mode
Running within a self-hosted environment
Follow the self-hosting guide to get started.
cd ..
cd docker
docker compose -f docker-compose.yml -f ./dev/docker-compose.dev.yml up
Once you've got that set up, update .env in the studio folder with the corresponding values.
POSTGRES_PASSWORD=
SUPABASE_ANON_KEY=
SUPABASE_SERVICE_KEY=
Then run the following commands to install dependencies and start the dashboard.
npm install
npm run dev
If you would like to configure different defaults for "Default Organization" and "Default Project", you will need to update the .env in the studio folder with the corresponding values.
DEFAULT_ORGANIZATION_NAME=
DEFAULT_PROJECT_NAME=