mirror of
https://github.com/supabase/supabase.git
synced 2026-10-06 09:55:06 +03:00
codex/fix-tanstack-e2e
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
bd76d7fc34 |
feat(studio): wrap assistant Edge Function approval in a Confirm card (#49168)
<img width="1512" height="862" alt="image" src="https://github.com/user-attachments/assets/79a6d4dc-dcd2-489f-97d7-3ee7a0196b7d" /> ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Feature / UI refactor. ## What is the current behavior? Assistant Edge Function approval nests `ConfirmFooter` under the function block. `addToolApprovalResponse` is wired whenever state is `approval-requested`, including automatic approvals. ## What is the new behavior? Introduces a `Confirm` card that owns the frame, with the footer attached below the body. Edge Function approval uses that card. Interactive Approve/Deny only runs for manual `approval-requested` parts (`!approval.isAutomatic`), matching the [AI SDK tool-approvals `useChat` guidelines](https://ai-sdk.dev/docs/agents/tool-approvals). SQL still uses `DisplayBlockRenderer` until #49170. `ConfirmFooter` is inlined into `Confirm` so SQL can keep importing the named footer until that PR. ## Additional context Part of stack #49171. Base: `chore/ai-sdk-7` (#49167). Notebook proposal Confirm wrapping is **not** in this stack — that file lives on [#49159](https://github.com/supabase/supabase/pull/49159). Follow up after that stack merges. ## Test plan - [ ] Deploy-edge-function tool part shows Confirm with Skip / Deploy - [ ] Existing-function replace warning still requires the second confirm - [ ] After approve, footer morphs to loading and buttons disable - [ ] `Confirm.utils.test.ts` and `EdgeFunctionRenderer.test.tsx` pass <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added confirmation cards for AI-assisted actions, including approve and cancel controls. * Improved handling of manual approval requests for SQL execution, notebook changes, and Edge Function deployment. * Added support for customizing report and Edge Function block styling. * **Bug Fixes** * Automatic approvals no longer appear as pending manual confirmations. * Skipped SQL actions now provide clearer messaging. * **Tests** * Expanded coverage for approval states, confirmation controls, and automatic decisions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Danny White
|
6f6badae51 |
fix(eslint): promote require-explicit-tabindex to error (#48170)
## What kind of change does this PR introduce? Accessibility / lint hardening (Safari keyboard focus). ## What is the current behavior? `supabase/require-explicit-tabindex` is `'warn'`. Studio’s ratchet was at 0 but the rule was still ratcheted; www / docs / design-system still had raw `<button>` / `role="button"` call sites without an explicit `tabIndex`. [DEPR-627](https://linear.app/supabase/issue/DEPR-627) · follow-up to #47984 / #48040 ## What is the new behavior? - Shared config: `'supabase/require-explicit-tabindex': 'error'` - Swept www / docs / design-system (+ Studio test fixtures the ratchet skipped) - Removed the rule from the Studio ratchet + baselines ## To test Prefer **Safari**. This PR only adds explicit `tabIndex` to raw `<button>` / `role="button"` call sites — not links, and not controls that already go through `Button` from `ui`. ### Marketing (`www`) ([staging link](https://zone-www-dot-com-git-danny-depr-627-promote-req-7ae43c-supabase.vercel.app/)) - [x] Homepage frameworks / dashboard feature tabs — Tab through each tab button - [x] Product pages (e.g. `/auth`, `/database`) — section tab switchers - [x] Narrow viewport — open the hamburger; Tab through menu buttons - [x] `/partners/catalog` — filter / view controls - [x] Blog view toggle (list ↔ grid) ### Docs ([staging link](https://docs-git-danny-depr-627-promote-require-explici-25e46d-supabase.vercel.app/)) - [x] **Desktop (≥ lg):** top-right **⋯ menu** (hamburger icon) — opens a dropdown that includes Theme. Not a separate theme button. - [x] **Mobile (< lg):** top-right **hamburger** opens the sheet; close (X) is the raw button we tagged. Theme inside the sheet uses `ThemeToggle` / `DropdownMenuTrigger` from `ui` (already supposed to set `tabIndex`). - [x] **Code blocks** — copy / language controls - [x] **Is this helpful?** — X / check are `Button` from `ui` (should already Tab). After voting **while signed in**, the follow-up “What went well?” / “How can we improve?” text button is the raw one we tagged. - [x] **AI Tools → Copy as Markdown** (right rail on a guide) — this is the only GuidesSidebar control this PR changed. “On this page” TOC items are **links**, not covered by this lint. - [x] **Reference docs** (e.g. JS client reference) — section headers that expand/collapse in the left nav (`Collapsible.Trigger`) - [x] **Troubleshooting index** — type in the search field, then Tab to the **clear (X)** control ### Dashboard (`studio`) No production UI changes in this PR (tests + lint config only). Quick Safari smoke that prior tabindex work still holds: - [x] Project sidebar — Tab through primary nav links - [x] Settings → General — Tab through inputs / buttons - [x] Storage → Files — Tab a bucket row / file actions |
||
|
Pamela Chia
|
47c084e51d |
refactor(studio): migrate telemetry to useTrack (#46140)
## Summary
I migrated every `useSendEventMutation` call site in `apps/studio` to
`useTrack`, deleted the legacy hook, and added a lint guardrail so it
can't return. `useTrack` is the type-safe replacement: it auto-injects
`groups: { project, organization }` from the selected project/org and
types `action` + `properties` against `TelemetryEvent`. Existing call
sites built groups manually and were not type-checked at the action
level. The migration covers 81 files (60 trivial swaps, 9 org-only, 3
pre-auth, 5 bespoke, 4 test mocks).
## Changes
- Migrated trivial call sites across `pages/project/[ref]`,
`components/interfaces/*` (Reports, Storage, Realtime/Inspector,
SQLEditor, Functions, EdgeFunctions, Integrations, ProjectAPIDocs,
Branching/BranchManagement, TableGridEditor, Connect, Docs, Auth,
Support, Home, ProjectHome, App), `components/layouts/*`, and
`components/ui/*`.
- Migrated org-only sites (`Organization/Documents/*`,
`Organization/BillingSettings/Subscription/*`,
`Organization/SecuritySettings.tsx`,
`Account/Preferences/DashboardSettingsToggles.tsx`) by dropping the
manual `groups: { organization: ... }` and letting `useTrack`
auto-inject. Verified `useSelectedProjectQuery` is disabled on org
routes (gates on URL `[ref]`).
- Migrated pre-auth sites (`SignInForm.tsx`, `sign-in-mfa.tsx`,
`profile.tsx`) where neither project nor org is resolved.
- Bespoke handling:
- `execute-sql-mutation.ts` and `table-row-create-mutation.ts`: pass `{
project: projectRef }` via `groupOverrides` since the mutation can
target a non-selected project ref.
- `useStudioCommandMenuTelemetry.ts`: kept a direct `sendTelemetryEvent`
call because studio groups must override pre-built event groups
(opposite of `useTrack`'s override direction).
- `AIAssistantOption.tsx`: passes sentinel-aware `groupOverrides` so
`NO_PROJECT_MARKER`/`NO_ORG_MARKER` continue to suppress group emission.
- `SidePanelEditor.utils.tsx`: utility functions `createTable` and
`updateTable` now take a `track: Track` parameter (threaded from
`SidePanelEditor.tsx`); dropped the `organizationSlug` arg since groups
are no longer assembled manually.
- Branch-event attribution: preserved `parentProjectRef` overrides on
`branch_updated`, `branch_merge_completed`, `branch_merge_failed`,
`branch_merge_submitted`, `branch_delete_button_clicked`,
`branch_review_with_assistant_clicked`, and
`branch_*_merge_request_button_clicked`. Original code grouped these
under the parent (production) project, not the branch ref;
auto-injection would have shifted them onto the branch.
- Switched 4 test mocks from `@/data/telemetry/send-event-mutation` to
`@/lib/telemetry/track`. Removed obsolete tests around manual groups and
`try/catch` on telemetry rejection.
- Deleted `apps/studio/data/telemetry/send-event-mutation.ts`. The
deleted module is its own guardrail: any reintroduction of the import
fails at TypeScript module resolution before lint runs.
## Testing
Tested on preview deploy:
- [x] SQL editor `CREATE TABLE` fires `table_created` with method
`sql_editor` and `groups.project` set to the mutation's `projectRef`.
- [x] Table editor creates a table from the side panel; `table_created`
fires from `SidePanelEditor.utils` via threaded `track`.
- [x] Help button (`/project/[ref]/...`) fires `help_button_clicked`
with auto-injected project + org groups.
- [x] Sign-in form fires `sign_in` with empty groups (pre-auth,
expected).
- [x] Org documents page (`/org/[slug]/documents`) fires
`document_view_button_clicked` with org group only, no stale project
ref.
- [x] Command menu (`Cmd+K`) inside a project still fires
`command_menu_opened` with studio's project/org overriding any
event-supplied groups.
- [x] Support form "Ask the Assistant" without selected org fires
`ai_assistant_in_support_form_clicked` with no project/org groups
(sentinels suppress).
- [x] On a branch, "Update branch" / "Merge branch" / "Close merge
request" events fire with `groups.project` set to the parent project
ref, not the branch ref.
Local checks:
- [x] 22/22 tests pass across the 4 updated test files
(`SidePanelEditor.utils.createTable`, `EdgeFunctionRenderer`,
`LayoutSidebar`, `PlanUpdateSidePanel`).
- [x] `rg useSendEventMutation apps/studio` returns 0 hits.
## Linear
- fixes GROWTH-860
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Standardized telemetry across the Studio to a unified tracking system;
events now send simplified payloads with less contextual/grouping data.
* No user-facing flows changed; UI behavior, permissions, and
interactions remain the same.
* **Tests**
* Updated telemetry mocks and tests to align with the new tracking
approach.
<!-- review_stack_entry_start -->
[](https://app.coderabbit.ai/change-stack/supabase/supabase/pull/46140?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack)
<!-- review_stack_entry_end -->
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Matt Rossman
|
d143571586 |
feat(assistant): trace-level scorers + server-side tool execution with needsApproval (#45654)
## Motivation When Assistant runs a potentially destructive tool like `execute_sql`, it stops the LLM request and prompts for client-side approval and execution of the tool. After approval, a second request kicks off under a separate trace. This has made scoring and [Topics](https://www.braintrust.dev/blog/topics) classification challenging, as the generated `output` is split across stateless requests. The [span-level scoring](https://www.braintrust.dev/docs/evaluate/custom-code#score-spans) approach we've used thusfar (after the LLM call, we massage the result into an `output` payload that's stuck onto the root span) has been cumbersome and led to invalid scores / topics where only part of the assistant response is considered. It's also inefficient, as we're duplicating potentially large info (like the `search_docs` output) that already exists within the trace. An alternative to scoring spans is to [score traces](https://www.braintrust.dev/docs/evaluate/custom-code#score-traces). Braintrust [best practices](https://www.braintrust.dev/docs/evaluate/score-online#best-practices) advise: > Use span scope for evaluating individual operations or outputs. Use trace scope for evaluating multi-turn conversations, overall workflow completion, or when your scorer needs access to the full execution context. We've also received [direct guidance](https://supabase.slack.com/archives/C05QYJBLX89/p1777925770927149?thread_ts=1777905716.911979&cid=C05QYJBLX89) from their team to use this approach. ## Changes Migrates eval scorers from custom `AssistantEvalOutput` shape to trace-level scoring via `trace.getThread()` / `trace.getSpans()`, with thread parsing that scores the full latest Assistant turn and passes prior conversation separately where relevant. Moves `execute_sql` and `deploy_edge_function` from client-side execution after approval to AI SDK `needsApproval` + server-side `execute()`. SQL results returned to the model are gated by AI opt-in level, so row data is only included with `schema_and_log_and_data`; otherwise the tool returns the no-data-permissions sentinel. Adds `metadata.isFinalStep` to disambiguate multiple LLM requests within an "assistant" turn due to tool call requests/responses. For online evals, this means we should configure automations to only score traces with `metadata.isFinalStep = true` to ensure we're judging the complete generated response. Other minor kaizen changes: - Renamed `promptProviderOptions` to `systemProviderOptions` to clarify that this is associated with the "system" message and disambiguate from the root `providerOptions` - Adds `evals/trace-utils.ts` to handle Zod validation of the `unknown` span shapes from Braintrust, to more easily access typed inputs/output on tool spans. - Bumps AI SDK floor version `^6.0.116` → `^6.0.174` - Tweaked the "Conciseness" scorer to not unfairly dock points for the new `[called tool_name]` labels in serialized assistant response ## Verification In the studio staging build, I asked Assistant to create a todos table with 3 sample todos. I manually approved the `execute_sql` call and saw Assistant generate text before & after the call. In Braintrust I verified two traces were produced (see [filtered logs](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs?v=Staging&tvt=trace&search={%22filter%22:[{%22text%22:%22metadata.environment%2520%253D%2520%27staging%27%22,%22label%22:%22metadata.environment%2520%253D%2520%27staging%27%22,%22originType%22:%22btql%22},{%22text%22:%22%2560Chat%2520ID%2560%2520%253D%2520%25221cb2ac45-e5e7-458c-9da4-3bf6863b8842%2522%22,%22label%22:%22Chat%2520ID%2520equals%25201cb2ac45-e5e7-458c-9da4-3bf6863b8842%22,%22originType%22:%22form%22}]})), the first with `metadata.isFinalStep = false` and the second with `metadata.isFinalStep = true`. In the Braintrust staging scorers, I ran the preview Completeness scorer on the second trace and verified it sees the complete Assistant response including markers for tool calls ([link to trace](https://www.braintrust.dev/app/supabase.io/p/Assistant%20(Staging%20Scorers)/trace?object_type=project_logs&object_id=b5214b62-ad1e-4929-9d5b-40b1daebe948&r=0ed0a4f8-8aff-4a34-bb1d-1df1d88a5070&s=ff9015f8-6bf7-4ab3-83a9-ca4e69e27e82)) <img width="1193" height="960" alt="CleanShot 2026-05-07 at 11 27 10@2x" src="https://github.com/user-attachments/assets/509d4858-c3a1-4068-986d-3aa4d5617d1a" /> I also tested the `deploy_edge_function` workflow and verified it still prompts for permission and warns on deployment of existing functions. **References** - https://www.braintrust.dev/docs/evaluate/custom-code#score-traces - https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling#tool-execution-approval Supercedes https://github.com/supabase/supabase/pull/45556 and https://github.com/supabase/supabase/pull/45339 Closes AI-473 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Tool actions (SQL execution, edge-function deploy) now require explicit user Approve/Deny before proceeding. * **Improvements** * Assistant pauses for approval responses before sending follow-ups, giving clearer control over risky actions. * Deploy/replace flows show confirmation and clearer replace warnings. * Evaluation/scoring updated to use richer trace data for more accurate assistant performance signals. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
