Commit Graph
14 Commits
Author SHA1 Message Date
Matt Rossman fd5ef806d6 feat(studio): enable Assistant tracing for High Compliance projects (#50759)
Assistant chats from High Compliance projects now flow to Braintrust
like any other project. The constraint that required suppressing them no
longer applies, see AI-1241 for the details.

`isTracingAllowed` now takes only the project region to maintain EU
exclusion. Traces also carry an `isHighComplianceProject` metadata
field, so the project's status at the time of the trace is recorded
rather than looked up later against a setting customers can toggle.

To verify, see [this sample
trace](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs?r=afabbdcc-aa89-446e-aa52-78aaa90d44a4&v=Production&s=afabbdcc-aa89-446e-aa52-78aaa90d44a4&tvt=trace)
from a High Compliance project on staging which indicates that tracing
is now enabled for these projects and that it carries metadata showing
the high compliance status.

| High Compliance project setting | `isHighComplianceProject` metadata |
|--------|--------|
| <img width="1554" height="454" alt="CleanShot 2026-09-22 at 5 14 58
PM@2x"
src="https://github.com/user-attachments/assets/23901c6e-0d79-44e8-a6dd-43cdedba1799"
/> | <img width="1674" height="990" alt="CleanShot 2026-09-22 at 5 17 40
PM@2x"
src="https://github.com/user-attachments/assets/fb2fba55-bcc1-4136-a432-b33a5c7f9ca2"
/> |

Closes AI-1241


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Changes**
* AI project compliance information is now represented by a unified
high-compliance project status.
* AI response tracing is now determined by project region: tracing
remains disabled for EU and unknown regions, while known non-EU regions
are eligible.
* AI feedback and SQL generation now use the updated compliance and
regional handling.
* **Tests**
* Updated coverage to reflect the revised compliance and tracing
behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-09-23 09:39:01 -04:00
Ali Waseem b04d14856a Resolve AI opt-in and tracing settings server-side (#48855)
The AI endpoints resolved organization and project settings
independently and applied them together without confirming they belonged
to the same pairing. Consolidates both into a single `getAIDetails` that
reconciles them and falls back to the most restrictive posture when
unconfirmed, and applies the HIPAA sensitivity gate to the opt-in level,
which previously only existed on the client.

Fixes FE-4110

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **New Features**
- Consolidated AI access details across organization and project
settings.
- AI functionality now validates project ownership and disables access
for mismatched or HIPAA-sensitive projects.
- AI responses include plan, region, opt-in status, sensitivity,
authorization, and advanced model access information.

- **Bug Fixes**
- Improved fail-closed behavior when project or organization data is
missing or inconsistent.
- Updated AI generation, feedback, rate, and policy flows to
consistently apply consolidated access settings.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-08-10 10:34:31 -06:00
Joshen Lim dc23320e43 Add sentry capture exception to apiWrapper (#47804)
## Context

As per PR title - also adjusts the imports for files consuming
`apiWrapper` to remove the default export for `apiWrapper`

Have tested locally by throwing an error in one of the API routes -
verified that the event shows up on Sentry

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* API errors are now captured in Sentry before returning server error
responses, improving production visibility while keeping endpoint
behavior the same.
* **Tests**
* Added coverage to confirm rejected handler executions are reported to
Sentry and return the expected HTTP 500 JSON payload.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-10 16:28:42 +08:00
Matt Rossman 36ae9beb0c chore(ai): remove DPA signer killswitch for assistant tracing (#45134)
Removes the temporary killswitch added when Braintrust was onboarded as
a subprocessor, to satisfy the 30-day DPA notice obligation. The window
has elapsed and legal has cleared removal.

Drops the `orgIsDpaSigned` check from `isTracingAllowed`, removes the
extra `/platform/organizations/{slug}/documents/dpa-signed` network hop
from `getOrgAIDetails`, and cleans up all call sites and tests.

Closes AI-596

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Simplified AI tracing eligibility logic by removing DPA signing status
checks. Tracing authorization decisions now depend solely on region,
HIPAA addon status, and project sensitivity settings.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-27 13:53:24 -04:00
Charis 205cbe7d26 chore(studio}: enforce import order, remove bare import specifiers (#44585) 2026-04-07 20:34:10 -04:00
Matt Rossman 8aeacc6152 feat(assistant): disable Braintrust tracing for EU regions and DPA signers (#44504)
**Changes**

- Extracted tracing conditional to an `isTracingAllowed` helper with
unit tests (the function is simple but sensitive hence the extra testing
precaution)
- Disables Braintrust tracing for projects in EU database regions
(region prefix `eu-`) to address GDPR data residency concerns
- Disables Braintrust tracing for orgs whose owners have signed the
previous DPA, as a stopgap during the 30-day notice period for the
updated DPA that adds Braintrust as a subprocessor
- Refactored `org-ai-details.ts` → `ai-details.ts`, splitting
`getOrgAIDetails` into separate org and project helpers to cleanly scope
the EU-region check at the project level

DPA check uses the newly added `/documents/dpa-signed` endpoint from
https://github.com/supabase/platform/pull/31060. This PR includes
regenerated `api.d.ts` and `platform.d.ts` from running `pnpm codegen`
in `packages/api-types` to get type safety on this new endpoint.

Note tracing is still yet to be activated in production, this is a
preparatory step.

**To verify**

Send a chat message and check for the `x-braintrust-span-id` response
header on `POST /api/ai/sql/generate-v4` — it should be absent for
DPA-signed orgs or EU-region projects, and present otherwise.

<img width="3594" height="1992" alt="CleanShot 2026-04-03 at 14 28
58@2x"
src="https://github.com/user-attachments/assets/4c91d7ad-2604-4531-a78e-dedf41632fa5"
/>

If you have access to the Braintrust dashboard, you can also verify
whether logs are produced or not in the Assistant project there.


Closes AI-570
Closes AI-569

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Tracks organization DPA signing and detects EU-region projects
* Assistant tracing now follows a combined compliance policy (HIPAA
addon, DPA, project sensitivity, region)
  * Added helpers to fetch org and project AI details

* **Documentation**
* Expanded API docs with additional examples and clarified parameter
descriptions
  * Added response schemas for subscription preview and document status

* **Tests**
* Added/updated tests covering DPA/region behavior and tracing policy
enforcement
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-07 13:09:34 -04:00
Matt Rossman adf8b0c67c feat(assistant): per-endpoint reasoningEffort + model config cleanup (#43981)
We're exploring support for newer models like
[gpt-5.4-nano](https://openai.com/index/introducing-gpt-5-4-mini-and-nano/)
in Assistant. This model doesn't support the `'minimal'` reasoning
effort level we use for gpt-5-mini which leads to vague errors.

<img width="595" height="263" alt="CleanShot 2026-03-18 at 17 13 05@2x"
src="https://github.com/user-attachments/assets/cf7c2370-322d-4a8a-be55-23e680db0aa0"
/>


Also, we've [previously
discussed](https://supabase.slack.com/archives/C0161K73J1J/p1771544464850199?thread_ts=1771493920.775699&cid=C0161K73J1J)
that reasoning adds unnecessary latency to otherwise simple AI
completion endpoints like `title-v2`. We want more control of reasoning
level independent of model/endpoint.

This PR aims to solve both problems by:
- making reasoning effort configurable on a per-request basis
- adding compile-time guardrails to prevent selecting an incompatible
reasoning level for models
- adding a `DEFAULT_COMPLETION_MODEL` with minimal reasoning that we can
update with newer models that support disabling reasoning (independent
of Assistant chat model reasoning)

Other improvements to our model config logic:
- Fixes bug in `onboarding/design.ts` and `assistant.eval.ts` where
`providerOptions` was being dropped
- `getModel()` now returns a bundled `modelParams` object (spread into
AI SDK calls) so `providerOptions` can't be accidentally omitted (this
[has happened
before](https://supabase.slack.com/archives/C0161K73J1J/p1771518443534309?thread_ts=1771493920.775699&cid=C0161K73J1J))
- Introduces an `ASSISTANT_MODELS` registry as a single source of truth
for assistant model config, eliminating hardcoded model IDs across the
codebase
- Aligns free/pro model conditional logic with `assistant.advance_model`
entitlement naming conventions instead of the `isLimited` pattern
- Adds `console.error` logging of Assistant stream errors so we can
interpret reasoning effort compatibility errors in the future (instead
of just opaque "Sorry, I'm having trouble responding right now" card)
- Removes unnecessary type casts and generally making the model config
logic stricter
- Removes pre-existing dead code: `anthropic` provider variant in
`GetModelParams` / `PROVIDERS` registry that was never implemented in
`getModel()`

Now if you try to select an unsupported reasoning level you get a type
error:

<img width="1306" height="320" alt="CleanShot 2026-03-20 at 14 37 24@2x"
src="https://github.com/user-attachments/assets/a6ac234b-5ea5-4d81-8e01-ac4be34a0800"
/>

And if for some reason an invalid reasoning level slips through, you now
get a server-side error surfacing the issue:

<img width="1268" height="204" alt="CleanShot 2026-03-20 at 14 58 14@2x"
src="https://github.com/user-attachments/assets/aadc1b7a-9495-475f-9741-39979bd27cd7"
/>

I've tested gpt-5 and gpt-5-mini are still working on the staging
preview and verified the models were selected properly in Braintrust
logs. Both models are available on my Pro test account, and my Free test
account shows the Pro upgrade CTA.


Closes AI-446
Closes AI-551
2026-03-25 11:29:23 -04:00
Matt Rossman d29fbf6eb7 feat(assistant): upgrade AI SDK v5 → v6 (#43931)
Upgrades `ai` from v5 to v6 and all related packages.

**Package bumps:**
- `ai`: `5.0.52` → `^6.0.116`
- `@ai-sdk/openai`: `2.0.32` → `^3.0.41`
- `@ai-sdk/react`: `2.0.52` → `^3.0.118`
- `@ai-sdk/provider`: `^2.0.0` → `^3.0.8`
- `@ai-sdk/provider-utils`: `^3.0.0` → `^4.0.19`
- `@ai-sdk/amazon-bedrock`: `^3.0.0` → `^4.0.81`
- `@ai-sdk/mcp`: N/A → `^1.0.25`
- `openai`: bumped to `^4.104.0`
- `braintrust`: `3.0.x` → `^3.4.0`

**Breaking change migrations:**
- `generateObject` removed in v6 — migrated 5 API routes to
`generateText` with `Output.object({ schema })`, returning
`result.output`
- `convertToModelMessages` is now async — added `await`
- MCP import path changed: `experimental_createMCPClient` from `ai` →
`createMCPClient` from `@ai-sdk/mcp`
- `openai()` defaults to Responses API — added `store: false` to
provider options for ZDR org compatibility

**Streaming fix:**
Added `Content-Encoding: none` header to `pipeUIMessageStreamToResponse`
calls. Without it, proxy middleware buffers the entire SSE response
before flushing, causing the full reply to appear at once.

**Zero Data Retention fix:**
In recent AI SDK versions, `openai()` default to Responses API instead
of the legacy chat completions API. This produces a 404 from OpenAI with
message `"Items are not persisted for Zero Data Retention organizations.
Remove this item from your input and try again."` The Responses API is
OpenAI's [recommended
endpoint](https://developers.openai.com/api/docs/guides/migrate-to-responses).

This PR adds `store: false` as mentioned in
https://github.com/vercel/ai/issues/10060 to avoid incompatible
persistence attempts.

**References:**
- https://ai-sdk.dev/docs/migration-guides/migration-guide-6-0
-
https://ai-sdk.dev/docs/troubleshooting/streaming-not-working-when-proxied
- https://github.com/vercel/ai/issues/10060

Closes AI-514
Related AI-509
2026-03-19 09:07:33 -04:00
Matt Rossman 3fcb6cbe2c fix(studio): pass providerOptions to all AI SDK calls (#43031)
`reasoningEffort: 'minimal'` was
[configured](https://github.com/supabase/supabase/blob/d5cc70560d/apps/studio/lib/ai/model.utils.ts#L55-L59)
in the provider registry but `getModel()` returns it as a separate value
that callers must destructure and forward — and 7 of 8 endpoints weren't
doing so. This meant `gpt-5-mini` (a reasoning model) was running at
default reasoning effort for every call.

This PR destructures `providerOptions` from `getModel()` and passes it
to `generateObject`/`generateText` in all affected endpoints.

## Benchmark (local, median of 5 runs)

| Endpoint | Before (s) | After (s) | Speedup |
|----------|-----------|----------|---------|
| title-v2 | 7.0 | 1.9 | 3.7x |
| cron-v2 | 2.3 | 0.9 | 2.6x |
| filter-v1 | 5.8 | 2.2 | 2.6x |
| feedback/classify | 3.5 | 0.9 | 3.9x |
| feedback/rate | 2.9 | 0.9 | 3.2x |

`code/complete` and `policy` also received the fix but aren't
benchmarked here as they require a live DB connection and use multi-step
tool calls (separate latency concern tracked in AI-419).

To test the SQL naming, visit the SQL Editor in sidebar, add some SQL
like:

```sql
create table todos (                                                                                                 
  id serial primary key,                                                                                             
  task text not null,                                                                                                
  completed boolean default false
);
```

Right click on the snippet, "Rename" and "Rename with Supabase AI"

Closes AI-443
2026-02-20 11:36:06 -05:00
Matt Rossman 20d1ac0f83 feat(assistant): send user feedback to Braintrust traces (#43021)
- Expose Braintrust span ID to client via `x-braintrust-span-id`
response header, captured in chat transport
- On feedback submission, call `logFeedback()` with `scores["User
Rating"]` (1/0), `comment`, and on the root span assign
`metadata.feedbackCategory`
- Silently skipped when tracing is disabled (HIPAA, missing env vars)
- Log `requestedModel` in trace metadata so we can see what model the
user selected vs what was actually used after throttling

Example traces:

- [Thumbs
up](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=project_logs&object_id=5a8d02e5-b3b6-40cc-ba76-ecee286478f4&r=0bb71680-784c-45c1-a234-cba0242562d6&s=0bb71680-784c-45c1-a234-cba0242562d6)
- [Thumbs down + negative
feedback](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=project_logs&object_id=5a8d02e5-b3b6-40cc-ba76-ecee286478f4&r=d5a78084-6c9a-4230-8615-1e864bb9bac7&s=d5a78084-6c9a-4230-8615-1e864bb9bac7)

<img width="645" height="173" alt="CleanShot 2026-02-19 at 13 30 25@2x"
src="https://github.com/user-attachments/assets/6c463e83-27c6-4afb-a8d0-a329ed61270a"
/>

Closes AI-442
2026-02-19 15:00:07 -05:00
Saxon Fletcher d5bf4ef13b Rate assistant responses (#39224)
* rate assistant responses

* test

* always show
2025-10-06 22:46:30 +00:00
Saxon FletcherandJoshen Lim bf004b8a1d Choose provider and model (#38267)
* choose provider and model

* fix test

* Fix lint

* Nit clean up

* add provider options and remove anthropic

* remove package

* use chat model and update prompt

* prompt rendering

* default to pro

* fix test

* low

* remove log

* remove redundant part.text

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-09-09 10:05:39 +00:00
Terry SuttonandIvan Vasilov f9921569a9 Nudge feedback widget, move request to bedrock (#38074)
* Nudge feedback widget, move request to bedrock

* Fix the response object

* Remove an import to deleted file.

---------

Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
2025-08-21 08:56:54 -02:30
Terry SuttonandJoshen Lim 8fe31d903b Chore/feedback classifier (#36147)
* First

* Update

* Cleanup

* Reduce debounce

* Add a picker before displaying feedback widget

* Update styles

* Clean up

* Fix error msg

* More prompt examples

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-06-06 16:01:10 -02:30