Commit Graph
50 Commits
Author SHA1 Message Date
Matt Rossman f49ed14dfb fix(assistant): preserve template URL placeholders in responses (#43495)
Fixes a rendering bug where template URLs like
`https://<project-ref>.supabase.co/...` were displayed incorrectly in
Assistant responses. The markdown parser was treating `<project-ref>` as
a tag and silently stripping it.

- Adds `wrapPlaceholderUrls()` preprocessing step that auto-wraps bare
URLs containing `<kebab-case>` placeholders in backticks before passing
to the markdown renderer
- Strengthens the prompt instruction to explicitly say "always format
template URLs as inline code using backticks"

Sample prompt:
```
What are the OAuth2 endpoints for Supabase projects? List the authorization, token, and JWKS URLs.
```

| Before | After |
|--------|--------|
| <img width="1314" height="841" alt="CleanShot 2026-03-06 at 15 32
13@2x"
src="https://github.com/user-attachments/assets/55eaa0c6-39fb-48e0-91a0-31903021a4c9"
/> | <img width="1421" height="827" alt="CleanShot 2026-03-06 at 15 31
37@2x"
src="https://github.com/user-attachments/assets/5335c71a-19c7-44a0-b027-6f2efd76eb8c"
/> |

Closes AI-470
2026-03-11 13:56:43 -04:00
Matt Rossman 517171b246 feat(assistant): online evals support and CI workflows (#43194)
Lays groundwork for online evals on Assistant chat logs.

https://www.braintrust.dev/docs/observe/score-online

### Changes

- New workflows:
- `braintrust-scorers-deploy.yml` keeps prod scorers in sync on push to
`master`
- `braintrust-preview-scorers-deploy.yml` deploys preview scorers to the
staging project for PRs labeled `preview-scorers`, posting a comment
with scorer links
([example](https://github.com/supabase/supabase/pull/43194#issuecomment-4000097222))
- `braintrust-preview-scorers-cleanup.yml` deletes preview scorers when
the PR is closed
([example](https://github.com/supabase/supabase/pull/43194#issuecomment-4000749847))
- Adds `evals/scorer-online.ts` entry point invoked with `pnpm
scorers:deploy`, registering scorers for online evals in the Braintrust
"Assistant" project
- Refactors scorer code to separate online-compatible scorers
(`scorer-online.ts`) from WASM-dependent ones (`scorer-wasm.ts`)
- "URL Validity" scorer now only checks Supabase domains to prevent
requests to untrusted origins
- Span `input` is now shaped `{ prompt: string }` instead of plain
`string` for compatibility with offline eval scorers
- Env vars `BRAINTRUST_STAGING_PROJECT_ID` and `BRAINTRUST_PROJECT_ID`
configured in GitHub repo settings
- `generateAssistantResponse` now uses `startSpan` + `withCurrent`
instead of `traced()` to manually manage the root span lifecycle — this
ensures `onFinish` logs output to the span _before_ `span.end()` is
called, which is when Braintrust triggers scoring automations

### Online Scorers

We share scoring logic across offline and online evals, but some of our
scorers aren't transferrable to an "online" setting due to runtime
challenges or ground truth requirements.

**Supported**
- Goal Completion
- Conciseness
- Completeness
- Docs Faithfulness
- URL Validity

**Unsupported**
- Correctness (requires ground truth output)
- Tool Usage (requires ground truth requiredTools)
- SQL Syntax (uses libpg-query WASM)
- SQL Identifier Quoting (uses libpg-query WASM)
 
### How to use these scorers

Going forward if you want to add/edit online eval scorers, add the
`preview-scorers` label to a PR. This deploys scorers to the [Assistant
(Staging
Scorers)](https://www.braintrust.dev/app/supabase.io/p/Assistant%20(Staging%20Scorers)?v=Overview)
project in Braintrust with branch-specific slugs, and comments on the PR
([example](https://github.com/supabase/supabase/pull/43194#issuecomment-4000097222)).
From the Braintrust dashboard you can "Test" the scorer with traces from
any project.

<img width="1866" height="528" alt="CleanShot 2026-03-05 at 15 15 00@2x"
src="https://github.com/user-attachments/assets/4f15cebc-3f2d-4e8a-9ee2-fe8ef7bf4199"
/>

Once merged, scorers are deployed to the primary
[Assistant](https://www.braintrust.dev/app/supabase.io/p/Assistant)
project, and preview scorers are deleted from the staging project. Down
the road, scorers on the Assistant project will run automatically on a
sample of production traces.

Closes AI-437
2026-03-09 13:05:26 -04:00
Matt Rossman 20d1ac0f83 feat(assistant): send user feedback to Braintrust traces (#43021)
- Expose Braintrust span ID to client via `x-braintrust-span-id`
response header, captured in chat transport
- On feedback submission, call `logFeedback()` with `scores["User
Rating"]` (1/0), `comment`, and on the root span assign
`metadata.feedbackCategory`
- Silently skipped when tracing is disabled (HIPAA, missing env vars)
- Log `requestedModel` in trace metadata so we can see what model the
user selected vs what was actually used after throttling

Example traces:

- [Thumbs
up](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=project_logs&object_id=5a8d02e5-b3b6-40cc-ba76-ecee286478f4&r=0bb71680-784c-45c1-a234-cba0242562d6&s=0bb71680-784c-45c1-a234-cba0242562d6)
- [Thumbs down + negative
feedback](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=project_logs&object_id=5a8d02e5-b3b6-40cc-ba76-ecee286478f4&r=d5a78084-6c9a-4230-8615-1e864bb9bac7&s=d5a78084-6c9a-4230-8615-1e864bb9bac7)

<img width="645" height="173" alt="CleanShot 2026-02-19 at 13 30 25@2x"
src="https://github.com/user-attachments/assets/6c463e83-27c6-4afb-a8d0-a329ed61270a"
/>

Closes AI-442
2026-02-19 15:00:07 -05:00
Ignacio Dobronich d5cc70560d chore: ai assistant model entitlement (#42955)
### Changes
Replace hard-coded plan checks with `assistant.advance_model`
entitlement for AI model access control.

- `AIAssistant.tsx` & `ModelSelector.tsx`: Use
`useCheckEntitlements('assistant.advance_model')` instead of `plan.id
!== 'free'`

**Server-side:**
- `org-ai-details.ts`: Check `assistant.advance_model` entitlement via
new `checkEntitlement()` helper
- `entitlements-query.ts`: Add `checkEntitlement()` function with auth
headers support for server-side calls


### Test 1: Free Plan (No Entitlement)
- Open AI Assistant with a Free plan org
- Check model selector defaults to gpt-5-mini
- Click dropdown - verify gpt-5 shows "Upgrade" badge
- Click gpt-5 - should redirect to billing page
- Assert that you can send a message to the Assistant and that you get a
response
<img width="335" height="156" alt="image"
src="https://github.com/user-attachments/assets/2458cbd1-46ab-46cd-babb-09a47e163fd0"
/>


### Test 2: Paid Plan (Has Entitlement)
- Switch to Pro/Team/Enterprise org
- Check model selector defaults to gpt-5
- Click dropdown - verify gpt-5 shows checkmark (no badge)
- Switch to gpt-5-mini, then back to gpt-5 - should work without
redirect
- Assert that you can send a message to the Assistant and that you get a
response
2026-02-19 14:28:31 -03:00
Matt Rossman e8ab92408f feat(assistant): enable Braintrust tracing for non-sensitive chats (#42963)
Enables Braintrust tracing for AI Assistant chats to support debugging
and future online evals.

**Code Changes**

- Wraps `generateAssistantResponse` in a Braintrust `traced()` span,
logging the user's latest message as input along with metadata
(`chatId`, `chatName`, `projectRef`, `userId`, `orgId`, `planId`, etc.)
- Threads JWT claims from `apiWrapper` → handler to log `userId` in
Braintrust without an extra API call (+ expanded `apiWrapper` tests)
- Threads `orgId` and `planId` from `getOrgAIDetails` to log in
Braintrust

**Infrastructure Changes**

- Created a "Vercel" service account in Braintrust
- Added `BRAINTRUST_API_KEY` and `BRAINTRUST_PROJECT_ID` env vars to the
studio-staging project in Vercel using a service token for the above
service account
- Added an "Overview" view to the Logs tab in the Braintrust Assistant
project to surface the new metadata

**Precautions**

- HIPAA sensitive projects are excluded from logging (see
https://github.com/supabase/supabase/pull/42787 for the detection logic)
- Production is temporarily excluded from logging until we're confident
in the setup

**Testing steps**

- Chat with the AI Assistant in the [studio-staging preview
build](https://github.com/supabase/supabase/pull/42963#issuecomment-3917178023)
below
- Visit the [Logs tab in the Braintrust Assistant
project](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs)
and inspect the trace

<img width="4680" height="962" alt="CleanShot 2026-02-18 at 17 43 55@2x"
src="https://github.com/user-attachments/assets/c3a11b21-4e7f-4e90-bdab-a25ab8ee0d1f"
/>

<img width="2632" height="1288" alt="CleanShot 2026-02-18 at 17 45
04@2x"
src="https://github.com/user-attachments/assets/6c7b6ebc-5090-4ede-8f71-859ff7e386aa"
/>

**References**
- https://www.braintrust.dev/docs/integrations/sdk-integrations/vercel
- https://www.braintrust.dev/docs/instrument/custom-tracing

Closes AI-438
2026-02-19 11:43:47 -05:00
Matt Rossman 2fc062a725 feat(assistant): detect HIPAA customers in assistant logic (#42787)
Detects HIPAA customers server-side in the assistant code path. Threads
`isHipaaEnabled` boolean through `getOrgAIDetails` → `generate-v4` →
`generateAssistantResponse`. The motivation is to support online evals
down the road, where we'll want to exclude HIPAA projects from Assistant
tracing.

This PR follows existing patterns for checking if HIPAA is enabled for a
project (org has HIPAA addon + project is sensitive). Example
[[1]](https://github.com/supabase/supabase/blob/a5dd0a96716561443778f38a518b61d6cac95c19/apps/studio/components/interfaces/Settings/Addons/Addons.tsx#L75),
[[2]](https://github.com/supabase/supabase/blob/6858d4e18d9359d573fe3dff73bc4e5fa1cfe219/apps/studio/hooks/misc/useOrgOptedIntoAi.ts#L69).

```ts
const hasHipaaAddon = subscriptionHasHipaaAddon(subscription) && settings?.is_sensitive
```

(I call it `isHipaaEnabled` in this PR to avoid it being misunderstood
as just the org-level addon, rather it's a combo of that addon being
present AND high compliance being enabled on the project).

### Verification steps

<details><summary>Click to view the steps I followed to sanity check it
works with the local stack</summary>

Tested locally with `mise fullstack`:

1. Found my org's subscription ID:

   ```sh
docker exec platform-db-1 psql -U postgres -c "SELECT id, customer_id,
status FROM orb.subscriptions;"
   ```

2. Added HIPAA addon to it:

   ```sh
   docker exec platform-db-1 psql -U postgres -c "
     UPDATE orb.subscriptions
SET price_intervals = price_intervals || '[{\"price\": {\"unit_config\":
{\"unit_amount\": \"350.00\"}, \"external_price_id\":
\"addon_security_hipaa\", \"item\": {\"name\": \"HIPAA\"}}}]'::jsonb
     WHERE id = '<subscription_id>';"
   ```

2. Toggled on High Compliance (Project Settings → General)

3. Added a temporary log after `getOrgAIDetails` in `generate-v4.ts`:

   ```ts
   console.log('[HIPAA]', { isHipaaEnabled })
   ```

4. Sent a message in the AI Assistant → `isHipaaEnabled: true`

5. Toggled off High Compliance → resent → `isHipaaEnabled: false`

6. Removed addon from subscription, left project toggle on →
`isHipaaEnabled: false`

   ```sql
   -- Find addon index:
   SELECT ordinality - 1 as idx FROM orb.subscriptions,
jsonb_array_elements(price_intervals) WITH ORDINALITY AS elem(val,
ordinality)
     WHERE id = '<subscription_id>'
     AND val->'price'->>'external_price_id' = 'addon_security_hipaa';

   -- Remove by index:
UPDATE orb.subscriptions SET price_intervals = price_intervals - <idx>
     WHERE id = '<subscription_id>';
   ```

All three cases confirm `isHipaaEnabled` requires both the org addon and
the project-level toggle.


</details> 

Closes AI-434

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Added HIPAA mode detection and exposed it in AI workflows.
* API request functions now accept optional custom authorization headers
for downstream calls.

* **Tests**
* Added tests covering HIPAA scenarios and verifying authorization
header propagation in related flows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-02-17 15:59:26 -05:00
Jordi Enric ac64a902c1 chore: adds tests (#42653) 2026-02-11 09:50:11 +01:00
Matt Rossman 4b8bab4d14 feat(assistant): score URL validity and fix support ticket URL guidance (#42227)
**Logic changes**
- Adds function in `helpers.ts` to extract URLs from text via regex
- I also considering using a library like
[linkify-it](https://www.npmjs.com/package/linkify-it) for this but
figured it's not worth the extra dep
- Adds associated tests in `helpers.test.ts`
- Adds "URL Validity" scorer which performs a HEAD request for links in
Assistant response text and determins what portion of links have `.ok`
responses
- Adds eval case to check correctness of support ticket URL answers

**Prompt changes**
- Informs Assistant of https://supabase.com/dashboard/support/new being
the URL to create support tickets
- Encourages Assistant to "self-debug" issues before directing users to
create support tickets

See [Eval
Report](https://github.com/supabase/supabase/pull/42227#issuecomment-3807772871)
and
[Correctness](https://www.braintrust.dev/app/supabase.io/p/Assistant/trace?object_type=experiment&object_id=1ad0f9b0-5adb-436c-9812-a87aac62c036&r=1ef13459-a98c-4904-925e-6d81276cebb2&s=dbe5c607-a560-462b-8745-41d430744431)
analysis for new support ticket test case.

Resolves AI-384

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added URL validity scoring to evaluations and helper utilities for
extracting/cleaning URLs.
* Added evaluation cases for support-ticket URL handling and OAuth
callback guidance.

* **Documentation**
* Updated assistant guidance to prefer self-resolution, include
support-ticket direction, clarified data-recovery search steps, and
added template-URL notation.

* **Tests**
* Expanded URL extraction and related utility tests to cover many
formats and edge cases.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-30 09:53:21 -05:00
Matt Rossman eb259f1364 feat(assistant): score and improve SQL identifier quoting (#42122)
* feat: SQL correctness scorer, override mock tables

* feat: replace "SQL Correctness" with "SQL Identifier Quoting" scorer

* fix(prompt): discourage simulating confirmation of execute_sql tool

this is already handled at the UI layer

* fix(prompt): encourage quotes on identifiers with caps

* feat: move extractIdentifiers to own file, add tests

* chore: shorten tests

* feat: extract ColumnDef column names in extractIdentifiers

* refactor: sqlIdentifierQuotingScorer with more thorough checks

* refactor: consolidate into `sql-identifier-quoting.ts`

* feat: support mocking schemas, eval test case with case sensitive schema

* fix: test cases that don't match default mock schema

* chore: format

* feat(prompt): mention special characters and reserved words

* feat: optional description in metadata, test with special characters

* feat: consolidated comprehensive test case

* fix(prompt): revert conflicting instruction
2026-01-27 09:28:56 -05:00
Danny WhiteandCharis Lam 05dc676d36 feat(studio): incident-aware AI assistant (#41603)
* handling and mock data support

* admonition

* only show in empty state

* rabbit

* mock without local overrides

* remove admonition

* make incident banner more prominent

* remove mock data

* move to tool

* prettier

* fix(studio): get_active_incidents tool

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
2026-01-21 13:52:29 +11:00
Matt Rossman 4553f09bb5 feat(assistant): hallucination scorers + corrective measures for storage versioning answers (#41655)
* feat: "Docs Faithfulness" scorer

* feat: test case for storage object restoration

* feat: "Factuality" scorer

* feat: "Factuality" -> "Correctness"

* feat: update Storage recovery test case

* feat: finishReason in task output

* feat: encourage parallel tool calls + docs search, discourage superfluous context gathering

* prompt tuning (tool selection strategy)

* add data recovery section in chat prompt

* test: S3 versioning support correctness

* refactor: derive stepsSerialized/textOnly from shared steps data

* fix: input in correctness scorer
2026-01-15 14:22:46 +07:00
072883bcec feat: assistant evals (#41311)
* chore: bump `supabase` CLI

* chore: stricter message types in `generate-v4.ts`

* feat: tutorial eval

https://www.braintrust.dev/docs/evaluation

* feat: project ID for eval

* refactor: `generateAssistantResponse` out of `handlePost`

* refactor: generateAssistantResponse to lib/ai

* feat: factuality eval with assistant response

* chore: upgrade braintrust to v1.0.1

* chore: silence tsconfig warning

* feat: assertion scorer

* fix: aggregate tools across all steps

* refactor: strict tool names, remove need for `as const`

* refactor: generic tool name type in assertions

* feat: transfer mocks from `feature/braintrust`

* feat: LLM criteria assertion

* feat: braintrust evals workflow

* fix: BRAINTRUST_PROJECT_ID

* feat: `sql_similar` assertion

* fix: `OPENAI_API_KEY` in workflow env

* feat: split AssertionScorer into separate scorers

* feat: remove tutorial eval

* feat: 20 minute CI timeout

* feat: category in test case metadata

* feat: score with gpt-5

* refactor: dataset to own file, colocate scorers

* feat: "gpt-5.2-2025-12-11" for llm as a judge

* feat: SQL syntax scorer with `libpg-query`

* feat: `evals:setup` and `evals:run` scripts

* feat: `evals:setup` in CI

* feat: human readable scorer names

* chore: rename to "SQL Validity"

* feat: add 2 "sql_generation" test cases

* feat: update requiredTools in test cases

* chore: ignore Cursor MCP config

* feat: "Conciseness" score

* feat: "Completeness" scorer

* fix: generate-v4 test mocks

* feat: serialize "steps" for scorer inputs

* updated node mem options for typecheck

* updated runner

* remove ram update as actions handle this

* feat: read `BRAINTRUST_PROJECT_ID` from secrets

* feat: score helpfulness, remove old scorers

* feat: separate `evals:run` and `evals:upload` scripts

* feat: passthrough entire classifier result

* feat: use live `search_docs` impl, store docs result in metadata

* feat: reduce classifier options

* feat: filter workflow by `run-evals` PR label or `master` branch

* chore: cleanup stubbed mock tools

* fix: checkout actual branch with `ref:`

* fix: capture search_docs results from all content parts

* feat: simplify sql syntax score calculation

* feat: use AI SDK's UI message validator

* docs: justification for relative `extends`

* fix: cleanup leftover validatedMessages

* doc: note mock token isn't secret for snyk

* fix: mock ui message to pass validation

* feat: revert ignoring Cursor MCP config

Using `.git/info/exclude` instead until we have an opinion on this

* feat: add "tsconfig" as shared-data devDependency, revert relative path in tsconfig

* refactor: tool call parsing into function

* Update apps/studio/evals/assistant.eval.ts

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* refactor: organize mock schemas and tool factories

---------

Co-authored-by: Ali Waseem <waseema393@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-12-22 23:45:48 -05:00
Saxon FletcherandKevin Grüneberg 5579b8e7eb Added billing context (#40933)
* added billing context

* Update apps/studio/lib/ai/prompts.ts

Co-authored-by: Kevin Grüneberg <k.grueneberg1994@gmail.com>

* direct links billing

---------

Co-authored-by: Kevin Grüneberg <k.grueneberg1994@gmail.com>
2025-12-08 16:27:41 +10:00
Joshen Lim 0f4fc7eee9 Chore/projects pagination part 04 (#39348)
* CmdK OrgProjectSwitcher to swap useProjectsQuery with useProjectsInfiniteQuery

* Remove usage of useProjectsQuery in ProjectDropdown

* Remove usage of useProjectsQuery in NotificationsPopover

* Remove usage of useProjectsQuery in NotificationsFilter

* Remove usage of useProjectsQuery from LoadingState

* Clean

* Remove usage of getProjects from org-ai-details and fix missing key props in AIOnboarding

* Remove useAutoProjectsPrefetch from org/[slug]/index

* Fix TS  + clean up

* Clean

* Remove usage of useProjectsQuery in NewOrgForm
2025-10-08 21:42:49 +08:00
54ad0b7495 Chore/ai realtime (#39145)
* update onboarding

* update model and fix part issue

* action orientated assistant

* fix tool

* lock

* remove unused filter

* fix tests

* fix again

* update package

* update container

* fix tests

* ai realtime

* ai realtime

* refactor(ai assistant): break out message markdown and profile picture

* wip

* refactor(ai assistant): break up message component

* refactor: break ai assistant message down into multiple files

* add limitations prompt

* limitations prompt

* link prompt

* refactor: simplify ReportBlock state

* fix: styling of draggable report block header

When the drag handle is showing, it overlaps with the block header.
Decrease the opacity of the header so the handle can be seen and the two
can be distinguished.

* fix: minor tweaks to tool ui

* refactor: simplify DisplayBlockRenderer state

* fix: remove double deploy button in edge function block

When the confirm footer is shown, the deploy button on the top right should be
hidden (not just disabled) to avoid confusion.

* refactor, test: message sanitization by opt-in level

Refactor the message sanitization to have more type safety and be more testable.
Add tests to ensure:

- Message sanitization always runs on generate-v4
- Message sanitization correctly works by opt-in level

* Fix conflicts in pnpm lock

* Couple of nits and refactors

* Revert casing for report block snippet

* adjust sanitised prompt

* Fix tests

* empty states

* prompt otpimise

* refine prompt

* prompt optimizer

* remove realtime for now

* Update apps/studio/lib/ai/prompts.ts

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

* Update apps/studio/lib/ai/prompts.ts

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

* updates

* feature flag

* use flag and additional check

* remove sort

* messages copy

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-10-06 10:31:40 +10:00
Saxon Fletcher 1ae6f4f44f re-add chart config (#39149) 2025-10-02 12:41:29 +10:00
Saxon Fletcher e90b8ac495 Context gathering prominence (#39072) 2025-09-30 06:20:13 +10:00
1973615a42 Prompt GPT-5 Optimizations (#39031)
* update onboarding

* update model and fix part issue

* action orientated assistant

* fix tool

* lock

* remove unused filter

* fix tests

* fix again

* update package

* update container

* fix tests

* refactor(ai assistant): break out message markdown and profile picture

* wip

* refactor(ai assistant): break up message component

* refactor: break ai assistant message down into multiple files

* add limitations prompt

* limitations prompt

* link prompt

* refactor: simplify ReportBlock state

* fix: styling of draggable report block header

When the drag handle is showing, it overlaps with the block header.
Decrease the opacity of the header so the handle can be seen and the two
can be distinguished.

* fix: minor tweaks to tool ui

* refactor: simplify DisplayBlockRenderer state

* fix: remove double deploy button in edge function block

When the confirm footer is shown, the deploy button on the top right should be
hidden (not just disabled) to avoid confusion.

* refactor, test: message sanitization by opt-in level

Refactor the message sanitization to have more type safety and be more testable.
Add tests to ensure:

- Message sanitization always runs on generate-v4
- Message sanitization correctly works by opt-in level

* Fix conflicts in pnpm lock

* Couple of nits and refactors

* Revert casing for report block snippet

* adjust sanitised prompt

* Fix tests

* prompt optimizer

* remove realtime for now

* Update apps/studio/lib/ai/prompts.ts

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

* Update apps/studio/lib/ai/prompts.ts

Co-authored-by: Charis <26616127+charislam@users.noreply.github.com>

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-09-29 14:07:53 +10:00
626eb30e77 Assistant action orientated approach (#38806)
* update onboarding

* update model and fix part issue

* action orientated assistant

* fix tool

* lock

* remove unused filter

* fix tests

* fix again

* update package

* update container

* fix tests

* refactor(ai assistant): break out message markdown and profile picture

* wip

* refactor(ai assistant): break up message component

* refactor: break ai assistant message down into multiple files

* refactor: simplify ReportBlock state

* fix: styling of draggable report block header

When the drag handle is showing, it overlaps with the block header.
Decrease the opacity of the header so the handle can be seen and the two
can be distinguished.

* fix: minor tweaks to tool ui

* refactor: simplify DisplayBlockRenderer state

* fix: remove double deploy button in edge function block

When the confirm footer is shown, the deploy button on the top right should be
hidden (not just disabled) to avoid confusion.

* refactor, test: message sanitization by opt-in level

Refactor the message sanitization to have more type safety and be more testable.
Add tests to ensure:

- Message sanitization always runs on generate-v4
- Message sanitization correctly works by opt-in level

* Fix conflicts in pnpm lock

* Couple of nits and refactors

* Revert casing for report block snippet

* adjust sanitised prompt

* Fix tests

---------

Co-authored-by: Charis Lam <26616127+charislam@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-09-29 03:57:36 +00:00
5f533247e1 Update docs url to env var (#38772)
* Update Supabase docs URLs to use env variable

Co-authored-by: a <a@alaisteryoung.com>

* Refactor: Use DOCS_URL constant for documentation links

This change centralizes documentation links using a new DOCS_URL constant, improving maintainability and consistency.

Co-authored-by: a <a@alaisteryoung.com>

* Refactor: Use DOCS_URL constant for all documentation links

This change replaces hardcoded documentation URLs with a centralized constant, improving maintainability and consistency.

Co-authored-by: a <a@alaisteryoung.com>

* replace more instances

* ci: Autofix updates from GitHub workflow

* remaining instances

* fix duplicate useRouter

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: alaister <10985857+alaister@users.noreply.github.com>
2025-09-26 10:16:33 +00:00
Greg Richardson 9820707d71 feat: local mcp server (#38797)
* feat: local mcp server

* feat(local-mcp): implement migrations

* fix: remove unsupported mcp args

* feat(local-mcp): tests

* fix(local-mcp): packages to adhere to minimumReleaseAge

* fix(mcp): import path for createSupabaseApiPlatform

* fix(local-mcp): move tests out of pages/api dir

* refactor: self-hosted execute sql logic

* fix: deps
2025-09-23 12:00:08 -06:00
Saxon Fletcher d276aca6cb Chore/ai flow model (#38805)
* update onboarding

* update model and fix part issue

* lock

* remove unused filter

* fix tests

* fix again

* update package

* fix merge issue
2025-09-23 12:46:20 +10:00
Charis 2e6a7093d6 refactor: make MCP docs url configurable (#38763) 2025-09-17 12:25:31 -04:00
Saxon Fletcher b3a687e7a4 Fix filtering issue (#38613)
filter issue
2025-09-11 16:32:26 +10:00
Saxon Fletcher 3775607213 Re-add get_logs tool (#38612)
readd log tool
2025-09-11 16:32:04 +10:00
Saxon Fletcher 577c3bf9bf Refine prompt (#38576)
refine prompt
2025-09-10 15:14:34 +08:00
Saxon FletcherandJoshen Lim bf004b8a1d Choose provider and model (#38267)
* choose provider and model

* fix test

* Fix lint

* Nit clean up

* add provider options and remove anthropic

* remove package

* use chat model and update prompt

* prompt rendering

* default to pro

* fix test

* low

* remove log

* remove redundant part.text

---------

Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-09-09 10:05:39 +00:00
2538a5c8af Assistant UI refine (#38228)
* try a really long context window to maximize caching

* update examples

* attempt to update packages and useChat

* update endpoints

* update zod

* zod

* update to v5

* message update

* Revert "zod"

This reverts commit ec39bac6b6.

* revert zod

* zod i

* fix complete endpoints

* remove async

* change to content

* type cleanup

* Revert the package bumps to rebuild them.

* Bump zod to 2.25.76 in all packages.

* Bump openai in all packages.

* Bump ai and ai-related packages.

* Remove unneeded files.

* Fix the rest of the migration stuff.

* Prettier fixes.

* testing new gptoss models

* sonnet as pro

* back to 3.7

* revert test

* add policy list tool

* fix

* refactor

* ai sdk 5 fixes

* refactor complete endpoint

* edge function prompt

* remove example

* use limited model for completions

* remove duplicate

* Update bedrock.ts

Co-authored-by: divit <27228526+delgado3d@users.noreply.github.com>

* add default check

* only add prefix if more then one region

* temp use api

* refactor to use openai api

* test

* remove unused import

* Prettier lint

* Fix test

* revert back to bedrock

* Tiny style fix for spacing of code blocks from edit message button

* use conversation

* try generate text normally

* prompt adjust

* add list style to prompt

* prompt adjust

* status

* prompt adjust emphasise display query

* rendering

* prompt

* ui updates

* update prompts

* more ui

* ui finesse

* fix

* use streamdown

* remove package

* refactor header

* copy

* prompt adjust

* remove package

* prompt adjust

* Revert "use streamdown"

This reverts commit 22d0e88e54.

* remove streamdown

* margin adjust

* Nit

* Minor clean up

---------

Co-authored-by: Jordi Enric <jordi.err@gmail.com>
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
Co-authored-by: divit <27228526+delgado3d@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-08-28 12:39:26 +07:00
Joshen Lim 0af359a864 Swap to use paginated projects endpoint (#38192) 2025-08-27 14:09:58 +07:00
cddc472705 Test GPT-OSS (#37710)
* try a really long context window to maximize caching

* update examples

* attempt to update packages and useChat

* update endpoints

* update zod

* zod

* update to v5

* message update

* Revert "zod"

This reverts commit ec39bac6b6.

* revert zod

* zod i

* fix complete endpoints

* remove async

* change to content

* type cleanup

* Revert the package bumps to rebuild them.

* Bump zod to 2.25.76 in all packages.

* Bump openai in all packages.

* Bump ai and ai-related packages.

* Remove unneeded files.

* Fix the rest of the migration stuff.

* Prettier fixes.

* testing new gptoss models

* sonnet as pro

* back to 3.7

* revert test

* add policy list tool

* fix

* refactor

* ai sdk 5 fixes

* refactor complete endpoint

* edge function prompt

* remove example

* use limited model for completions

* remove duplicate

* Update bedrock.ts

Co-authored-by: divit <27228526+delgado3d@users.noreply.github.com>

* add default check

* only add prefix if more then one region

* temp use api

* refactor to use openai api

* test

* remove unused import

* Prettier lint

* Fix test

* revert back to bedrock

* Tiny style fix for spacing of code blocks from edit message button

* try generate text normally

* prompt adjust

* add list style to prompt

* prompt adjust

* prompt adjust emphasise display query

* update prompts

* prompt adjust

* remove package

* prompt adjust

---------

Co-authored-by: Jordi Enric <jordi.err@gmail.com>
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
Co-authored-by: divit <27228526+delgado3d@users.noreply.github.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-08-27 11:34:44 +07:00
d60aceb562 Prompt and tool refactoring (#37500)
* try a really long context window to maximize caching

* update examples

* attempt to update packages and useChat

* update endpoints

* update zod

* zod

* update to v5

* message update

* Revert "zod"

This reverts commit ec39bac6b6.

* revert zod

* zod i

* fix complete endpoints

* remove async

* change to content

* type cleanup

* Revert the package bumps to rebuild them.

* Bump zod to 2.25.76 in all packages.

* Bump openai in all packages.

* Bump ai and ai-related packages.

* Remove unneeded files.

* Fix the rest of the migration stuff.

* Prettier fixes.

* add policy list tool

* refactor

* ai sdk 5 fixes

* refactor complete endpoint

* edge function prompt

* remove example

* slight prompt change

* Minor clean up

* More clean up

---------

Co-authored-by: Jordi Enric <jordi.err@gmail.com>
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
2025-08-08 15:25:57 +07:00
d52e2c252b Update AI SDK 5 (#37656)
* attempt to update packages and useChat

* update endpoints

* update zod

* zod

* update to v5

* message update

* Revert "zod"

This reverts commit ec39bac6b6.

* revert zod

* zod i

* fix complete endpoints

* remove async

* change to content

* type cleanup

* Revert the package bumps to rebuild them.

* Bump zod to 2.25.76 in all packages.

* Bump openai in all packages.

* Bump ai and ai-related packages.

* Remove unneeded files.

* Fix the rest of the migration stuff.

* Prettier fixes.

* zod

* v4 again

---------

Co-authored-by: Jordi Enric <jordi.err@gmail.com>
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
2025-08-08 14:15:39 +07:00
Joshen Lim 7505c2923e Update AI model check (#37395)
* Update AI model chekc

* Add comment

* Fix test
2025-07-23 17:40:19 +08:00
Joshen LimandAlaister Young e70242f822 Chore/clean up assistant mcp feature flags (#36772)
* Clean up usage of newOrgAiOptIn and useBedrockAssistant feature flags

* Remove all OpenAI endpoints

* Fix for self-hosted

* Default isLimited to false

* Update PG meta tests

* Fix unit tests for model

* Revert pg meta tests

* Fix test

---------

Co-authored-by: Alaister Young <alaister@users.noreply.github.com>
2025-07-22 16:05:16 +08:00
Greg Richardson 2afd0c92ab fix(ai): temporarily remove log tools (#37274)
* fix(ai): temporarily remove log tools

* fix: temp remove log tools from tool filter logic

* fix: temp remove log tools from tests

* fix: remove references to get_logs in system prompt
2025-07-17 21:27:54 +00:00
divit 2835e8875f feat: distribute bedrock requests per regional limits (#37049) 2025-07-17 18:11:06 +10:00
Saxon Fletcher 834dcb5a29 Dynamic model setting (#37095) 2025-07-15 17:57:38 +10:00
Saxon Fletcher 80b9b6fcbd Temporarily switch to Haiku (#37050) 2025-07-12 08:42:02 +10:00
Saxon Fletcher ff49dee4be haiku model (#37043) 2025-07-11 11:53:52 +00:00
Saxon Fletcher fd670667ab Context reduction (#37032)
* context saving

* fix model name

* add results comment

* fix errors
2025-07-11 16:03:37 +10:00
divit 3cf6a2d17f feat: add us-west-2 for bedrock regions (#37022) 2025-07-10 20:58:18 +05:30
Saxon Fletcher e3125e59c2 Limit tools based on extensions (#37006)
* limit tools based on extensions

* if failed, fall back to disabled
2025-07-10 15:54:19 +10:00
Saxon Fletcher 0419d45b6f Add search docs, advisors, and custom logging tools (#36852)
* ui refinements

* rename chat

* copy

* prose message styles

* add icon back

* fix message save

* simplify empty state

* update suggestions

* pass through props

* button styles

* onboarding icons

* name button

* remove results

* current chat name

* use type

* add advisor and search

* fix down arrow

* logging tools

* more general filter tools

* fixes

* pageparam fix

* move search docs
2025-07-10 09:43:38 +10:00
divit c976afac31 fix: remove apac bedrock region and add us region (#36881)
* fix: remove apac bedrock region and add us region

* fix: update test
2025-07-07 09:53:08 +05:30
Greg Richardsonanddelgado3d ea91bf4739 Distributes bedrock requests evenly across regions (#36837)
* wip

* feat: distribute bedrock requests across regions

* fix: tests and type errors

* chore: remove AWS_BEDROCK_REGION from turbo.json

* fix: vercel oidc provider to fallback in credential chain

---------

Co-authored-by: delgado3d <27228526+delgado3d@users.noreply.github.com>
2025-07-02 17:35:37 +00:00
Saxon Fletcher 9f75d02ae0 change bedrock model (#36830)
* change bedrock model

* use 3.5

* Revert "use 3.5"

This reverts commit a76ff04513.
2025-07-02 12:54:26 +00:00
Greg Richardson 1aa173d360 fix: bedrock region (#36706) 2025-06-26 07:37:16 -06:00
Joshen Lim 63164fe165 Revert "Revert "Feature/assistant mcp"" (#36687)
* Revert "Revert "Feature/assistant mcp" (#36684)"

This reverts commit 2bff117cde.

* Attempt
2025-06-26 15:02:49 +08:00
Joshen Lim 2bff117cde Revert "Feature/assistant mcp" (#36684)
Revert "Feature/assistant mcp (#35318)"

This reverts commit 3a50dd1f72.
2025-06-26 13:06:50 +08:00
3a50dd1f72 Feature/assistant mcp (#35318)
* step 1

* use mcp

* system prompt

* clean up

* space

* clean up

* add three state opt in toggle

* clean up

* todo

* hooks

* refactor opt in form and modal

* refinements

* add bedrock

* remove console

* update mcp util

* use bedrock

* remove openai sdk package

* re-add tools

* update complete endpoints

* fix: ai assistant markdown formatting (#35541)

* fix: mcp types and project ref (#35540)

* feat: more flexible aws credential provider (#35538)

* feat: more flexible aws credential provider

* fix: add AWS_REGION to turbo env vars

* change to allowed

* update complete endpoints

* add an additional permission

* refinements

* use claud 4

* legal copy changes

* update other ai functions to use bedrock

* update generate v3 copy

* remove generate sql modal

* fixes for query block

* re-add dragging to reports

* clean up

* add open ai edge function example

* use handle error from fetchers

* remove schema and lean on tools

* copy

* Assistant MCP tests (#36049)

* feat: refactor and test mcp and model logic

* fix: remove get_project tool

* fix: remove additional get_project tool references

* update copy

* Clean up, fixes, refactors

* oops

* Float errors from AI endpoionts as toasts

* Use a env var AWS_BEDROCK_PROFILE for bedrock.

* Rename the env var for AWS bedrock profile.

* feat: support custom aws bedrock env vars

* chore: add comments explaining aws credential chain

* MCP Self Hosted Check (#36185)

support self hosted

* feat: bedrock auth via vercel oidc

* Fix broken unit test

* Feeeex

* Refactor useOrgOptedIntoAi

* Remove useDisallowHipaa hook

* small system prompt change

* readd vercel packages

* fix self hosted

* increase max duration

* try more direct prompt

* max duration 90

* reduce max steps and add loading

* mono font

* backwards compat styling

* Chore/limit number of messages sent to assistant (#36388)

* Limit number of historical messages that get sent to assistant

* Update max chat history to 5

* alignment

* bump mcp server version

* Add feature flag for opt in tags (#36466)

* Add feature flag for opt in tags

* Add one more check

* security section system prompt

* rely on default link and replace image markdown

* Add custom link component to assistant message block (#36527)

* Add custom link component to assistant message block

* Update based on feedback

* Render plain text if URL is deemed unsafe

* fix mcp tools and parse data (#36593)

* Update Admonition for AI Assistant for when opt in is re-enabled (#36663)

* Update Admonition for AI Assistant for when opt in is re-enabled

* Update

* Smol fix

* Fix TS

* Tiny

---------

Co-authored-by: Greg Richardson <greg.nmr@gmail.com>
Co-authored-by: Joshen Lim <joshenlimek@gmail.com>
Co-authored-by: Ivan Vasilov <vasilov.ivan@gmail.com>
2025-06-26 12:13:27 +08:00