mirror of
https://github.com/supabase/supabase.git
synced 2026-10-06 09:55:06 +03:00
Matt Rossman
e8ab92408f
Enables Braintrust tracing for AI Assistant chats to support debugging and future online evals. **Code Changes** - Wraps `generateAssistantResponse` in a Braintrust `traced()` span, logging the user's latest message as input along with metadata (`chatId`, `chatName`, `projectRef`, `userId`, `orgId`, `planId`, etc.) - Threads JWT claims from `apiWrapper` → handler to log `userId` in Braintrust without an extra API call (+ expanded `apiWrapper` tests) - Threads `orgId` and `planId` from `getOrgAIDetails` to log in Braintrust **Infrastructure Changes** - Created a "Vercel" service account in Braintrust - Added `BRAINTRUST_API_KEY` and `BRAINTRUST_PROJECT_ID` env vars to the studio-staging project in Vercel using a service token for the above service account - Added an "Overview" view to the Logs tab in the Braintrust Assistant project to surface the new metadata **Precautions** - HIPAA sensitive projects are excluded from logging (see https://github.com/supabase/supabase/pull/42787 for the detection logic) - Production is temporarily excluded from logging until we're confident in the setup **Testing steps** - Chat with the AI Assistant in the [studio-staging preview build](https://github.com/supabase/supabase/pull/42963#issuecomment-3917178023) below - Visit the [Logs tab in the Braintrust Assistant project](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs) and inspect the trace <img width="4680" height="962" alt="CleanShot 2026-02-18 at 17 43 55@2x" src="https://github.com/user-attachments/assets/c3a11b21-4e7f-4e90-bdab-a25ab8ee0d1f" /> <img width="2632" height="1288" alt="CleanShot 2026-02-18 at 17 45 04@2x" src="https://github.com/user-attachments/assets/6c7b6ebc-5090-4ede-8f71-859ff7e386aa" /> **References** - https://www.braintrust.dev/docs/integrations/sdk-integrations/vercel - https://www.braintrust.dev/docs/instrument/custom-tracing Closes AI-438