mirror of
https://github.com/supabase/supabase.git
synced 2026-10-11 20:35:07 +03:00
Enables Braintrust tracing for AI Assistant chats to support debugging and future online evals. **Code Changes** - Wraps `generateAssistantResponse` in a Braintrust `traced()` span, logging the user's latest message as input along with metadata (`chatId`, `chatName`, `projectRef`, `userId`, `orgId`, `planId`, etc.) - Threads JWT claims from `apiWrapper` → handler to log `userId` in Braintrust without an extra API call (+ expanded `apiWrapper` tests) - Threads `orgId` and `planId` from `getOrgAIDetails` to log in Braintrust **Infrastructure Changes** - Created a "Vercel" service account in Braintrust - Added `BRAINTRUST_API_KEY` and `BRAINTRUST_PROJECT_ID` env vars to the studio-staging project in Vercel using a service token for the above service account - Added an "Overview" view to the Logs tab in the Braintrust Assistant project to surface the new metadata **Precautions** - HIPAA sensitive projects are excluded from logging (see https://github.com/supabase/supabase/pull/42787 for the detection logic) - Production is temporarily excluded from logging until we're confident in the setup **Testing steps** - Chat with the AI Assistant in the [studio-staging preview build](https://github.com/supabase/supabase/pull/42963#issuecomment-3917178023) below - Visit the [Logs tab in the Braintrust Assistant project](https://www.braintrust.dev/app/supabase.io/p/Assistant/logs) and inspect the trace <img width="4680" height="962" alt="CleanShot 2026-02-18 at 17 43 55@2x" src="https://github.com/user-attachments/assets/c3a11b21-4e7f-4e90-bdab-a25ab8ee0d1f" /> <img width="2632" height="1288" alt="CleanShot 2026-02-18 at 17 45 04@2x" src="https://github.com/user-attachments/assets/6c7b6ebc-5090-4ede-8f71-859ff7e386aa" /> **References** - https://www.braintrust.dev/docs/integrations/sdk-integrations/vercel - https://www.braintrust.dev/docs/instrument/custom-tracing Closes AI-438
18 lines
555 B
TypeScript
18 lines
555 B
TypeScript
import { initLogger } from 'braintrust'
|
|
|
|
const BRAINTRUST_API_KEY = process.env.BRAINTRUST_API_KEY
|
|
const BRAINTRUST_PROJECT_ID = process.env.BRAINTRUST_PROJECT_ID
|
|
|
|
const IS_PRODUCTION = process.env.NEXT_PUBLIC_ENVIRONMENT === 'prod'
|
|
|
|
// NOTE(mattrossman): Temporary killswitch to disable tracing in production
|
|
export const IS_TRACING_ENABLED =
|
|
BRAINTRUST_API_KEY !== undefined && BRAINTRUST_PROJECT_ID !== undefined && !IS_PRODUCTION
|
|
|
|
if (IS_TRACING_ENABLED) {
|
|
initLogger({
|
|
apiKey: BRAINTRUST_API_KEY,
|
|
projectId: BRAINTRUST_PROJECT_ID,
|
|
})
|
|
}
|