mirror of
https://github.com/supabase/supabase.git
synced 2026-10-06 01:45:10 +03:00
Matt RossmanandAli Waseem
851cc00545
Runs 3 trials for Assistant evals in CI to reduce random variation. Locally, only 1 trial is run. Also adds `CI` to `studio#build` env in turbo.json. This env var is [automatically set by GitHub Actions](https://github.blog/changelog/2020-04-15-github-actions-sets-the-ci-environment-variable-to-true/). Compare number of trials: - [Assistant (mattrossman/ai-398-increase-trial-count-for-assistant-evals-1770305591)](https://www.braintrust.dev/app/supabase.io/p/Assistant/experiments/mattrossman%2Fai-398-increase-trial-count-for-assistant-evals-1770305591) - [Assistant (master)](https://www.braintrust.dev/app/supabase.io/p/Assistant/experiments/master-1770305906?c=mattrossman/ai-398-increase-trial-count-for-assistant-evals-1770305591) References: - https://www.braintrust.dev/docs/evaluate/run-evaluations#trials Closes AI-398 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated evaluation configuration to adjust trial counts based on CI environment * Integrated CI environment variable into build system configuration <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Ali Waseem <waseema393@gmail.com>