mirror of
https://github.com/supabase/supabase.git
synced 2026-10-05 17:35:10 +03:00
chore(studio): retry flaky unit tests in CI (#48939)
<!-- ccr-slack-attribution --> _Requested via [Slack thread](https://supabase.slack.com/archives/C063LNYJJKS/p1786454906416269?thread_ts=1786454906.416269&cid=C063LNYJJKS)_ ## I have read the [CONTRIBUTING.md](https://github.com/supabase/supabase/blob/master/CONTRIBUTING.md) file. YES ## What kind of change does this PR introduce? Chore / CI reliability. One-line config change to `apps/studio/vitest.config.ts`. ## What is the current behavior? **Before:** the `Studio Unit Tests & Build Check` workflow sometimes goes red on `master` for no reason anyone can act on. Since 2026-07-29 it failed 3 out of 82 test executions (3.7%), every time at job `test (1)`, step `Run Tests`. Every one of those three passed on a re-run with no code change: - https://github.com/supabase/supabase/actions/runs/31495760766 (`4587d177`, Aug 11) - https://github.com/supabase/supabase/actions/runs/31409667566 (`b04d1485`, Aug 10) - https://github.com/supabase/supabase/actions/runs/31203465483 (`777c02c2`, Aug 7) Each failure also posts a Slack alert to #team-frontend-alerts via `.github/workflows/studio-master-alert.yml`, so someone gets pinged, opens the run, clicks re-run, and it goes green. ## What is the new behavior? **After:** a test that fails in CI gets up to two more attempts before the job is marked failed. A genuinely broken test still fails all three attempts and still goes red. Locally nothing changes — the first failure is the result you see, so you are never waiting on retries while debugging. ## Additional context **How:** added `retry: IS_CI ? 2 : 0` to the `test` block of `apps/studio/vitest.config.ts`, with `const IS_CI = !!process.env.CI` matching the pattern already used in `e2e/studio/playwright.config.ts:51` (`retries: IS_CI ? 5 : 0`). **Known limitation — we do not know which test is flaking.** The GitHub Actions log downloads for those three runs were not retrievable, and the API only surfaces `Process completed with exit code 1`. So this treats the symptom without naming the cause. The natural follow-up is to upload a JUnit or JSON vitest report as an artifact with `if: always()`, which would name the flaking test on the next failure. That is deliberately **not** in this PR — it is a workflow change and was scoped out. One more honest caveat: per-test retry only helps if the failure is an assertion or timeout inside a test. If the real cause is a worker crash or OOM, retrying will not save the run. That is a live possibility here — the workflow sets `NODE_OPTIONS: '--max_old_space_size=3072'` with the in-repo comment "Default is 2 GB, increase to have less frequent OOM errors", which says someone has already hit memory pressure in this job. So: worth landing as a cheap reduction in false alarms, but if the 3.7% does not drop, the report artifact is the next step rather than more retries. --- _Generated by [Claude Code](https://claude.ai/code/session_01U4338RsMYAc1uGuwTFNGBD)_ Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Ali Waseem <waseema393@gmail.com>
This commit is contained in:
1 file changed
+4
@@ -9,6 +9,8 @@ import { configDefaults, defineConfig } from 'vitest/config'
|
||||
// `setupFiles` live next to the test file itself. This forces them to always resolve correctly.
|
||||
const dirname = fileURLToPath(new URL('.', import.meta.url))
|
||||
|
||||
const IS_CI = !!process.env.CI
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [
|
||||
react(),
|
||||
@@ -24,6 +26,8 @@ export default defineConfig({
|
||||
test: {
|
||||
globals: true,
|
||||
environment: 'jsdom', // TODO(kamil): This should be set per test via header in .tsx files only
|
||||
// Retry flaky tests in CI only; failures locally should surface immediately.
|
||||
retry: IS_CI ? 2 : 0,
|
||||
setupFiles: [
|
||||
resolve(dirname, './tests/setup/polyfills.ts'),
|
||||
resolve(dirname, './tests/vitestSetup.ts'),
|
||||
|
||||
Reference in new issue
Block a user